mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-07-21 16:32:12 +00:00
72 lines
4.4 KiB
Markdown
72 lines
4.4 KiB
Markdown
|
# 0.10.0 Release
|
|||
|
|
|
|||
|
|
{% note noteType="Tip" %}
|
|||
|
|
**2022, April 27th**
|
|||
|
|
|
|||
|
|
[OpenMetadata 0.10.0 Release](https://blog.open-metadata.org/openmetadata-0-10-0-release-82c4f5533c3f)
|
|||
|
|
{% /note %}
|
|||
|
|
|
|||
|
|
## Support for Database Schema
|
|||
|
|
|
|||
|
|
OpenMetadata supports databases, service name databases, and tables. We’ve added Database Schema as part of the FQN.
|
|||
|
|
For each external data source, we ingest the database, as well as the tables that are contained underneath the schemas.
|
|||
|
|
|
|||
|
|
## Support for Hard Delete
|
|||
|
|
|
|||
|
|
OpenMetadata supported soft deletions. Now, we also support the hard deletion of entities through the UI, APIs,
|
|||
|
|
and ingestion. Hard deleting an entity removes the entity and all of its relationships. This will also generate a change event.
|
|||
|
|
|
|||
|
|
## Deploy Ingestion from UI
|
|||
|
|
|
|||
|
|
OpenMetadata has refactored the service connections to simplify the ingestion jobs from both the ingestion framework
|
|||
|
|
and the UI. We now use the pydantic models automatically generated from the JSON schemas for the connection
|
|||
|
|
definition. The ‘Add Service’ form is automatically generated in the UI based on the JSON schema specifications for the
|
|||
|
|
various connectors that are supported in OpenMetadata.
|
|||
|
|
|
|||
|
|
## Download dbt Manifest Files from Amazon S3 or Google Cloud Storage
|
|||
|
|
|
|||
|
|
Previously, when ingesting the models and lineage from dbt, we passed the path of the dbt manifest and catalog files
|
|||
|
|
directly into the workflow. We’ve worked on improving the quality of life of dbt. Now, we can dynamically download
|
|||
|
|
these files from Amazon S3 or Google Cloud Storage. This way we can have any other process to connect to the dbt,
|
|||
|
|
extract the catalog, and put it into any cloud service. We just need the path name and workflow job details from the
|
|||
|
|
metadata extraction to be able to ingest metadata.
|
|||
|
|
|
|||
|
|
## JSON Schema based Connection Definition
|
|||
|
|
|
|||
|
|
Each service (database, dashboard, messaging, or pipeline service) has its own configuration specifications, with some
|
|||
|
|
unique requirements for some services. Instead of the ad hoc definitions of the source module in Python for each
|
|||
|
|
connector, we’ve worked on the full refactoring of the ingestion framework. We now use the pydantic models automatically
|
|||
|
|
generated from the JSON schemas for the connection definition.
|
|||
|
|
|
|||
|
|
## Airflow Rest APIs
|
|||
|
|
|
|||
|
|
The Airflow REST APIs have been refactored. With our API centric model, we are creating a custom airflow rest API
|
|||
|
|
directly on top of Airflow using plugins. This passes the connection information to automatically generate all the dags
|
|||
|
|
and prepares handy methods to help us test the connection to the source before creating the service.
|
|||
|
|
|
|||
|
|
## UI Changes
|
|||
|
|
|
|||
|
|
- The UI improvements are directed toward providing a consistent user experience.
|
|||
|
|
- Hard Deletion of Entities: With the support for the hard deletion of entities, we can permanently delete tables,
|
|||
|
|
topics, or services. When the entity is hard deleted, the entity and all its relationships are removed.
|
|||
|
|
This generates an ‘EntityDeleted’ change event.
|
|||
|
|
- Dynamic “Add Service” Forms: The ‘Add Service’ form is automatically generated in the UI based on the JSON
|
|||
|
|
schema specifications for the various connectors that are supported in OpenMetadata.
|
|||
|
|
- UI Support for Database Schema as part of FQN: The database schema has been introduced in the 0.10 release. All the
|
|||
|
|
entity pages now support Database Schema in the UI.
|
|||
|
|
- Lineage Editor: Improvements have been made to the lineage editor.
|
|||
|
|
- Teams: While signing up in OpenMetadata, the teams with restricted access are hidden and only the joinable teams are displayed.
|
|||
|
|
- Team Owner: An Owner field has been added to the Team entity. Only team owners can update the teams.
|
|||
|
|
- Activity Feeds: The Activity Feeds UI supports infinite scrolling.
|
|||
|
|
- Add User: A user can be added from the Users page.
|
|||
|
|
|
|||
|
|
## Security Changes
|
|||
|
|
|
|||
|
|
- **Support Refresh Tokens for Auth0 and Okta SSO**: The JWT tokens generated by the SSO providers expire by default
|
|||
|
|
in about an hour, making the user re-login often. In this release, we’ve added support for refresh tokens for Auth0
|
|||
|
|
and Okta SSO. The tokens are refreshed silently behind the scenes to provide an uninterrupted user experience.
|
|||
|
|
In future releases, we’ll continue to stabilize authentication and add refresh tokens for the other SSO providers.
|
|||
|
|
- **Custom OIDC SSO**: OpenMetadata now supports integration with your custom-built OIDC SSO for authentication.
|
|||
|
|
This is supported both on the front end for user authentication and on the ingestion side.
|
|||
|
|
- **Azure SSO**: Support has been added for Azure SSO on Airflow.
|