yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-09-01 05:03:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Changes to 1.3 Docs (#15152)

Browse Source
This commit is contained in:
Shilpa Vernekar 2024-02-12 18:41:11 +05:30 committed by GitHub
parent 95bb221025
commit ad23c76119
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
10 changed files with 292 additions and 125 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

145
openmetadata-docs/content/partials/v1.3/releases/latest.md
View File

@ -1,4 +1,4 @@
# 1.2.0 Release 🎉 # 1.3.0 Release 🎉
{% inlineCalloutContainer %} {% inlineCalloutContainer %}
{% inlineCallout {% inlineCallout
@ -6,78 +6,99 @@ color="violet-70"
icon="celebration" icon="celebration"
bold="Upgrade OpenMetadata" bold="Upgrade OpenMetadata"
href="/deployment/upgrade" %} href="/deployment/upgrade" %}
Learn how to upgrade your OpenMetadata instance to 1.2.0! Learn how to upgrade your OpenMetadata instance to 1.3.0!
{% /inlineCallout %} {% /inlineCallout %}
{% /inlineCalloutContainer %} {% /inlineCalloutContainer %}
You can find the GitHub release [here](https://github.com/open-metadata/OpenMetadata/releases/tag/1.2.0-release). You can find the GitHub release [here](https://github.com/open-metadata/OpenMetadata/releases/tag/1.3.0-release).
## Domains and Data Products
- Added support for Domains and Data Products.
- Assets can be added to a Domain, and users can scope their discovery experience to one Domain.
- Assets can also be added as Data Products in a Domain.
## Search Index
- Elasticsearch or Opensearch connectors can now bring in the search index metadata into OpenMetadata.
- The connector will populate the index’s mapping, settings, and sample data.
## Stored Procedures
- Added support for Stored Procedures.
- Snowflake, Redshift, and BigQuery connectors are updated to bring stored procedure metadata into OpenMetadata.
- The metadata workflow will bring the Stored Procedures and parse their executions to extract lineage information.
## Glossary Approval Workflow & Glossary Styling
- Introduced a glossary approval workflow. An approval workflow is created if Reviewers are added to a glossary.
- A task is added for reviewers to approve or reject the glossary term. The terms will show up in Draft status.
- Only the reviewers can approve or reject the term.
- Conversations are supported to discuss further about the terms.
- If no reviewer is added, then the glossary terms are approved by default.
- Introduced styling for glossary terms. Now you can add icons and color code the glossary terms for easy identification.
- Color coding helps to visually differentiate and identify the data assets, when glossary terms are added to them.
## OpenMetadata Browser Extension
- Updated the Chrome browser extension for OpenMetadata with the new UI.
- Added support for Databases, Database Schemas, Tables, Dashboards, Charts, Pipelines, and Topics.
## Build Automation Applications
{% youtube videoId="pUS9-RevqsU" start="0:00" end="0:57" width="560px" height="315px" /%}
- Added Applications into OpenMetadata, giving users a unique view of processes that can be scheduled and run in the platform.
- Search Indexing and Data Insights Report have been converted into Applications.
- UI displays all the available applications, which Admins can add or schedule.
- We will continue to add new Applications in upcoming releases.
## Lineage ## Lineage
- Performance improvements made for lineage based on the new release of SQLfluff. - Revamped the lineage UI for an intuitive and comprehensive view of data flow and transformations.
- Added support for `UPDATE … FROM Snowflake` queries - Organized nodes for better visibility with pagination support.
- Added column-level lineage support for `SELECT *` queries - Improved the display of circular dependencies.
- Nodes display the service icons, highlight dbt models, and show Data Quality results.
- Lineage can be filtered to search by Ownership, Domain, Service, Service Type, Tier, and Classification Tags.
- Supports search by Column and traces lineage even when the columns are renamed.
- Enhanced user control with collapsible sub-graphs.
- Supports editing the SQL queries for lineage edges from the UI.
- Performance improvements for faster load of large graphs.
## Connectors ## Data Observability Alerts
- Greenplum connector is now supported. - Data observability alerts have been distinguished from other general-purpose notifications, making it easy to get to the crucial alerts quickly.
- Couchbase connector is now supported. - Sends alerts for schema changes and test case failures for the data assets that you follow.
- Azure Data Lake Storage is now supported. (Collate) - The overall flow has been simplified to let you easily create alerts for schema changes in your data.
- You can now get Data Quality alerts for specific Test Suites.
- Users will be alerted for all the changes to the data assets that they own.
## Customizable Landing Page ## Incident Manager
- Introduced Incidents Manager to improve the data quality resolution flow.
{% youtube videoId="Y-5cPQgzNdo" start="0:00" end="2:08" width="560px" height="315px" /%} - Incidents Manager summarizes all the test case results with information about the failure severity and resolution flow.
- Supports assigning a resolution task to the users in OpenMetadata.
- Admins can create Personas to group individuals in their company, such as Data Engineers, Data Stewards, or Data Scientists. - Tasks are created when a data quality test has been assigned to an Assignee or a Reviewer.
- Admins can customize the landing page for each Persona with a set of supported widgets: Activity Feed, Announcements, Knowledge Center, etc. - Resolved test failure also displays the comments posted on the resolution.
- We will add support for more widgets in upcoming releases. - The Resolved Tab displays information on the Test case name, Execution date, Reason, Comments, and information on who Resolved the issue.
## Knowledge Center (Collate) ## Knowledge Center (Collate)
- Supports hierarchical pages to structure the articles.
- You can easily associate knowledge articles with data assets.
- The data assets page displays the related articles.
- The block editor supports callouts to add notes, warnings, tables, and task lists.
- Quicklinks are no longer separate pages; they redirect to external links.
- Data assets can be associated with Quicklinks.
- Added Search support for Knowledge articles to filter by Owner or Tags.
- Supports preview for articles and Quicklinks.
{% youtube videoId="DfOgeZ9f7no" start="0:00" end="3:04" width="560px" height="315px" /%} ## Custom Metrics for Profiler
- Supports custom metrics for the data profiler with custom SQL to keep track of your business metrics.
- Custom metrics can be created at Table and Column levels.
- Backend APIs support creating, editing, and listing knowledge articles (with external links). ## Profiler and Data Quality
- Knowledge articles and links can be associated with a Domain, Team, or an Entity. - The Profiler has been improved to support sample data ingestion without computing other metrics.
- UI support to build a Knowledge Center and expand the documentation of your company. - Admins can configure the profiler to fetch up to 10,000 rows of sample data.
- Sample data can be stored in S3 buckets.
- Refined the default time range on the test case results page, adjusting it from the Last 3 days to the Last 30 days for a more encompassing view.
## Cost Analysis Report (Collate) ## Connectors
- New Google Cloud Storage for storage services. (Collate)
- New Alation connector to migrate metadata into Collate. (Collate)
- New Iceberg, SAS Viya, and Doris connectors.
- Introduced the Spark Lineage Agent to extract metadata and end-to-end lineage from Spark jobs.
- MSSQL and Oracle now support Stored Procedures.
- We now exclude system indices from the Elasticsearch connector by default.
- Added support for DB2 IBM I Series.
- Pipeline services now get owner information.
- Performance improvements for the Tableau Connector.
- We now support metadata tag extraction from Databricks.
- Supports the attribute Table Owner for metadata ingestion from Postgres.
- We now extract table descriptions when ingesting metadata from Salesforce.
{% youtube videoId="KI58oBHxTOU" start="0:00" end="0:33" width="560px" height="315px" /%} ## Glossary
- Supports soft delete for the default glossaries in OpenMetadata.
- Supports the creation of tasks to request tags or a description.
- Only the Owner can edit the Glossary term.
- Version history displays the Username instead of the User ID.
- The Usage Workflow will now also track how tables are Accessed and Updated. ## Localization
- This information will be used in the Data Insights workflow to show the evolution of your used and unused assets and compare them by size. - Now supports RTL UI for the Hebrew language.
- Support has been added for Snowflake, and we will continue to add more sources in upcoming releases. - New Dutch language translation.
## Settings UI
- The Settings page UI has been revamped.
## Data Insights
- Cost Analysis expanded to support BigQuery & Redshift. (Collate)
- Improved the Data Insights Report sent via email.
## Other Changes
- Announcements can be notified over email, Slack, or Teams.
- Alerts are sent to a user when they are mentioned in a task or activity feed.
- We have improved the display of search results for column matches. When searching for columns, the matched results will be displayed and highlighted in the Preview pane.
- Table Type filter has been added in the Advanced Search, so that users can exclude the temporary or staging tables from search.
- Now it is easy to filter the Data assets without a Owner.
- Database and Schema were added to the Explore menu to enhance data discovery.
- Custom properties are displayed on the right of the data asset details page.
- We now display the Domain on the Users page.
- Supports the sorting of data assets by popularity based on the number of followers and thumbs up as signals.
- OpenMetadata can now handle metric history for ML models.
- When configuring the Email settings, the Username and Password fields can be left blank.
- We now support a test email button on the Email SMTP page.

13
openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/chrome-ext.md Normal file
View File

@ -0,0 +1,13 @@
---
title: OpenMetadata Google Chrome Extension
slug: /how-to-guides/guide-for-data-users/chrome-ext
---
# OpenMetadata Google Chrome Extension
OpenMetadata’s Chrome Browser Extension bridges the gap between users and metadata. Visit the Chrome webstore and search for OpenMetadata to add the extension to your Chrome browser with just a few clicks. You can find the Chrome browser extension [here](https://chromewebstore.google.com/detail/openmetadata/pakbbdhbbiclnceabdmnghamabjloofc?hl=en&authuser=0&pli=1)
Once installed, pin the extension for easy accessibility. Simply click on the OpenMetadata icon in your Chrome browser to start using the extension. Add the base URL for your OpenMetadata instance, and connect by signing in. Once signed in, the OpenMetadata Chrome extension will display your Activity Feeds from My Data page. While working on a third party tool, which is integrated with OpenMetadata, you can simply click on the extension to view the associated metadata to access details about ownership, description, associated tags, glossary terms, schema, lineage, custom properties, and more.
Watch the video to learn how you can access all the metadata you need, directly into your active workspace.
{% youtube videoId="ZQckSIXAA6k" start="0:00" end="1:37" width="560px" height="315px" /%}

2
openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/follow-data-asset.md
View File

@ -7,6 +7,8 @@ slug: /how-to-guides/guide-for-data-users/follow-data-asset
Users can get timely information about all the **activities**, **announcements**, and **feeds** related to a data asset by following those assets. The owners of the data assets will receive all the updates by default. The follow option can be used for the assets that you do not own. Users can get timely information about all the **activities**, **announcements**, and **feeds** related to a data asset by following those assets. The owners of the data assets will receive all the updates by default. The follow option can be used for the assets that you do not own.
{% youtube videoId="xKg596qN7LA" start="0:00" end="1:52" width="560px" height="315px" /%}
To follow a data asset, To follow a data asset,
- Navigate to the **Explore** page. - Navigate to the **Explore** page.
- Select the relevant type of data asset (Tables, Topics, Dashboards, Pipelines, ML Models, and Containers). - Select the relevant type of data asset (Tables, Topics, Dashboards, Pipelines, ML Models, and Containers).

2
openmetadata-docs/content/v1.2.x/menu.md
View File

@ -255,6 +255,8 @@ site_menu:
url: /how-to-guides/guide-for-data-users/versions url: /how-to-guides/guide-for-data-users/versions
- category: How-to Guides / Guide for Data Users / How to Delete a Data Asset - category: How-to Guides / Guide for Data Users / How to Delete a Data Asset
url: /how-to-guides/guide-for-data-users/delete url: /how-to-guides/guide-for-data-users/delete
- category: How-to Guides / Guide for Data Users / OpenMetadata Google Chrome Extension
url: /how-to-guides/guide-for-data-users/chrome-ext
- category: How-to Guides / Data Discovery - category: How-to Guides / Data Discovery
url: /how-to-guides/data-discovery url: /how-to-guides/data-discovery

13
openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/chrome-ext.md Normal file
View File

@ -0,0 +1,13 @@
---
title: OpenMetadata Google Chrome Extension
slug: /how-to-guides/guide-for-data-users/chrome-ext
---
# OpenMetadata Google Chrome Extension
OpenMetadata’s Chrome Browser Extension bridges the gap between users and metadata. Visit the Chrome webstore and search for OpenMetadata to add the extension to your Chrome browser with just a few clicks. You can find the Chrome browser extension [here](https://chromewebstore.google.com/detail/openmetadata/pakbbdhbbiclnceabdmnghamabjloofc?hl=en&authuser=0&pli=1)
Once installed, pin the extension for easy accessibility. Simply click on the OpenMetadata icon in your Chrome browser to start using the extension. Add the base URL for your OpenMetadata instance, and connect by signing in. Once signed in, the OpenMetadata Chrome extension will display your Activity Feeds from My Data page. While working on a third party tool, which is integrated with OpenMetadata, you can simply click on the extension to view the associated metadata to access details about ownership, description, associated tags, glossary terms, schema, lineage, custom properties, and more.
Watch the video to learn how you can access all the metadata you need, directly into your active workspace.
{% youtube videoId="ZQckSIXAA6k" start="0:00" end="1:37" width="560px" height="315px" /%}

2
openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/follow-data-asset.md
View File

@ -7,6 +7,8 @@ slug: /how-to-guides/guide-for-data-users/follow-data-asset
Users can get timely information about all the **activities**, **announcements**, and **feeds** related to a data asset by following those assets. The owners of the data assets will receive all the updates by default. The follow option can be used for the assets that you do not own. Users can get timely information about all the **activities**, **announcements**, and **feeds** related to a data asset by following those assets. The owners of the data assets will receive all the updates by default. The follow option can be used for the assets that you do not own.
{% youtube videoId="xKg596qN7LA" start="0:00" end="1:52" width="560px" height="315px" /%}
To follow a data asset, To follow a data asset,
- Navigate to the **Explore** page. - Navigate to the **Explore** page.
- Select the relevant type of data asset (Tables, Topics, Dashboards, Pipelines, ML Models, and Containers). - Select the relevant type of data asset (Tables, Topics, Dashboards, Pipelines, ML Models, and Containers).

44
openmetadata-docs/content/v1.3.x/menu.md
View File

@ -259,6 +259,8 @@ site_menu:
url: /how-to-guides/guide-for-data-users/versions url: /how-to-guides/guide-for-data-users/versions
- category: How-to Guides / Guide for Data Users / How to Delete a Data Asset - category: How-to Guides / Guide for Data Users / How to Delete a Data Asset
url: /how-to-guides/guide-for-data-users/delete url: /how-to-guides/guide-for-data-users/delete
- category: How-to Guides / Guide for Data Users / OpenMetadata Google Chrome Extension
url: /how-to-guides/guide-for-data-users/chrome-ext
- category: How-to Guides / Data Discovery - category: How-to Guides / Data Discovery
url: /how-to-guides/data-discovery url: /how-to-guides/data-discovery
@ -370,6 +372,48 @@ site_menu:
url: /releases/roadmap url: /releases/roadmap
- category: Releases / All Releases - category: Releases / All Releases
url: /releases/all-releases url: /releases/all-releases
- category: Releases / All Releases / 1.2.0 Release
url: /releases/all-releases/#120-release
- category: Releases / All Releases / 1.1.2 Release
url: /releases/all-releases/#112-release
- category: Releases / All Releases / 1.1.1 Release
url: /releases/all-releases/#111-release
- category: Releases / All Releases / 1.1.0 Release
url: /releases/all-releases/#110-release
- category: Releases / All Releases / 1.0.0 Release
url: /releases/all-releases/#100-release
- category: Releases / All Releases / 0.13.3 Release
url: /releases/all-releases/#0133-release
- category: Releases / All Releases / 0.13.2 Release
url: /releases/all-releases/#0132-release
- category: Releases / All Releases / 0.13.1 Release
url: /releases/all-releases/#0131-release
- category: Releases / All Releases / 0.13.0 Release
url: /releases/all-releases/#0130-release
- category: Releases / All Releases / 0.12.3 Release
url: /releases/all-releases/#0123-release
- category: Releases / All Releases / 0.12.2 Release
url: /releases/all-releases/#0122-release
- category: Releases / All Releases / 0.12.1 Release
url: /releases/all-releases/#0121-release
- category: Releases / All Releases / 0.12.0 Release
url: /releases/all-releases/#0120-release
- category: Releases / All Releases / 0.10.1 Release
url: /releases/all-releases/#0101-release
- category: Releases / All Releases / 0.10.0 Release
url: /releases/all-releases/#0100-release
- category: Releases / All Releases / 0.9.0 Release
url: /releases/all-releases/#090-release
- category: Releases / All Releases / 0.8.0 Release
url: /releases/all-releases/#080-release
- category: Releases / All Releases / 0.7.0 Release
url: /releases/all-releases/#070-release
- category: Releases / All Releases / 0.6.0 Release
url: /releases/all-releases/#060-release
- category: Releases / All Releases / 0.5.0 Release
url: /releases/all-releases/#050-release
- category: Releases / All Releases / 0.4.0 Release
url: /releases/all-releases/#040-release
- category: Connectors - category: Connectors
url: /connectors url: /connectors

188
openmetadata-docs/content/v1.3.x/releases/releases/index.md
View File

@ -14,7 +14,79 @@ version. To see what's coming in next releases, please check our [Roadmap](/rele
{% partial file="/v1.3/releases/latest.md" /%} {% partial file="/v1.3/releases/latest.md" /%}
# 1.1.2 Release 🎉 # 1.2.0 Release
## Domains and Data Products
- Added support for Domains and Data Products.
- Assets can be added to a Domain, and users can scope their discovery experience to one Domain.
- Assets can also be added as Data Products in a Domain.
## Search Index
- Elasticsearch or Opensearch connectors can now bring in the search index metadata into OpenMetadata.
- The connector will populate the index’s mapping, settings, and sample data.
## Stored Procedures
- Added support for Stored Procedures.
- Snowflake, Redshift, and BigQuery connectors are updated to bring stored procedure metadata into OpenMetadata.
- The metadata workflow will bring the Stored Procedures and parse their executions to extract lineage information.
## Glossary Approval Workflow & Glossary Styling
- Introduced a glossary approval workflow. An approval workflow is created if Reviewers are added to a glossary.
- A task is added for reviewers to approve or reject the glossary term. The terms will show up in Draft status.
- Only the reviewers can approve or reject the term.
- Conversations are supported to discuss further about the terms.
- If no reviewer is added, then the glossary terms are approved by default.
- Introduced styling for glossary terms. Now you can add icons and color code the glossary terms for easy identification.
- Color coding helps to visually differentiate and identify the data assets, when glossary terms are added to them.
## OpenMetadata Browser Extension
- Updated the Chrome browser extension for OpenMetadata with the new UI.
- Added support for Databases, Database Schemas, Tables, Dashboards, Charts, Pipelines, and Topics.
## Build Automation Applications
{% youtube videoId="pUS9-RevqsU" start="0:00" end="0:57" width="560px" height="315px" /%}
- Added Applications into OpenMetadata, giving users a unique view of processes that can be scheduled and run in the platform.
- Search Indexing and Data Insights Report have been converted into Applications.
- UI displays all the available applications, which Admins can add or schedule.
- We will continue to add new Applications in upcoming releases.
## Lineage
- Performance improvements made for lineage based on the new release of SQLfluff.
- Added support for `UPDATE … FROM Snowflake` queries
- Added column-level lineage support for `SELECT *` queries
## Connectors
- Greenplum connector is now supported.
- Couchbase connector is now supported.
- Azure Data Lake Storage is now supported. (Collate)
## Customizable Landing Page
{% youtube videoId="Y-5cPQgzNdo" start="0:00" end="2:08" width="560px" height="315px" /%}
- Admins can create Personas to group individuals in their company, such as Data Engineers, Data Stewards, or Data Scientists.
- Admins can customize the landing page for each Persona with a set of supported widgets: Activity Feed, Announcements, Knowledge Center, etc.
- We will add support for more widgets in upcoming releases.
## Knowledge Center (Collate)
{% youtube videoId="DfOgeZ9f7no" start="0:00" end="3:04" width="560px" height="315px" /%}
- Backend APIs support creating, editing, and listing knowledge articles (with external links).
- Knowledge articles and links can be associated with a Domain, Team, or an Entity.
- UI support to build a Knowledge Center and expand the documentation of your company.
## Cost Analysis Report (Collate)
{% youtube videoId="KI58oBHxTOU" start="0:00" end="0:33" width="560px" height="315px" /%}
- The Usage Workflow will now also track how tables are Accessed and Updated.
- This information will be used in the Data Insights workflow to show the evolution of your used and unused assets and compare them by size.
- Support has been added for Snowflake, and we will continue to add more sources in upcoming releases.
# 1.1.2 Release
## Data Quality ## Data Quality
- Added support for Postgres version 11.19. - Added support for Postgres version 11.19.
@ -43,7 +115,7 @@ version. To see what's coming in next releases, please check our [Roadmap](/rele
- Deprecated Flyway and using native migrations. - Deprecated Flyway and using native migrations.
- Improved Test Suite UI performance. - Improved Test Suite UI performance.
# 1.1.1 Release 🎉 # 1.1.1 Release
## UI Improvements ## UI Improvements
@ -76,7 +148,7 @@ version. To see what's coming in next releases, please check our [Roadmap](/rele
- Test Suite Improvements - Test Suite Improvements
# 1.1 Release 🎉 # 1.1.0 Release
{% inlineCalloutContainer %} {% inlineCalloutContainer %}
{% inlineCallout {% inlineCallout
@ -132,7 +204,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1!
- Added AWS OpenSearch client-specific support. This allows us to update the Elasticsearch version support up to 7.16. - Added AWS OpenSearch client-specific support. This allows us to update the Elasticsearch version support up to 7.16.
# 1.0 Release 🎉 # 1.0.0 Release
## APIs & Schema ## APIs & Schema
- **Stabilized** and improved the Schemas and APIs. - **Stabilized** and improved the Schemas and APIs.
@ -196,7 +268,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1!
- **Entity names** support apostrophes. - **Entity names** support apostrophes.
- The **Summary panel** has been improved to be consistent across the UI. - The **Summary panel** has been improved to be consistent across the UI.
# [0.13.3 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.3-release) - March 30th 2023 🎉 # 0.13.3 Release
## Ingestion Framework ## Ingestion Framework
- Datalake Avro & Json, JsonZip support - Datalake Avro & Json, JsonZip support
@ -235,7 +307,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1!
## Data Quality ## Data Quality
- Fix column values between test - Fix column values between test
# [0.13.2 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.2-release) - Latest - Jan 30th 2023 🎉 # 0.13.2 Release
## Improved SQL Lineage ## Improved SQL Lineage
- We have collaborated with the [sqllineage](https://github.com/reata/sqllineage) and [sqlfluff](https://www.sqlfluff.com/) communities - We have collaborated with the [sqllineage](https://github.com/reata/sqllineage) and [sqlfluff](https://www.sqlfluff.com/) communities
@ -269,7 +341,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1!
- Added support for Postgres Foreign Tables - Added support for Postgres Foreign Tables
- Added support for Datalake profiler row-based sampling - Added support for Datalake profiler row-based sampling
# [0.13.1 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.1-release) - Dec 20th 2022** 🎉 # 0.13.1 Release
## Profiler and Data Quality ## Profiler and Data Quality
- Freshness Metric has been introduced. Data freshness shows DML operations performed against a table and the number of rows affected. All this is displayed within the data profiler with filterable graphs. This is currently supported for BigQuery, Snowflake, and Redshift. - Freshness Metric has been introduced. Data freshness shows DML operations performed against a table and the number of rows affected. All this is displayed within the data profiler with filterable graphs. This is currently supported for BigQuery, Snowflake, and Redshift.
- Support has been added for data quality tests on Data Lake. - Support has been added for data quality tests on Data Lake.
@ -293,7 +365,7 @@ The logic for Notification Support has been improved. Users can define Alerts ba
- Improvements have been made to Advanced Search. Now, when a filter is applied, the details of the filter selected are displayed for clarity. - Improvements have been made to Advanced Search. Now, when a filter is applied, the details of the filter selected are displayed for clarity.
- On the Explore page UI, the Side Preview is now available for all data assets. Previously it was only displayed for tables. - On the Explore page UI, the Side Preview is now available for all data assets. Previously it was only displayed for tables.
# [0.13.0 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.0-release) - Dec 8th 2022 🎉 # 0.13.0 Release
{% youtube videoId="oNbMnTW5AkE" start="0:00" end="7:51" width="560px" height="315px" /%} {% youtube videoId="oNbMnTW5AkE" start="0:00" end="7:51" width="560px" height="315px" /%}
## Data Insights and KPI ## Data Insights and KPI
@ -324,19 +396,19 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan
- Soft deleted entities can be restored. Currently, only the ML Models are not supported. - Soft deleted entities can be restored. Currently, only the ML Models are not supported.
- Soft deleted teams can be restored. When restoring a soft deleted parent team, the child teams will not be restored by default. - Soft deleted teams can be restored. When restoring a soft deleted parent team, the child teams will not be restored by default.
# [0.12.3 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.3-release) - Nov 18th 2022 🎉 # 0.12.3 Release
## Bug Fixes ## Bug Fixes
- User suggestion index mapping - User suggestion index mapping
- Tag and Glossary terms caching - Tag and Glossary terms caching
# [0.12.2 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.2-release) - Oct 20th 2022 🎉 # 0.12.2 Release
## Ingestion ## Ingestion
- Databricks lineage - Databricks lineage
- Added support for Airflow version 2.2.2 as a workflow scheduler - Added support for Airflow version 2.2.2 as a workflow scheduler
## Bug Fixes ## Bug Fixes
- Support same table across different databases for the profiler - Support same table across different databases for the profiler
# [0.12.1 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.1-release) - Oct 3rd 2022 🎉 # 0.12.1 Release
## Basic Authentication ## Basic Authentication
- User/Password signup and login - User/Password signup and login
@ -363,15 +435,13 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan
- Around 136 Features/Improvements/Tests made it into 0.12.1 release - Around 136 Features/Improvements/Tests made it into 0.12.1 release
# [0.12.0 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.0-release) - Sept 7th 2022 🎉 # 0.12.0 Release
You can read the Release Blog [here](https://blog.open-metadata.org/openmetadata-0-12-0-release-1ac059700de4) You can read the Release Blog [here](https://blog.open-metadata.org/openmetadata-0-12-0-release-1ac059700de4)
or watch an awesome video showing the new features! or watch an awesome video showing the new features!
{% youtube videoId="tv3pyCLcJfQ" start="0:00" end="17:04" width="560px" height="315px" /%} {% youtube videoId="tv3pyCLcJfQ" start="0:00" end="17:04" width="560px" height="315px" /%}
## Team Hierarchy ## Team Hierarchy
Prior releases supported a flat hierarchy of just Teams and Users. In 0.12, support has been added for the entire organizational hierarchy with Business Unit, Division, Department, and Groups. An organization from small to very large can now be modeled in OpenMetadata with this feature. Prior releases supported a flat hierarchy of just Teams and Users. In 0.12, support has been added for the entire organizational hierarchy with Business Unit, Division, Department, and Groups. An organization from small to very large can now be modeled in OpenMetadata with this feature.
@ -442,7 +512,7 @@ When creating a new user or when a user is registering for the first time, the d
UI improvements have been made on the Schema, Service, and Database details pages. UI improvements have been made on the Schema, Service, and Database details pages.
Manage Tab has been replaced with the manage button on the UI. Manage Tab has been replaced with the manage button on the UI.
## 0.10.1 Release - May 17th, 2022 # 0.10.1 Release
- Support for Postgres as OpenMetadata Store [#4601](https://github.com/open-metadata/OpenMetadata/issues/4601) - Support for Postgres as OpenMetadata Store [#4601](https://github.com/open-metadata/OpenMetadata/issues/4601)
- UI Improvements in 0.10.1 Release [#4600](https://github.com/open-metadata/OpenMetadata/issues/4600) - UI Improvements in 0.10.1 Release [#4600](https://github.com/open-metadata/OpenMetadata/issues/4600)
@ -450,26 +520,26 @@ Manage Tab has been replaced with the manage button on the UI.
- UI Ingestion Improvements - Support for Dashboards & Messaging Services [#4843](https://github.com/open-metadata/OpenMetadata/issues/4843) - UI Ingestion Improvements - Support for Dashboards & Messaging Services [#4843](https://github.com/open-metadata/OpenMetadata/issues/4843)
- Security: Fix Azure SSO and support refresh tokens in [#4989](https://github.com/open-metadata/OpenMetadata/issues/4989) - Security: Fix Azure SSO and support refresh tokens in [#4989](https://github.com/open-metadata/OpenMetadata/issues/4989)
## 0.10.0 Release - Apr 27th, 2022 # 0.10.0 Release
### Support for Database Schema ## Support for Database Schema
OpenMetadata supports databases, service name databases, and tables. We’ve added Database Schema as part of the FQN. OpenMetadata supports databases, service name databases, and tables. We’ve added Database Schema as part of the FQN.
For each external data source, we ingest the database, as well as the tables that are contained underneath the schemas. For each external data source, we ingest the database, as well as the tables that are contained underneath the schemas.
### Support for Hard Delete ## Support for Hard Delete
OpenMetadata supported soft deletions. Now, we also support the hard deletion of entities through the UI, APIs, OpenMetadata supported soft deletions. Now, we also support the hard deletion of entities through the UI, APIs,
and ingestion. Hard deleting an entity removes the entity and all of its relationships. This will also generate a change event. and ingestion. Hard deleting an entity removes the entity and all of its relationships. This will also generate a change event.
### Deploy Ingestion from UI ## Deploy Ingestion from UI
OpenMetadata has refactored the service connections to simplify the ingestion jobs from both the ingestion framework OpenMetadata has refactored the service connections to simplify the ingestion jobs from both the ingestion framework
and the UI. We now use the pydantic models automatically generated from the JSON schemas for the connection and the UI. We now use the pydantic models automatically generated from the JSON schemas for the connection
definition. The ‘Add Service’ form is automatically generated in the UI based on the JSON schema specifications for the definition. The ‘Add Service’ form is automatically generated in the UI based on the JSON schema specifications for the
various connectors that are supported in OpenMetadata. various connectors that are supported in OpenMetadata.
### Download dbt Manifest Files from Amazon S3 or Google Cloud Storage ## Download dbt Manifest Files from Amazon S3 or Google Cloud Storage
Previously, when ingesting the models and lineage from dbt, we passed the path of the dbt manifest and catalog files Previously, when ingesting the models and lineage from dbt, we passed the path of the dbt manifest and catalog files
directly into the workflow. We’ve worked on improving the quality of life of dbt. Now, we can dynamically download directly into the workflow. We’ve worked on improving the quality of life of dbt. Now, we can dynamically download
@ -477,20 +547,20 @@ these files from Amazon S3 or Google Cloud Storage. This way we can have any oth
extract the catalog, and put it into any cloud service. We just need the path name and workflow job details from the extract the catalog, and put it into any cloud service. We just need the path name and workflow job details from the
metadata extraction to be able to ingest metadata. metadata extraction to be able to ingest metadata.
### JSON Schema based Connection Definition ## JSON Schema based Connection Definition
Each service (database, dashboard, messaging, or pipeline service) has its own configuration specifications, with some Each service (database, dashboard, messaging, or pipeline service) has its own configuration specifications, with some
unique requirements for some services. Instead of the ad hoc definitions of the source module in Python for each unique requirements for some services. Instead of the ad hoc definitions of the source module in Python for each
connector, we’ve worked on the full refactoring of the ingestion framework. We now use the pydantic models automatically connector, we’ve worked on the full refactoring of the ingestion framework. We now use the pydantic models automatically
generated from the JSON schemas for the connection definition. generated from the JSON schemas for the connection definition.
### Airflow Rest APIs ## Airflow Rest APIs
The Airflow REST APIs have been refactored. With our API centric model, we are creating a custom airflow rest API The Airflow REST APIs have been refactored. With our API centric model, we are creating a custom airflow rest API
directly on top of Airflow using plugins. This passes the connection information to automatically generate all the dags directly on top of Airflow using plugins. This passes the connection information to automatically generate all the dags
and prepares handy methods to help us test the connection to the source before creating the service. and prepares handy methods to help us test the connection to the source before creating the service.
### UI Changes ## UI Changes
- The UI improvements are directed toward providing a consistent user experience. - The UI improvements are directed toward providing a consistent user experience.
- Hard Deletion of Entities: With the support for the hard deletion of entities, we can permanently delete tables, - Hard Deletion of Entities: With the support for the hard deletion of entities, we can permanently delete tables,
@ -506,7 +576,7 @@ and prepares handy methods to help us test the connection to the source before c
- Activity Feeds: The Activity Feeds UI supports infinite scrolling. - Activity Feeds: The Activity Feeds UI supports infinite scrolling.
- Add User: A user can be added from the Users page. - Add User: A user can be added from the Users page.
### Security Changes ## Security Changes
- **Support Refresh Tokens for Auth0 and Okta SSO**: The JWT tokens generated by the SSO providers expire by default - **Support Refresh Tokens for Auth0 and Okta SSO**: The JWT tokens generated by the SSO providers expire by default
in about an hour, making the user re-login often. In this release, we’ve added support for refresh tokens for Auth0 in about an hour, making the user re-login often. In this release, we’ve added support for refresh tokens for Auth0
and Okta SSO. The tokens are refreshed silently behind the scenes to provide an uninterrupted user experience. and Okta SSO. The tokens are refreshed silently behind the scenes to provide an uninterrupted user experience.
@ -515,9 +585,9 @@ and prepares handy methods to help us test the connection to the source before c
This is supported both on the front end for user authentication and on the ingestion side. This is supported both on the front end for user authentication and on the ingestion side.
- **Azure SSO**: Support has been added for Azure SSO on Airflow. - **Azure SSO**: Support has been added for Azure SSO on Airflow.
## 0.9.0 - March 10th, 2022 # 0.9.0 Release
### Collaboration ## Collaboration
- Conversations in the main feed. - Conversations in the main feed.
- Users can ask each other questions, add suggestions and replies. - Users can ask each other questions, add suggestions and replies.
@ -525,12 +595,12 @@ and prepares handy methods to help us test the connection to the source before c
- Glossary. - Glossary.
- Table details - Click through on usage to see who or what services are using it, what queries are pulling from it. - Table details - Click through on usage to see who or what services are using it, what queries are pulling from it.
### Data Quality ## Data Quality
- Ability to create and monitor the test cases. - Ability to create and monitor the test cases.
- Data Quality Tests support with Json Schemas and APIs. - Data Quality Tests support with Json Schemas and APIs.
- UI Integration to enable user to write tests and run them on Airflow. - UI Integration to enable user to write tests and run them on Airflow.
### Glossary ## Glossary
- Glossaries are a Controlled Vocabulary in an organization used to define the concepts and terminologies specific to a - Glossaries are a Controlled Vocabulary in an organization used to define the concepts and terminologies specific to a
particular domain. particular domain.
@ -538,7 +608,7 @@ and prepares handy methods to help us test the connection to the source before c
- UI support to add Glossary and Glossary Terms. - UI support to add Glossary and Glossary Terms.
- Support for using Glossary terms to annotate Entities and Search using Glossary Terms. - Support for using Glossary terms to annotate Entities and Search using Glossary Terms.
### Connectors ## Connectors
- Apache Iceberg - Apache Iceberg
- Azure SQL - Azure SQL
- Clickhouse - Clickhouse
@ -554,52 +624,52 @@ and prepares handy methods to help us test the connection to the source before c
- Apache Atlas ,Import Metadata from Apache Atlas into OpenMetadata - Apache Atlas ,Import Metadata from Apache Atlas into OpenMetadata
- Amundsen, Import Metadata from Amundsen into OpenMetadata - Amundsen, Import Metadata from Amundsen into OpenMetadata
### Lineage ## Lineage
- DataSource SQL Parsing support to extract Lineage - DataSource SQL Parsing support to extract Lineage
- View Lineage support - View Lineage support
### Pipeline ## Pipeline
- Capture pipeline status as it happens - Capture pipeline status as it happens
### Security ## Security
- Security policies through the UI. - Security policies through the UI.
- Configuration personas and authorization based on policies. - Configuration personas and authorization based on policies.
- AWS SSO support. - AWS SSO support.
## 0.8 Release - Jan 22nd, 2022 # 0.8.0 Release
### Access Control Policies ## Access Control Policies
- Design of Access Control Policies. - Design of Access Control Policies.
- Provide Role based access control with community feedback. - Provide Role based access control with community feedback.
### Eventing Webhook ## Eventing Webhook
- Register webhooks to get metadata event notifications. - Register webhooks to get metadata event notifications.
- Metadata Change Event integration into Slack and framework for integration into other services such as - Metadata Change Event integration into Slack and framework for integration into other services such as
Kafka or other Notification frameworks Kafka or other Notification frameworks
### Connectors ## Connectors
- Delta Lake - Delta Lake
- Iceberg - Iceberg
- PowerBI - PowerBI
- Azure SQL - Azure SQL
## 0.7 Release - Dec 15th, 2021 # 0.7.0 Release
### UI - Activity Feed, Improved UX for Search ## UI - Activity Feed, Improved UX for Search
- Users will have access to Activity Feed of all the changes to the Metadata. - Users will have access to Activity Feed of all the changes to the Metadata.
- New and Improved UX for Search and Landing page. - New and Improved UX for Search and Landing page.
### Support for Table Location ## Support for Table Location
- Extract Location information from Glue, Redshift. - Extract Location information from Glue, Redshift.
- Show Location details on the Table Page. - Show Location details on the Table Page.
### ElasticSearch Improvements ## ElasticSearch Improvements
- Support SSL (including self-signed certs) enabled ElasticSearch. - Support SSL (including self-signed certs) enabled ElasticSearch.
- New entities will be indexed into ElasticSearch directly - New entities will be indexed into ElasticSearch directly
### Connectors ## Connectors
- Metabase - Metabase
- Apache Druid - Apache Druid
- Glue Improvements - Glue Improvements
@ -607,72 +677,72 @@ and prepares handy methods to help us test the connection to the source before c
- Apache Atlas Import connector - Apache Atlas Import connector
- Amundsen Import connector - Amundsen Import connector
### Other features ## Other features
- Metadata Change Event integration into Slack and framework for integration into other services such as Kafka or - Metadata Change Event integration into Slack and framework for integration into other services such as Kafka or
other Notification frameworks other Notification frameworks
- Delta Lake support, Databricks, Iceberg - Delta Lake support, Databricks, Iceberg
## 0.6 Release - Nov 17th, 2021 # 0.6.0 Release
### Metadata Versioning and Eventing Framework ## Metadata Versioning and Eventing Framework
- Capture changes to Entity Metadata from source and user interactions as versions. - Capture changes to Entity Metadata from source and user interactions as versions.
- Versioned changes will be published as events for clients to consume to take actions on. - Versioned changes will be published as events for clients to consume to take actions on.
### Data Reliability ## Data Reliability
- Improvements to Data Reliability library. - Improvements to Data Reliability library.
- Capture custom measurements through user provided SQL. - Capture custom measurements through user provided SQL.
### Airflow APIs ## Airflow APIs
- Airflow APIs to deploy DAGS and manage them. - Airflow APIs to deploy DAGS and manage them.
- UI integration to deploy ingestion workflows. - UI integration to deploy ingestion workflows.
### Connectors ## Connectors
- AWS Glue - AWS Glue
- dbt - dbt
- MariaDB - MariaDB
## 0.5 Release - Oct 19th, 2021 # 0.5.0 Release
### Support for Lineage ## Support for Lineage
- Lineage related schemas and APIs. - Lineage related schemas and APIs.
- Lineage metadata integration from AirFlow for tables. - Lineage metadata integration from AirFlow for tables.
- UI changes to show lineage information to the users. - UI changes to show lineage information to the users.
### Data Reliability ## Data Reliability
- Improvements to Data Profiler. - Improvements to Data Profiler.
- UI integration with Data Profiler to show how the table profile looks over the period of time. - UI integration with Data Profiler to show how the table profile looks over the period of time.
### Complex Types ## Complex Types
- Support complex types such as Struct, Array with nested fields. - Support complex types such as Struct, Array with nested fields.
- UI support to add expand complex types and tag, add description for nested fields. - UI support to add expand complex types and tag, add description for nested fields.
### Connectors ## Connectors
- Trino - Trino
- Redash - Redash
### Other features ## Other features
- Pipeline Entities are supported. - Pipeline Entities are supported.
- Integration with Airflow to extract Pipeline details. - Integration with Airflow to extract Pipeline details.
## 0.4 Release - Sep 20th, 2021 # 0.4.0 Release
### Support for Kafka (and Pulsar WIP) ## Support for Kafka (and Pulsar WIP)
- Support for Message Service and Topic entities in schemas, APIs, and UI. - Support for Message Service and Topic entities in schemas, APIs, and UI.
- Kafka connector and ingestion support for Confluent Schema Registry. - Kafka connector and ingestion support for Confluent Schema Registry.
### Support for Dashboards ## Support for Dashboards
- Support for Dashboard services, Dashboards, and Charts entities in schemas, APIs, and UI. - Support for Dashboard services, Dashboards, and Charts entities in schemas, APIs, and UI.
- Looker, Superset, Tableau connector, and ingestion support. - Looker, Superset, Tableau connector, and ingestion support.
### User Interface ## User Interface
- Sort search results based on Usage, Relevance, and Last updated time. - Sort search results based on Usage, Relevance, and Last updated time.
- Search string highlighted in search results. - Search string highlighted in search results.
- Support for Kafka and Dashboards from Looker, Superset, and Tableau. - Support for Kafka and Dashboards from Looker, Superset, and Tableau.
### Other features ## Other features
- Pluggable SSO integration - Auth0 support. - Pluggable SSO integration - Auth0 support.
- Support for Presto. - Support for Presto.
### Work in progress ## Work in progress
- Salesforce CRM connector. - Salesforce CRM connector.
- Data profiler to profile tables in ingestion framework and show it table details page. - Data profiler to profile tables in ingestion framework and show it table details page.

6
openmetadata-docs/content/v1.3.x/releases/releases/supported.md
View File

@ -10,6 +10,6 @@ major releases.
| Release | Release Date | | Release | Release Date |
|:--------|:-------------| |:--------|:-------------|
| 1.2.0 | 2023/10/27 | | 1.3.x | 2024/02/06 |
| 1.1.7 | 2023/10/5 | | 1.2.x | 2023/10/27 |
| 1.0.5 | 2023/06/22 | | 1.1.x | 2023/10/05 |