diff --git a/openmetadata-docs/content/partials/v1.3/releases/latest.md b/openmetadata-docs/content/partials/v1.3/releases/latest.md index 5a1694bf081..8b75db737f3 100644 --- a/openmetadata-docs/content/partials/v1.3/releases/latest.md +++ b/openmetadata-docs/content/partials/v1.3/releases/latest.md @@ -1,4 +1,4 @@ -# 1.2.0 Release 🎉 +# 1.3.0 Release 🎉 {% inlineCalloutContainer %} {% inlineCallout @@ -6,78 +6,99 @@ color="violet-70" icon="celebration" bold="Upgrade OpenMetadata" href="/deployment/upgrade" %} -Learn how to upgrade your OpenMetadata instance to 1.2.0! +Learn how to upgrade your OpenMetadata instance to 1.3.0! {% /inlineCallout %} {% /inlineCalloutContainer %} -You can find the GitHub release [here](https://github.com/open-metadata/OpenMetadata/releases/tag/1.2.0-release). - -## Domains and Data Products -- Added support for Domains and Data Products. -- Assets can be added to a Domain, and users can scope their discovery experience to one Domain. -- Assets can also be added as Data Products in a Domain. - -## Search Index -- Elasticsearch or Opensearch connectors can now bring in the search index metadata into OpenMetadata. -- The connector will populate the index’s mapping, settings, and sample data. - -## Stored Procedures -- Added support for Stored Procedures. -- Snowflake, Redshift, and BigQuery connectors are updated to bring stored procedure metadata into OpenMetadata. -- The metadata workflow will bring the Stored Procedures and parse their executions to extract lineage information. - -## Glossary Approval Workflow & Glossary Styling -- Introduced a glossary approval workflow. An approval workflow is created if Reviewers are added to a glossary. -- A task is added for reviewers to approve or reject the glossary term. The terms will show up in Draft status. -- Only the reviewers can approve or reject the term. -- Conversations are supported to discuss further about the terms. -- If no reviewer is added, then the glossary terms are approved by default. -- Introduced styling for glossary terms. Now you can add icons and color code the glossary terms for easy identification. -- Color coding helps to visually differentiate and identify the data assets, when glossary terms are added to them. - -## OpenMetadata Browser Extension -- Updated the Chrome browser extension for OpenMetadata with the new UI. -- Added support for Databases, Database Schemas, Tables, Dashboards, Charts, Pipelines, and Topics. - -## Build Automation Applications - -{% youtube videoId="pUS9-RevqsU" start="0:00" end="0:57" width="560px" height="315px" /%} - -- Added Applications into OpenMetadata, giving users a unique view of processes that can be scheduled and run in the platform. -- Search Indexing and Data Insights Report have been converted into Applications. -- UI displays all the available applications, which Admins can add or schedule. -- We will continue to add new Applications in upcoming releases. +You can find the GitHub release [here](https://github.com/open-metadata/OpenMetadata/releases/tag/1.3.0-release). ## Lineage -- Performance improvements made for lineage based on the new release of SQLfluff. -- Added support for `UPDATE … FROM Snowflake` queries -- Added column-level lineage support for `SELECT *` queries +- Revamped the lineage UI for an intuitive and comprehensive view of data flow and transformations. +- Organized nodes for better visibility with pagination support. +- Improved the display of circular dependencies. +- Nodes display the service icons, highlight dbt models, and show Data Quality results. +- Lineage can be filtered to search by Ownership, Domain, Service, Service Type, Tier, and Classification Tags. +- Supports search by Column and traces lineage even when the columns are renamed. +- Enhanced user control with collapsible sub-graphs. +- Supports editing the SQL queries for lineage edges from the UI. +- Performance improvements for faster load of large graphs. -## Connectors -- Greenplum connector is now supported. -- Couchbase connector is now supported. -- Azure Data Lake Storage is now supported. (Collate) +## Data Observability Alerts +- Data observability alerts have been distinguished from other general-purpose notifications, making it easy to get to the crucial alerts quickly. +- Sends alerts for schema changes and test case failures for the data assets that you follow. +- The overall flow has been simplified to let you easily create alerts for schema changes in your data. +- You can now get Data Quality alerts for specific Test Suites. +- Users will be alerted for all the changes to the data assets that they own. -## Customizable Landing Page - -{% youtube videoId="Y-5cPQgzNdo" start="0:00" end="2:08" width="560px" height="315px" /%} - -- Admins can create Personas to group individuals in their company, such as Data Engineers, Data Stewards, or Data Scientists. -- Admins can customize the landing page for each Persona with a set of supported widgets: Activity Feed, Announcements, Knowledge Center, etc. -- We will add support for more widgets in upcoming releases. +## Incident Manager +- Introduced Incidents Manager to improve the data quality resolution flow. +- Incidents Manager summarizes all the test case results with information about the failure severity and resolution flow. +- Supports assigning a resolution task to the users in OpenMetadata. +- Tasks are created when a data quality test has been assigned to an Assignee or a Reviewer. +- Resolved test failure also displays the comments posted on the resolution. +- The Resolved Tab displays information on the Test case name, Execution date, Reason, Comments, and information on who Resolved the issue. ## Knowledge Center (Collate) +- Supports hierarchical pages to structure the articles. +- You can easily associate knowledge articles with data assets. +- The data assets page displays the related articles. +- The block editor supports callouts to add notes, warnings, tables, and task lists. +- Quicklinks are no longer separate pages; they redirect to external links. +- Data assets can be associated with Quicklinks. +- Added Search support for Knowledge articles to filter by Owner or Tags. +- Supports preview for articles and Quicklinks. -{% youtube videoId="DfOgeZ9f7no" start="0:00" end="3:04" width="560px" height="315px" /%} +## Custom Metrics for Profiler +- Supports custom metrics for the data profiler with custom SQL to keep track of your business metrics. +- Custom metrics can be created at Table and Column levels. -- Backend APIs support creating, editing, and listing knowledge articles (with external links). -- Knowledge articles and links can be associated with a Domain, Team, or an Entity. -- UI support to build a Knowledge Center and expand the documentation of your company. +## Profiler and Data Quality +- The Profiler has been improved to support sample data ingestion without computing other metrics. +- Admins can configure the profiler to fetch up to 10,000 rows of sample data. +- Sample data can be stored in S3 buckets. +- Refined the default time range on the test case results page, adjusting it from the Last 3 days to the Last 30 days for a more encompassing view. -## Cost Analysis Report (Collate) +## Connectors +- New Google Cloud Storage for storage services. (Collate) +- New Alation connector to migrate metadata into Collate. (Collate) +- New Iceberg, SAS Viya, and Doris connectors. +- Introduced the Spark Lineage Agent to extract metadata and end-to-end lineage from Spark jobs. +- MSSQL and Oracle now support Stored Procedures. +- We now exclude system indices from the Elasticsearch connector by default. +- Added support for DB2 IBM I Series. +- Pipeline services now get owner information. +- Performance improvements for the Tableau Connector. +- We now support metadata tag extraction from Databricks. +- Supports the attribute Table Owner for metadata ingestion from Postgres. +- We now extract table descriptions when ingesting metadata from Salesforce. -{% youtube videoId="KI58oBHxTOU" start="0:00" end="0:33" width="560px" height="315px" /%} +## Glossary +- Supports soft delete for the default glossaries in OpenMetadata. +- Supports the creation of tasks to request tags or a description. +- Only the Owner can edit the Glossary term. +- Version history displays the Username instead of the User ID. -- The Usage Workflow will now also track how tables are Accessed and Updated. -- This information will be used in the Data Insights workflow to show the evolution of your used and unused assets and compare them by size. -- Support has been added for Snowflake, and we will continue to add more sources in upcoming releases. +## Localization +- Now supports RTL UI for the Hebrew language. +- New Dutch language translation. + +## Settings UI +- The Settings page UI has been revamped. + +## Data Insights +- Cost Analysis expanded to support BigQuery & Redshift. (Collate) +- Improved the Data Insights Report sent via email. + +## Other Changes +- Announcements can be notified over email, Slack, or Teams. +- Alerts are sent to a user when they are mentioned in a task or activity feed. +- We have improved the display of search results for column matches. When searching for columns, the matched results will be displayed and highlighted in the Preview pane. +- Table Type filter has been added in the Advanced Search, so that users can exclude the temporary or staging tables from search. +- Now it is easy to filter the Data assets without a Owner. +- Database and Schema were added to the Explore menu to enhance data discovery. +- Custom properties are displayed on the right of the data asset details page. +- We now display the Domain on the Users page. +- Supports the sorting of data assets by popularity based on the number of followers and thumbs up as signals. +- OpenMetadata can now handle metric history for ML models. +- When configuring the Email settings, the Username and Password fields can be left blank. +- We now support a test email button on the Email SMTP page. \ No newline at end of file diff --git a/openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/chrome-ext.md b/openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/chrome-ext.md new file mode 100644 index 00000000000..6c3a2b7b04b --- /dev/null +++ b/openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/chrome-ext.md @@ -0,0 +1,13 @@ +--- +title: OpenMetadata Google Chrome Extension +slug: /how-to-guides/guide-for-data-users/chrome-ext +--- + +# OpenMetadata Google Chrome Extension + +OpenMetadata’s Chrome Browser Extension bridges the gap between users and metadata. Visit the Chrome webstore and search for OpenMetadata to add the extension to your Chrome browser with just a few clicks. You can find the Chrome browser extension [here](https://chromewebstore.google.com/detail/openmetadata/pakbbdhbbiclnceabdmnghamabjloofc?hl=en&authuser=0&pli=1) + +Once installed, pin the extension for easy accessibility. Simply click on the OpenMetadata icon in your Chrome browser to start using the extension. Add the base URL for your OpenMetadata instance, and connect by signing in. Once signed in, the OpenMetadata Chrome extension will display your Activity Feeds from My Data page. While working on a third party tool, which is integrated with OpenMetadata, you can simply click on the extension to view the associated metadata to access details about ownership, description, associated tags, glossary terms, schema, lineage, custom properties, and more. + +Watch the video to learn how you can access all the metadata you need, directly into your active workspace. +{% youtube videoId="ZQckSIXAA6k" start="0:00" end="1:37" width="560px" height="315px" /%} \ No newline at end of file diff --git a/openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/follow-data-asset.md b/openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/follow-data-asset.md index dbb24bca599..4b5f5dded8b 100644 --- a/openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/follow-data-asset.md +++ b/openmetadata-docs/content/v1.2.x/how-to-guides/user-guide-data-users/follow-data-asset.md @@ -7,6 +7,8 @@ slug: /how-to-guides/guide-for-data-users/follow-data-asset Users can get timely information about all the **activities**, **announcements**, and **feeds** related to a data asset by following those assets. The owners of the data assets will receive all the updates by default. The follow option can be used for the assets that you do not own. +{% youtube videoId="xKg596qN7LA" start="0:00" end="1:52" width="560px" height="315px" /%} + To follow a data asset, - Navigate to the **Explore** page. - Select the relevant type of data asset (Tables, Topics, Dashboards, Pipelines, ML Models, and Containers). diff --git a/openmetadata-docs/content/v1.2.x/menu.md b/openmetadata-docs/content/v1.2.x/menu.md index f988b72523d..2f6143e3159 100644 --- a/openmetadata-docs/content/v1.2.x/menu.md +++ b/openmetadata-docs/content/v1.2.x/menu.md @@ -255,6 +255,8 @@ site_menu: url: /how-to-guides/guide-for-data-users/versions - category: How-to Guides / Guide for Data Users / How to Delete a Data Asset url: /how-to-guides/guide-for-data-users/delete + - category: How-to Guides / Guide for Data Users / OpenMetadata Google Chrome Extension + url: /how-to-guides/guide-for-data-users/chrome-ext - category: How-to Guides / Data Discovery url: /how-to-guides/data-discovery diff --git a/openmetadata-docs/content/v1.2.x/releases/releases/supported.md b/openmetadata-docs/content/v1.2.x/releases/releases/supported.md index ccac98dbbba..c1a81639866 100644 --- a/openmetadata-docs/content/v1.2.x/releases/releases/supported.md +++ b/openmetadata-docs/content/v1.2.x/releases/releases/supported.md @@ -12,4 +12,4 @@ major releases. |:--------|:-------------| | 1.2.0 | 2023/10/27 | | 1.1.7 | 2023/10/5 | -| 1.0.5 | 2023/06/22 | +| 1.0.5 | 2023/06/22 | \ No newline at end of file diff --git a/openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/chrome-ext.md b/openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/chrome-ext.md new file mode 100644 index 00000000000..6c3a2b7b04b --- /dev/null +++ b/openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/chrome-ext.md @@ -0,0 +1,13 @@ +--- +title: OpenMetadata Google Chrome Extension +slug: /how-to-guides/guide-for-data-users/chrome-ext +--- + +# OpenMetadata Google Chrome Extension + +OpenMetadata’s Chrome Browser Extension bridges the gap between users and metadata. Visit the Chrome webstore and search for OpenMetadata to add the extension to your Chrome browser with just a few clicks. You can find the Chrome browser extension [here](https://chromewebstore.google.com/detail/openmetadata/pakbbdhbbiclnceabdmnghamabjloofc?hl=en&authuser=0&pli=1) + +Once installed, pin the extension for easy accessibility. Simply click on the OpenMetadata icon in your Chrome browser to start using the extension. Add the base URL for your OpenMetadata instance, and connect by signing in. Once signed in, the OpenMetadata Chrome extension will display your Activity Feeds from My Data page. While working on a third party tool, which is integrated with OpenMetadata, you can simply click on the extension to view the associated metadata to access details about ownership, description, associated tags, glossary terms, schema, lineage, custom properties, and more. + +Watch the video to learn how you can access all the metadata you need, directly into your active workspace. +{% youtube videoId="ZQckSIXAA6k" start="0:00" end="1:37" width="560px" height="315px" /%} \ No newline at end of file diff --git a/openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/follow-data-asset.md b/openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/follow-data-asset.md index 35d018c6bcc..6d0a7ec6cec 100644 --- a/openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/follow-data-asset.md +++ b/openmetadata-docs/content/v1.3.x/how-to-guides/user-guide-data-users/follow-data-asset.md @@ -7,6 +7,8 @@ slug: /how-to-guides/guide-for-data-users/follow-data-asset Users can get timely information about all the **activities**, **announcements**, and **feeds** related to a data asset by following those assets. The owners of the data assets will receive all the updates by default. The follow option can be used for the assets that you do not own. +{% youtube videoId="xKg596qN7LA" start="0:00" end="1:52" width="560px" height="315px" /%} + To follow a data asset, - Navigate to the **Explore** page. - Select the relevant type of data asset (Tables, Topics, Dashboards, Pipelines, ML Models, and Containers). diff --git a/openmetadata-docs/content/v1.3.x/menu.md b/openmetadata-docs/content/v1.3.x/menu.md index 2216f7ff43d..84dba3d4303 100644 --- a/openmetadata-docs/content/v1.3.x/menu.md +++ b/openmetadata-docs/content/v1.3.x/menu.md @@ -259,6 +259,8 @@ site_menu: url: /how-to-guides/guide-for-data-users/versions - category: How-to Guides / Guide for Data Users / How to Delete a Data Asset url: /how-to-guides/guide-for-data-users/delete + - category: How-to Guides / Guide for Data Users / OpenMetadata Google Chrome Extension + url: /how-to-guides/guide-for-data-users/chrome-ext - category: How-to Guides / Data Discovery url: /how-to-guides/data-discovery @@ -370,6 +372,48 @@ site_menu: url: /releases/roadmap - category: Releases / All Releases url: /releases/all-releases + - category: Releases / All Releases / 1.2.0 Release + url: /releases/all-releases/#120-release + - category: Releases / All Releases / 1.1.2 Release + url: /releases/all-releases/#112-release + - category: Releases / All Releases / 1.1.1 Release + url: /releases/all-releases/#111-release + - category: Releases / All Releases / 1.1.0 Release + url: /releases/all-releases/#110-release + - category: Releases / All Releases / 1.0.0 Release + url: /releases/all-releases/#100-release + - category: Releases / All Releases / 0.13.3 Release + url: /releases/all-releases/#0133-release + - category: Releases / All Releases / 0.13.2 Release + url: /releases/all-releases/#0132-release + - category: Releases / All Releases / 0.13.1 Release + url: /releases/all-releases/#0131-release + - category: Releases / All Releases / 0.13.0 Release + url: /releases/all-releases/#0130-release + - category: Releases / All Releases / 0.12.3 Release + url: /releases/all-releases/#0123-release + - category: Releases / All Releases / 0.12.2 Release + url: /releases/all-releases/#0122-release + - category: Releases / All Releases / 0.12.1 Release + url: /releases/all-releases/#0121-release + - category: Releases / All Releases / 0.12.0 Release + url: /releases/all-releases/#0120-release + - category: Releases / All Releases / 0.10.1 Release + url: /releases/all-releases/#0101-release + - category: Releases / All Releases / 0.10.0 Release + url: /releases/all-releases/#0100-release + - category: Releases / All Releases / 0.9.0 Release + url: /releases/all-releases/#090-release + - category: Releases / All Releases / 0.8.0 Release + url: /releases/all-releases/#080-release + - category: Releases / All Releases / 0.7.0 Release + url: /releases/all-releases/#070-release + - category: Releases / All Releases / 0.6.0 Release + url: /releases/all-releases/#060-release + - category: Releases / All Releases / 0.5.0 Release + url: /releases/all-releases/#050-release + - category: Releases / All Releases / 0.4.0 Release + url: /releases/all-releases/#040-release - category: Connectors url: /connectors diff --git a/openmetadata-docs/content/v1.3.x/releases/releases/index.md b/openmetadata-docs/content/v1.3.x/releases/releases/index.md index 46188b5efdb..d3e7e311b97 100644 --- a/openmetadata-docs/content/v1.3.x/releases/releases/index.md +++ b/openmetadata-docs/content/v1.3.x/releases/releases/index.md @@ -14,7 +14,79 @@ version. To see what's coming in next releases, please check our [Roadmap](/rele {% partial file="/v1.3/releases/latest.md" /%} -# 1.1.2 Release 🎉 +# 1.2.0 Release + +## Domains and Data Products +- Added support for Domains and Data Products. +- Assets can be added to a Domain, and users can scope their discovery experience to one Domain. +- Assets can also be added as Data Products in a Domain. + +## Search Index +- Elasticsearch or Opensearch connectors can now bring in the search index metadata into OpenMetadata. +- The connector will populate the index’s mapping, settings, and sample data. + +## Stored Procedures +- Added support for Stored Procedures. +- Snowflake, Redshift, and BigQuery connectors are updated to bring stored procedure metadata into OpenMetadata. +- The metadata workflow will bring the Stored Procedures and parse their executions to extract lineage information. + +## Glossary Approval Workflow & Glossary Styling +- Introduced a glossary approval workflow. An approval workflow is created if Reviewers are added to a glossary. +- A task is added for reviewers to approve or reject the glossary term. The terms will show up in Draft status. +- Only the reviewers can approve or reject the term. +- Conversations are supported to discuss further about the terms. +- If no reviewer is added, then the glossary terms are approved by default. +- Introduced styling for glossary terms. Now you can add icons and color code the glossary terms for easy identification. +- Color coding helps to visually differentiate and identify the data assets, when glossary terms are added to them. + +## OpenMetadata Browser Extension +- Updated the Chrome browser extension for OpenMetadata with the new UI. +- Added support for Databases, Database Schemas, Tables, Dashboards, Charts, Pipelines, and Topics. + +## Build Automation Applications + +{% youtube videoId="pUS9-RevqsU" start="0:00" end="0:57" width="560px" height="315px" /%} + +- Added Applications into OpenMetadata, giving users a unique view of processes that can be scheduled and run in the platform. +- Search Indexing and Data Insights Report have been converted into Applications. +- UI displays all the available applications, which Admins can add or schedule. +- We will continue to add new Applications in upcoming releases. + +## Lineage +- Performance improvements made for lineage based on the new release of SQLfluff. +- Added support for `UPDATE … FROM Snowflake` queries +- Added column-level lineage support for `SELECT *` queries + +## Connectors +- Greenplum connector is now supported. +- Couchbase connector is now supported. +- Azure Data Lake Storage is now supported. (Collate) + +## Customizable Landing Page + +{% youtube videoId="Y-5cPQgzNdo" start="0:00" end="2:08" width="560px" height="315px" /%} + +- Admins can create Personas to group individuals in their company, such as Data Engineers, Data Stewards, or Data Scientists. +- Admins can customize the landing page for each Persona with a set of supported widgets: Activity Feed, Announcements, Knowledge Center, etc. +- We will add support for more widgets in upcoming releases. + +## Knowledge Center (Collate) + +{% youtube videoId="DfOgeZ9f7no" start="0:00" end="3:04" width="560px" height="315px" /%} + +- Backend APIs support creating, editing, and listing knowledge articles (with external links). +- Knowledge articles and links can be associated with a Domain, Team, or an Entity. +- UI support to build a Knowledge Center and expand the documentation of your company. + +## Cost Analysis Report (Collate) + +{% youtube videoId="KI58oBHxTOU" start="0:00" end="0:33" width="560px" height="315px" /%} + +- The Usage Workflow will now also track how tables are Accessed and Updated. +- This information will be used in the Data Insights workflow to show the evolution of your used and unused assets and compare them by size. +- Support has been added for Snowflake, and we will continue to add more sources in upcoming releases. + +# 1.1.2 Release ## Data Quality - Added support for Postgres version 11.19. @@ -43,7 +115,7 @@ version. To see what's coming in next releases, please check our [Roadmap](/rele - Deprecated Flyway and using native migrations. - Improved Test Suite UI performance. -# 1.1.1 Release 🎉 +# 1.1.1 Release ## UI Improvements @@ -76,7 +148,7 @@ version. To see what's coming in next releases, please check our [Roadmap](/rele - Test Suite Improvements -# 1.1 Release 🎉 +# 1.1.0 Release {% inlineCalloutContainer %} {% inlineCallout @@ -132,7 +204,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1! - Added AWS OpenSearch client-specific support. This allows us to update the Elasticsearch version support up to 7.16. -# 1.0 Release 🎉 +# 1.0.0 Release ## APIs & Schema - **Stabilized** and improved the Schemas and APIs. @@ -196,7 +268,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1! - **Entity names** support apostrophes. - The **Summary panel** has been improved to be consistent across the UI. -# [0.13.3 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.3-release) - March 30th 2023 🎉 +# 0.13.3 Release ## Ingestion Framework - Datalake Avro & Json, JsonZip support @@ -235,7 +307,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1! ## Data Quality - Fix column values between test -# [0.13.2 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.2-release) - Latest - Jan 30th 2023 🎉 +# 0.13.2 Release ## Improved SQL Lineage - We have collaborated with the [sqllineage](https://github.com/reata/sqllineage) and [sqlfluff](https://www.sqlfluff.com/) communities @@ -269,7 +341,7 @@ Learn how to upgrade your OpenMetadata instance to 1.1! - Added support for Postgres Foreign Tables - Added support for Datalake profiler row-based sampling -# [0.13.1 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.1-release) - Dec 20th 2022** 🎉 +# 0.13.1 Release ## Profiler and Data Quality - Freshness Metric has been introduced. Data freshness shows DML operations performed against a table and the number of rows affected. All this is displayed within the data profiler with filterable graphs. This is currently supported for BigQuery, Snowflake, and Redshift. - Support has been added for data quality tests on Data Lake. @@ -293,7 +365,7 @@ The logic for Notification Support has been improved. Users can define Alerts ba - Improvements have been made to Advanced Search. Now, when a filter is applied, the details of the filter selected are displayed for clarity. - On the Explore page UI, the Side Preview is now available for all data assets. Previously it was only displayed for tables. -# [0.13.0 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.13.0-release) - Dec 8th 2022 🎉 +# 0.13.0 Release {% youtube videoId="oNbMnTW5AkE" start="0:00" end="7:51" width="560px" height="315px" /%} ## Data Insights and KPI @@ -324,19 +396,19 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan - Soft deleted entities can be restored. Currently, only the ML Models are not supported. - Soft deleted teams can be restored. When restoring a soft deleted parent team, the child teams will not be restored by default. -# [0.12.3 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.3-release) - Nov 18th 2022 🎉 +# 0.12.3 Release ## Bug Fixes - User suggestion index mapping - Tag and Glossary terms caching -# [0.12.2 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.2-release) - Oct 20th 2022 🎉 +# 0.12.2 Release ## Ingestion - Databricks lineage - Added support for Airflow version 2.2.2 as a workflow scheduler ## Bug Fixes - Support same table across different databases for the profiler -# [0.12.1 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.1-release) - Oct 3rd 2022 🎉 +# 0.12.1 Release ## Basic Authentication - User/Password signup and login @@ -363,15 +435,13 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan - Around 136 Features/Improvements/Tests made it into 0.12.1 release -# [0.12.0 Release](https://github.com/open-metadata/OpenMetadata/releases/tag/0.12.0-release) - Sept 7th 2022 🎉 +# 0.12.0 Release You can read the Release Blog [here](https://blog.open-metadata.org/openmetadata-0-12-0-release-1ac059700de4) or watch an awesome video showing the new features! {% youtube videoId="tv3pyCLcJfQ" start="0:00" end="17:04" width="560px" height="315px" /%} - - ## Team Hierarchy Prior releases supported a flat hierarchy of just Teams and Users. In 0.12, support has been added for the entire organizational hierarchy with Business Unit, Division, Department, and Groups. An organization from small to very large can now be modeled in OpenMetadata with this feature. @@ -442,7 +512,7 @@ When creating a new user or when a user is registering for the first time, the d UI improvements have been made on the Schema, Service, and Database details pages. Manage Tab has been replaced with the manage button on the UI. -## 0.10.1 Release - May 17th, 2022 +# 0.10.1 Release - Support for Postgres as OpenMetadata Store [#4601](https://github.com/open-metadata/OpenMetadata/issues/4601) - UI Improvements in 0.10.1 Release [#4600](https://github.com/open-metadata/OpenMetadata/issues/4600) @@ -450,26 +520,26 @@ Manage Tab has been replaced with the manage button on the UI. - UI Ingestion Improvements - Support for Dashboards & Messaging Services [#4843](https://github.com/open-metadata/OpenMetadata/issues/4843) - Security: Fix Azure SSO and support refresh tokens in [#4989](https://github.com/open-metadata/OpenMetadata/issues/4989) -## 0.10.0 Release - Apr 27th, 2022 +# 0.10.0 Release -### Support for Database Schema +## Support for Database Schema OpenMetadata supports databases, service name databases, and tables. We’ve added Database Schema as part of the FQN. For each external data source, we ingest the database, as well as the tables that are contained underneath the schemas. -### Support for Hard Delete +## Support for Hard Delete OpenMetadata supported soft deletions. Now, we also support the hard deletion of entities through the UI, APIs, and ingestion. Hard deleting an entity removes the entity and all of its relationships. This will also generate a change event. -### Deploy Ingestion from UI +## Deploy Ingestion from UI OpenMetadata has refactored the service connections to simplify the ingestion jobs from both the ingestion framework and the UI. We now use the pydantic models automatically generated from the JSON schemas for the connection definition. The ‘Add Service’ form is automatically generated in the UI based on the JSON schema specifications for the various connectors that are supported in OpenMetadata. -### Download dbt Manifest Files from Amazon S3 or Google Cloud Storage +## Download dbt Manifest Files from Amazon S3 or Google Cloud Storage Previously, when ingesting the models and lineage from dbt, we passed the path of the dbt manifest and catalog files directly into the workflow. We’ve worked on improving the quality of life of dbt. Now, we can dynamically download @@ -477,20 +547,20 @@ these files from Amazon S3 or Google Cloud Storage. This way we can have any oth extract the catalog, and put it into any cloud service. We just need the path name and workflow job details from the metadata extraction to be able to ingest metadata. -### JSON Schema based Connection Definition +## JSON Schema based Connection Definition Each service (database, dashboard, messaging, or pipeline service) has its own configuration specifications, with some unique requirements for some services. Instead of the ad hoc definitions of the source module in Python for each connector, we’ve worked on the full refactoring of the ingestion framework. We now use the pydantic models automatically generated from the JSON schemas for the connection definition. -### Airflow Rest APIs +## Airflow Rest APIs The Airflow REST APIs have been refactored. With our API centric model, we are creating a custom airflow rest API directly on top of Airflow using plugins. This passes the connection information to automatically generate all the dags and prepares handy methods to help us test the connection to the source before creating the service. -### UI Changes +## UI Changes - The UI improvements are directed toward providing a consistent user experience. - Hard Deletion of Entities: With the support for the hard deletion of entities, we can permanently delete tables, @@ -506,7 +576,7 @@ and prepares handy methods to help us test the connection to the source before c - Activity Feeds: The Activity Feeds UI supports infinite scrolling. - Add User: A user can be added from the Users page. -### Security Changes +## Security Changes - **Support Refresh Tokens for Auth0 and Okta SSO**: The JWT tokens generated by the SSO providers expire by default in about an hour, making the user re-login often. In this release, we’ve added support for refresh tokens for Auth0 and Okta SSO. The tokens are refreshed silently behind the scenes to provide an uninterrupted user experience. @@ -515,9 +585,9 @@ and prepares handy methods to help us test the connection to the source before c This is supported both on the front end for user authentication and on the ingestion side. - **Azure SSO**: Support has been added for Azure SSO on Airflow. -## 0.9.0 - March 10th, 2022 +# 0.9.0 Release -### Collaboration +## Collaboration - Conversations in the main feed. - Users can ask each other questions, add suggestions and replies. @@ -525,12 +595,12 @@ and prepares handy methods to help us test the connection to the source before c - Glossary. - Table details - Click through on usage to see who or what services are using it, what queries are pulling from it. -### Data Quality +## Data Quality - Ability to create and monitor the test cases. - Data Quality Tests support with Json Schemas and APIs. - UI Integration to enable user to write tests and run them on Airflow. -### Glossary +## Glossary - Glossaries are a Controlled Vocabulary in an organization used to define the concepts and terminologies specific to a particular domain. @@ -538,7 +608,7 @@ and prepares handy methods to help us test the connection to the source before c - UI support to add Glossary and Glossary Terms. - Support for using Glossary terms to annotate Entities and Search using Glossary Terms. -### Connectors +## Connectors - Apache Iceberg - Azure SQL - Clickhouse @@ -554,52 +624,52 @@ and prepares handy methods to help us test the connection to the source before c - Apache Atlas ,Import Metadata from Apache Atlas into OpenMetadata - Amundsen, Import Metadata from Amundsen into OpenMetadata -### Lineage +## Lineage - DataSource SQL Parsing support to extract Lineage - View Lineage support -### Pipeline +## Pipeline - Capture pipeline status as it happens -### Security +## Security - Security policies through the UI. - Configuration personas and authorization based on policies. - AWS SSO support. -## 0.8 Release - Jan 22nd, 2022 +# 0.8.0 Release -### Access Control Policies +## Access Control Policies - Design of Access Control Policies. - Provide Role based access control with community feedback. -### Eventing Webhook +## Eventing Webhook - Register webhooks to get metadata event notifications. - Metadata Change Event integration into Slack and framework for integration into other services such as Kafka or other Notification frameworks -### Connectors +## Connectors - Delta Lake - Iceberg - PowerBI - Azure SQL -## 0.7 Release - Dec 15th, 2021 +# 0.7.0 Release -### UI - Activity Feed, Improved UX for Search +## UI - Activity Feed, Improved UX for Search - Users will have access to Activity Feed of all the changes to the Metadata. - New and Improved UX for Search and Landing page. -### Support for Table Location +## Support for Table Location - Extract Location information from Glue, Redshift. - Show Location details on the Table Page. -### ElasticSearch Improvements +## ElasticSearch Improvements - Support SSL (including self-signed certs) enabled ElasticSearch. - New entities will be indexed into ElasticSearch directly -### Connectors +## Connectors - Metabase - Apache Druid - Glue Improvements @@ -607,72 +677,72 @@ and prepares handy methods to help us test the connection to the source before c - Apache Atlas Import connector - Amundsen Import connector -### Other features +## Other features - Metadata Change Event integration into Slack and framework for integration into other services such as Kafka or other Notification frameworks - Delta Lake support, Databricks, Iceberg -## 0.6 Release - Nov 17th, 2021 +# 0.6.0 Release -### Metadata Versioning and Eventing Framework +## Metadata Versioning and Eventing Framework - Capture changes to Entity Metadata from source and user interactions as versions. - Versioned changes will be published as events for clients to consume to take actions on. -### Data Reliability +## Data Reliability - Improvements to Data Reliability library. - Capture custom measurements through user provided SQL. -### Airflow APIs +## Airflow APIs - Airflow APIs to deploy DAGS and manage them. - UI integration to deploy ingestion workflows. -### Connectors +## Connectors - AWS Glue - dbt - MariaDB -## 0.5 Release - Oct 19th, 2021 +# 0.5.0 Release -### Support for Lineage +## Support for Lineage - Lineage related schemas and APIs. - Lineage metadata integration from AirFlow for tables. - UI changes to show lineage information to the users. -### Data Reliability +## Data Reliability - Improvements to Data Profiler. - UI integration with Data Profiler to show how the table profile looks over the period of time. -### Complex Types +## Complex Types - Support complex types such as Struct, Array with nested fields. - UI support to add expand complex types and tag, add description for nested fields. -### Connectors +## Connectors - Trino - Redash -### Other features +## Other features - Pipeline Entities are supported. - Integration with Airflow to extract Pipeline details. -## 0.4 Release - Sep 20th, 2021 +# 0.4.0 Release -### Support for Kafka (and Pulsar WIP) +## Support for Kafka (and Pulsar WIP) - Support for Message Service and Topic entities in schemas, APIs, and UI. - Kafka connector and ingestion support for Confluent Schema Registry. -### Support for Dashboards +## Support for Dashboards - Support for Dashboard services, Dashboards, and Charts entities in schemas, APIs, and UI. - Looker, Superset, Tableau connector, and ingestion support. -### User Interface +## User Interface - Sort search results based on Usage, Relevance, and Last updated time. - Search string highlighted in search results. - Support for Kafka and Dashboards from Looker, Superset, and Tableau. -### Other features +## Other features - Pluggable SSO integration - Auth0 support. - Support for Presto. -### Work in progress +## Work in progress - Salesforce CRM connector. - Data profiler to profile tables in ingestion framework and show it table details page. diff --git a/openmetadata-docs/content/v1.3.x/releases/releases/supported.md b/openmetadata-docs/content/v1.3.x/releases/releases/supported.md index ccac98dbbba..ac6064ffb20 100644 --- a/openmetadata-docs/content/v1.3.x/releases/releases/supported.md +++ b/openmetadata-docs/content/v1.3.x/releases/releases/supported.md @@ -10,6 +10,6 @@ major releases. | Release | Release Date | |:--------|:-------------| -| 1.2.0 | 2023/10/27 | -| 1.1.7 | 2023/10/5 | -| 1.0.5 | 2023/06/22 | +| 1.3.x | 2024/02/06 | +| 1.2.x | 2023/10/27 | +| 1.1.x | 2023/10/05 | \ No newline at end of file