mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2026-01-08 13:36:32 +00:00
Docs: Data Quality & Observability (#16701)
This commit is contained in:
Shilpa Vernekar
GitHub
parent
e2f845b2d1
commit
bcbc2c0ded
@ -1,7 +1,7 @@
|
||||
# 1.4.3 Release 🎉
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**June 15, 2024**
|
||||
**June 15th, 2024**
|
||||
{% /note %}
|
||||
|
||||
{% inlineCalloutContainer %}
|
||||
|
||||
@ -1,54 +0,0 @@
|
||||
---
|
||||
title: Incident Manager
|
||||
slug: /how-to-guides/data-observability/incident-manager
|
||||
---
|
||||
|
||||
# Incident Manager
|
||||
|
||||
Using Incident Manager, managing data quality issues becomes streamlined and efficient. By centralizing the resolution process, assigning tasks, and logging root causes, your team can quickly address and resolve failures. The historical record of past incidents serves as a comprehensive guide, aiding your team in troubleshooting and resolving issues more effectively. All the necessary context is readily available, making it easier to maintain high data quality standards.
|
||||
|
||||
## Overview of the Incident Manager
|
||||
|
||||
The Incident Manager serves as a centralized hub to handle the resolution flow of failed Data Quality Tests. When a test fails, users can:
|
||||
|
||||
- **Acknowledge the Issue:** Recognize and confirm that there is a problem that needs attention.
|
||||
- **Assign Responsibility:** Designate a specific person or team to address the errors.
|
||||
- **Log the Root Cause:** Document the underlying cause of the failure for future reference and analysis.
|
||||
|
||||
## Using the Test Resolution Flow
|
||||
|
||||
The Test Resolution flow is a critical feature of the Incident Manager. Here’s how it works:
|
||||
|
||||
1. **Failure Notification:** When a Data Quality Test fails, the system generates a notification.
|
||||
2. **Acknowledge the Failure:** The designated user acknowledges the issue within the Incident Manager.
|
||||
3. **Assignment:** The issue is then assigned to a knowledgeable user or team responsible for resolving it.
|
||||
4. **Status Updates:** The assigned user can update the status of the issue, keeping the organization informed about progress and any developments.
|
||||
5. **Sharing Updates:** All impacted users receive updates, ensuring everyone stays informed about the resolution process.
|
||||
|
||||
## Building a Troubleshooting Handbook
|
||||
|
||||
One of the powerful features of the Incident Manager is its ability to store all past failures. This historical data becomes a valuable troubleshooting handbook for your team. Here's how you can leverage it:
|
||||
|
||||
- **Explore Similar Scenarios:** Review previous incidents to understand how similar issues were resolved.
|
||||
- **Contextual Information:** Access all necessary context directly within OpenMetadata, including previous resolutions, root causes, and responsible teams.
|
||||
- **Continuous Improvement:** Use historical data to improve data quality tests and prevent future failures.
|
||||
|
||||
|
||||
## Steps to Get Started
|
||||
|
||||
1. **Access the Incident Manager:** Navigate to the Incident Manager within the OpenMetadata platform.
|
||||
2. **Monitor Data Quality Tests:** Keep an eye on your data quality tests to quickly identify any failures.
|
||||
3. **Acknowledge and Assign:** Acknowledge any issues promptly and assign them to the appropriate team members.
|
||||
4. **Log and Learn:** Document the root cause of each failure and use the stored information to learn and improve.
|
||||
|
||||
By following these steps, you'll ensure that your organization effectively manages data quality issues, maintains high standards, and continuously improves its data quality processes.
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="How to work with Incident Manager"
|
||||
icon="MdMenuBook"
|
||||
href="/how-to-guides/data-observability/incident-manager/workflow"%}
|
||||
Incident Manager Workflow
|
||||
{%/inlineCallout%}
|
||||
{%/inlineCalloutContainer%}
|
||||
@ -1,19 +0,0 @@
|
||||
---
|
||||
title: Data Observability
|
||||
slug: /how-to-guides/data-observability
|
||||
---
|
||||
|
||||
# Data Observability
|
||||
|
||||
OpenMetadata ensures the health and performance of your data systems by providing comprehensive data observability features. These features offer insights into the state of test cases, helping to detect, diagnose, and resolve data issues quickly. By monitoring data flows and data quality in real-time, data teams can ensure that data remains reliable and trustworthy. OpenMetadata supports [observability alerts and notifications](/how-to-guides/admin-guide/alerts) to help you maintain the integrity and performance of your data systems.
|
||||
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="Incident Manager"
|
||||
icon="MdMenuBook"
|
||||
href="/how-to-guides/data-observability/incident-manager"%}
|
||||
Set up incident management in OpenMetadata.
|
||||
{%/inlineCallout%}
|
||||
{%/inlineCalloutContainer%}
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
title: How to work with Incident Manager
|
||||
slug: /how-to-guides/data-observability/incident-manager/workflow
|
||||
title: How to work with the Incident Manager
|
||||
slug: /how-to-guides/data-quality-observability/incident-manager/workflow
|
||||
---
|
||||
|
||||
# How to Work with Incident Manager Workflow
|
||||
# How to Work with the Incident Manager Workflow
|
||||
|
||||
## 1. Incident Dashboard
|
||||
|
||||
@ -0,0 +1,105 @@
|
||||
---
|
||||
title: Incident Manager
|
||||
slug: /how-to-guides/data-quality-observability/incident-manager
|
||||
---
|
||||
|
||||
# Overview of the Incident Manager
|
||||
|
||||
Using Incident Manager, managing data quality issues becomes streamlined and efficient. By centralizing the resolution process, assigning tasks, and logging root causes, your team can quickly address and resolve failures. The historical record of past incidents serves as a comprehensive guide, aiding your team in troubleshooting and resolving issues more effectively. All the necessary context is readily available, making it easier to maintain high data quality standards.
|
||||
|
||||
## Opening and Triaging Incidents
|
||||
In v1.1.0, we introduced the ability for user to manage and triage incidents linked to failures. When a test case fails, it will automatically open a new incident and mark it as new. If enough information is available, OpenMetadata will automatically assign a severity to the incident; note that you can override this severity. It indicates that a new failure has happened.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-new.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
The Incident Manager serves as a centralized hub to handle the resolution flow of failed Data Quality Tests. Once an incident has been open you will be able to triage and manage it. You can perform different actions at this stage:
|
||||
|
||||
- **Acknowledge the Issue:** Recognize and confirm that there is a problem that needs attention. By marking with `ack` you can inform users that people are aware of the ongoing incident.
|
||||
- **Assign Responsibility:** Designate a specific person or team to address the errors. By marking with `assign` you can open a task for the assignee.
|
||||
- **Log the Root Cause:** Document the underlying cause of the failure for future reference and analysis.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-ack-form.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-ack.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
You can mark the incident as `resolved`. The user will be required to specify the reason and add a comment. This provides context regarding the incident and helps users further understand what might have gone wrong
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-resolved-form.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-resolved.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
## Using the Test Resolution Flow
|
||||
|
||||
The Test Resolution flow is a critical feature of the Incident Manager. Here’s how it works:
|
||||
|
||||
1. **Failure Notification:** When a Data Quality Test fails, the system generates a notification.
|
||||
2. **Acknowledge the Failure:** The designated user acknowledges the issue within the Incident Manager.
|
||||
3. **Assignment:** The issue is then assigned to a knowledgeable user or team responsible for resolving it.
|
||||
4. **Status Updates:** The assigned user can update the status of the issue, keeping the organization informed about progress and any developments.
|
||||
5. **Sharing Updates:** All impacted users receive updates, ensuring everyone stays informed about the resolution process.
|
||||
|
||||
## Incidents Context & History
|
||||
|
||||
When clicking on an open incident you will different information:
|
||||
**Open Incident:** this section will show you open incidents with the timeline and any comments/collaboration that might have been happening.
|
||||
**Closed Incidents:** this section will show you incidents that have been resolved in the past with the timeline and any comments/collaboration that might have been happening and the resolution reason.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/incident-management-page.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
## Building a Troubleshooting Handbook
|
||||
|
||||
One of the powerful features of the Incident Manager is its ability to store all past failures. This historical data becomes a valuable troubleshooting handbook for your team. Here's how you can leverage it:
|
||||
|
||||
- **Explore Similar Scenarios:** Review previous incidents to understand how similar issues were resolved.
|
||||
- **Contextual Information:** Access all necessary context directly within OpenMetadata, including previous resolutions, root causes, and responsible teams.
|
||||
- **Continuous Improvement:** Use historical data to improve data quality tests and prevent future failures.
|
||||
|
||||
|
||||
## Steps to Get Started
|
||||
|
||||
1. **Access the Incident Manager:** Navigate to the Incident Manager within the OpenMetadata platform.
|
||||
2. **Monitor Data Quality Tests:** Keep an eye on your data quality tests to quickly identify any failures.
|
||||
3. **Acknowledge and Assign:** Acknowledge any issues promptly and assign them to the appropriate team members.
|
||||
4. **Log and Learn:** Document the root cause of each failure and use the stored information to learn and improve.
|
||||
|
||||
By following these steps, you'll ensure that your organization effectively manages data quality issues, maintains high standards, and continuously improves its data quality processes.
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="How to work with the Incident Manager"
|
||||
icon="MdManageSearch"
|
||||
href="/how-to-guides/data-quality-observability/incident-manager/workflow"%}
|
||||
Set up the Incident Manager workflow.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="Root Cause Analysis (Collate)"
|
||||
icon="MdFactCheck"
|
||||
href="/how-to-guides/data-quality-observability/incident-manager/root-cause-analysis"%}
|
||||
Understand the nature of the failure and take corrective actions.
|
||||
{%/inlineCallout%}
|
||||
{%/inlineCalloutContainer%}
|
||||
@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Root Cause Analysis
|
||||
slug: /quality-and-observability/data-quality/root-cause-analysis
|
||||
slug: /how-to-guides/data-quality-observability/incident-manager/root-cause-analysis
|
||||
---
|
||||
|
||||
# Root Cause Analysis
|
||||
@ -0,0 +1,38 @@
|
||||
---
|
||||
title: Data Quality and Observability
|
||||
slug: /how-to-guides/data-quality-observability
|
||||
---
|
||||
|
||||
# Data Quality and Observability
|
||||
|
||||
OpenMetadata offers a simple and easy-to-use solution for quality and observability. With no code tests, observability metrics, incident management, and root cause analysis (Collate feature), you have a unified solution for discovery, governance, and observability.
|
||||
|
||||
OpenMetadata ensures the health and performance of your data systems by providing comprehensive data observability features. These features offer insights into the state of test cases, helping to detect, diagnose, and resolve data issues quickly. By monitoring data flows and data quality in real-time, data teams can ensure that data remains reliable and trustworthy. OpenMetadata supports [observability alerts and notifications](/how-to-guides/admin-guide/alerts) to help you maintain the integrity and performance of your data systems.
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
{%inlineCallout
|
||||
icon="MdGppGood"
|
||||
bold="Data Quality"
|
||||
href="/how-to-guides/data-quality-observability/quality"%}
|
||||
Deep dive into how to set up quality tests, alert and triage and resolve incidents!
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdVisibility"
|
||||
bold="Data Profiler"
|
||||
href="/how-to-guides/data-quality-observability/profiler"%}
|
||||
Deep dive into how to set up the profiler in OpenMetadata.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdAddAlert"
|
||||
bold="Observability Alerts"
|
||||
href="/how-to-guides/data-quality-observability/observability/alerts"%}
|
||||
Set up observability alerts in OpenMetadata.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="Incident Manager"
|
||||
icon="MdMenuBook"
|
||||
href="/how-to-guides/data-quality-observability/incident-manager"%}
|
||||
Set up incident management in OpenMetadata.
|
||||
{%/inlineCallout%}
|
||||
{%/inlineCalloutContainer%}
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
title: Alerts
|
||||
slug: /quality-and-observability/data-quality/alerts
|
||||
title: Observability Alerts
|
||||
slug: /how-to-guides/data-quality-observability/observability/alerts
|
||||
---
|
||||
|
||||
# Alerts
|
||||
# Observability Alerts
|
||||
OpenMetadata provides a native way to get alerted in case of test case failure allowing you to proactively resolve data incidents
|
||||
|
||||
## Setting Up Alerts
|
||||
@ -47,7 +47,6 @@ Trigger section will allow you set the condition for which an alert should be tr
|
||||
caption="Alerts Menu"
|
||||
/%}
|
||||
|
||||
|
||||
### Step 4 - Select a Destination
|
||||
In the destination section you will be able to select between `internal` and `external` destination:
|
||||
- `internal`: allow you to select the destination as an internal user, team or admin. The subscription set to this user, team or admin will be use to dispatch the alert
|
||||
@ -0,0 +1,25 @@
|
||||
---
|
||||
title: Data Observability
|
||||
slug: /how-to-guides/data-quality-observability/observability
|
||||
---
|
||||
|
||||
# Data Observability
|
||||
|
||||
OpenMetadata has been providing observability alerts right from the start to notify users of important data lifecycle events: schema modifications, ownership shifts, and tagging updates. Users can define fine-grained alerts and notifications.
|
||||
|
||||
Starting from the 1.3 release, Data Observability alerts have been completely revamped, simplifying the process of monitoring data. Users can quickly create alerts for:
|
||||
- **Changes in the Metadata:** such as schema changes,
|
||||
- **Data Quality Failures:** to filter by Test Suite,
|
||||
- **Pipeline Status Failures:** when ingesting runs from your ETL systems, and
|
||||
- **Ingestion Pipeline Monitoring:** for OpenMetadata’s ingestion workflows
|
||||
|
||||
Depending on your use cases, notifications can be sent to owners, admins, teams, or users, providing a more personalized and informed experience. Teams can configure their dedicated Slack, MS Teams, or Google Chat channels to receive notifications related to their data assets, streamlining communication and collaboration. With the alerts and notifications in OpenMetadata, users can send Announcements over email, Slack, or Teams. Alerts are sent to a user when they are mentioned in a task or an activity feed.
|
||||
|
||||
{% youtube videoId="qc-3sZ_eU5Y" start="0:00" end="2:04" width="560px" height="315px" /%}
|
||||
|
||||
{%inlineCallout
|
||||
icon="MdAddAlert"
|
||||
bold="Observability Alerts"
|
||||
href="/how-to-guides/data-quality-observability/observability/alerts"%}
|
||||
Set up observability alerts in OpenMetadata.
|
||||
{%/inlineCallout%}
|
||||
@ -1,6 +1,6 @@
|
||||
---
|
||||
title: Auto PII Tagging
|
||||
slug: /quality-and-observability/profiler/auto-pii-tagging
|
||||
slug: /how-to-guides/data-quality-observability/profiler/auto-pii-tagging
|
||||
---
|
||||
|
||||
# Auto PII Tagging
|
||||
@ -1,6 +1,6 @@
|
||||
---
|
||||
title: External Profiler Workflow
|
||||
slug: /quality-and-observability/profiler/external-workflow
|
||||
slug: /how-to-guides/data-quality-observability/profiler/external-workflow
|
||||
---
|
||||
|
||||
# External Profiler Workflow
|
||||
@ -23,7 +23,7 @@ You might also want to check out how to configure external sample data. You can
|
||||
{% tile
|
||||
title="External Sample Data"
|
||||
description="Configure OpenMetadata to store sample data in an external storage such as S3"
|
||||
link="/connectors/ingestion/workflows/profiler/external-sample-data"
|
||||
link="/how-to-guides/data-quality-observability/profiler/external-sample-data"
|
||||
/ %}
|
||||
{% /tilesContainer %}
|
||||
|
||||
@ -0,0 +1,52 @@
|
||||
---
|
||||
title: Data Profiler
|
||||
slug: /how-to-guides/data-quality-observability/profiler
|
||||
---
|
||||
|
||||
# Overview of Data Profiler
|
||||
|
||||
The profiler in OpenMetadata helps to understand the shape of your data and to quickly validate assumptions. The data profiler helps to capture table usage statistics over a period of time. This happens as part of profiler ingestion. Data profiles enable you to check for null values in non-null columns, for duplicates in a unique column, etc. You can gain a better understanding of column data distributions through the descriptive statistics provided.
|
||||
|
||||
Watch the video to understand OpenMetadata’s native Data Profiler and Data Quality tests.
|
||||
|
||||
{% youtube videoId="gLdTOF81YpI" start="0:00" end="1:08:10" width="560px" height="315px" /%}
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="Profiler Tab"
|
||||
icon="MdSecurity"
|
||||
href="/how-to-guides/data-quality-observability/profiler/tab"%}
|
||||
Get a complete picture of the Table Profile and Column Profile details.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdVisibility"
|
||||
bold="Profiler Workflow"
|
||||
href="/how-to-guides/data-quality-observability/profiler/workflow"%}
|
||||
Configure and run the Profiler Workflow to extract Profiler data.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdAnalytics"
|
||||
bold="Metrics"
|
||||
href="/how-to-guides/data-quality-observability/profiler/metrics"%}
|
||||
Learn about the supported profiler metrics.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdViewCompact"
|
||||
bold="Sample Data"
|
||||
href="/how-to-guides/data-quality-observability/profiler/external-sample-data"%}
|
||||
Learn about the external storage for sample data.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdOutlineSchema"
|
||||
bold="External Workflow"
|
||||
href="/how-to-guides/data-quality-observability/profiler/external-workflow"%}
|
||||
Run a single workflow profiler for the entire source externally.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdOutlinePersonPin"
|
||||
bold="Auto PII Tagging"
|
||||
href="/how-to-guides/data-quality-observability/profiler/auto-pii-tagging"%}
|
||||
Auto tag data as PII Sensitive/NonSensitive at the column level.
|
||||
{%/inlineCallout%}
|
||||
{%/inlineCalloutContainer%}
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
title: Metrics
|
||||
slug: /quality-and-observability/profiler/metrics
|
||||
slug: /how-to-guides/data-quality-observability/profiler/metrics
|
||||
---
|
||||
|
||||
# Metrics
|
||||
# Profiler Metrics
|
||||
|
||||
Here you can find information about the supported metrics for the different types.
|
||||
|
||||
@ -175,4 +175,4 @@ OpenMetadata will look at the previous day to fetch the operations that were per
|
||||
|
||||
## Reach out!
|
||||
|
||||
Is there any metric you'd like to see? Open an [issue](https://github.com/open-metadata/OpenMetadata/issues/new/choose) or reach out on [Slack](https://slack.open-metadata.org).
|
||||
Is there any metric you'd like to see? Open an [issue](https://github.com/open-metadata/OpenMetadata/issues/new/choose) or reach out on [Slack](https://slack.open-metadata.org).
|
||||
@ -1,13 +1,12 @@
|
||||
---
|
||||
title: Profiler Workflow
|
||||
slug: /quality-and-observability/profiler
|
||||
slug: /how-to-guides/data-quality-observability/profiler/workflow
|
||||
---
|
||||
|
||||
# Profiler Workflow
|
||||
|
||||
Learn how to configure and run the Profiler Workflow to extract Profiler data and execute the Data Quality.
|
||||
|
||||
|
||||
{% note %}
|
||||
|
||||
For Datalake Profiling, we drop NaN (Not a Number) values from the DataFrame using the dropna() method to allow metric computation. However, we make an exception for null values, which are retained. This ensures that our computations are accurate while handling missing data
|
||||
@ -25,15 +24,12 @@ This Pipeline will be in charge of feeding the Profiler tab of the Table Entity,
|
||||
caption="Table profile summary page"
|
||||
/%}
|
||||
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/profiler/profiler-summary-column.png"
|
||||
alt="Column profile summary page"
|
||||
caption="Column profile summary page"
|
||||
/%}
|
||||
|
||||
|
||||
|
||||
### 1. Add a Profiler Ingestion
|
||||
From the Service Page, go to the Ingestions tab to add a new ingestion and click on Add Profiler Ingestion.
|
||||
|
||||
@ -43,7 +39,6 @@ From the Service Page, go to the Ingestions tab to add a new ingestion and click
|
||||
caption="Add a profiler service"
|
||||
/%}
|
||||
|
||||
|
||||
### 2. Configure the Profiler Ingestion
|
||||
Here you can enter the Profiler Ingestion details.
|
||||
|
||||
@ -53,7 +48,6 @@ Here you can enter the Profiler Ingestion details.
|
||||
caption="Set profiler configuration"
|
||||
/%}
|
||||
|
||||
|
||||
#### Profiler Options
|
||||
**Name**
|
||||
Define the name of the Profiler Workflow. While we only support a single workflow for the Metadata and Usage ingestion, users can define different schedules and filters for Profiler workflows.
|
||||
@ -1,6 +1,6 @@
|
||||
---
|
||||
title: External Storage for Sample Data
|
||||
slug: /quality-and-observability/profiler/external-sample-data
|
||||
slug: /how-to-guides/data-quality-observability/profiler/external-sample-data
|
||||
---
|
||||
|
||||
# External Storage for Sample Data
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
title: Profiler and Data Quality Tab
|
||||
slug: /how-to-guides/data-quality-profiler/tab
|
||||
title: Profiler Tab
|
||||
slug: /how-to-guides/data-quality-observability/profiler/tab
|
||||
---
|
||||
|
||||
# Profiler and Data Quality Tab
|
||||
# Profiler Tab
|
||||
|
||||
The Profiler & Data Quality tab is displayed only for Tables. It has three sub-tabs for **Table Profile, Column Profile, and Data Quality**.
|
||||
|
||||
@ -126,30 +126,4 @@ The distribution of the character length inside the column is displayed to help
|
||||
src="/images/v1.4/how-to-guides/quality/dd.png"
|
||||
alt="Column Profile: Data Distribution"
|
||||
caption="Column Profile: Data Distribution"
|
||||
/%}
|
||||
|
||||
## Data Quality Tab
|
||||
|
||||
Data quality tests can be run on the sample data. We can add tests at the table and column level. The Data Quality tab displays the total number of tests that were run, and also the number of tests that were successful, aborted, or failed. The list of test cases displays the details of the table or column on which the test was run.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/how-to-guides/quality/dq1.png"
|
||||
alt="Profiler & Data Quality"
|
||||
caption="Profiler & Data Quality"
|
||||
/%}
|
||||
|
||||
You can click on a Test Case to view further details. You can use a time filter on these reports. You can also edit these tests by clicking on the pencil icon next to each test.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/how-to-guides/quality/dq2.png"
|
||||
alt="Details of a Test Case"
|
||||
caption="Details of a Test Case"
|
||||
/%}
|
||||
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="How to Write and Deploy No-Code Test Cases"
|
||||
icon="MdArrowForward"
|
||||
href="/how-to-guides/data-quality-profiler/test"%}
|
||||
Verify your data quality with table and column level tests.
|
||||
{%/inlineCallout%}
|
||||
/%}
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
title: Data Quality
|
||||
slug: /quality-and-observability/data-quality
|
||||
title: Configure Data Quality
|
||||
slug: /how-to-guides/data-quality-observability/quality/configure
|
||||
---
|
||||
|
||||
# Data Quality
|
||||
# Configure Data Quality
|
||||
|
||||
Learn how you can use OpenMetadata to define Data Quality tests and measure your data reliability.
|
||||
|
||||
@ -1,9 +1,8 @@
|
||||
---
|
||||
title: Custom Tests
|
||||
slug: /quality-and-observability/data-quality/custom-tests
|
||||
slug: /how-to-guides/data-quality-observability/quality/custom-tests
|
||||
---
|
||||
|
||||
|
||||
# Adding Custom Tests
|
||||
While OpenMetadata provides out of the box tests, you may want to write your test results from your own custom quality test suite or define your own data quality tests to be ran inside OpenMetadata. This is very easy to do using the API and our Python SDK.
|
||||
|
||||
@ -1,14 +1,12 @@
|
||||
---
|
||||
title: Data Quality and Profiler
|
||||
slug: /how-to-guides/data-quality-profiler
|
||||
title: Data Quality
|
||||
slug: /how-to-guides/data-quality-observability/quality
|
||||
---
|
||||
|
||||
# Overview of Data Quality and Profiler
|
||||
# Overview of Data Quality
|
||||
|
||||
With OpenMetadata, you can build trust in your data by creating tests to monitor that the data is complete, fresh, and accurate. OpenMetadata supports data quality tests for all of the supported database connectors. Users can run tests at the table and column levels to verify their data for business as well as technical use cases.
|
||||
|
||||
The profiler in OpenMetadata helps to understand the shape of your data and to quickly validate assumptions. The data profiler helps to capture table usage statistics over a period of time. This happens as part of profiler ingestion. Data profiles enable you to check for null values in non-null columns, for duplicates in a unique column, etc. You can gain a better understanding of column data distributions through the descriptive statistics provided.
|
||||
|
||||
OpenMetadata provides Data Quality workflows, which helps with:
|
||||
- **Native tests** for all database connectors to run assertions.
|
||||
- **Alerting system** to send notifications on test failure.
|
||||
@ -31,26 +29,40 @@ Watch the video on Data Quality Simplified to effortlessly build, deploy, monito
|
||||
|
||||
{% youtube videoId="ihwtuNHt1kI" start="0:00" end="29:08" width="560px" height="315px" /%}
|
||||
|
||||
Here's the latest on OpenMetadata's data quality.
|
||||
{% youtube videoId="UbNOje0kf6E" start="0:00" end="54:52" width="560px" height="315px" /%}
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="Profiler and Data Quality Tab"
|
||||
bold="Data Quality Tab"
|
||||
icon="MdSecurity"
|
||||
href="/how-to-guides/data-quality-profiler/tab"%}
|
||||
Get a complete picture of the Table Profile, Column Profile, and Data Quality details.
|
||||
href="/how-to-guides/data-quality-observability/quality/tab"%}
|
||||
Get a complete picture of the Data Quality details.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="Write and Deploy No-Code Test Cases"
|
||||
bold="Write and Deploy No-Code Test Cases from the UI"
|
||||
icon="MdSecurity"
|
||||
href="/how-to-guides/data-quality-profiler/test"%}
|
||||
href="/how-to-guides/data-quality-observability/quality/test"%}
|
||||
Verify your data quality with table and column level tests.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="Set Alerts for Test Case Fails"
|
||||
icon="MdSecurity"
|
||||
href="/how-to-guides/data-quality-profiler/alerts"%}
|
||||
Get notified when a data quality test fails.
|
||||
icon="MdGppGood"
|
||||
bold="Configure Data Quality"
|
||||
href="/how-to-guides/data-quality-observability/quality/configure"%}
|
||||
Configure and run data quality pipelines with the built-in tests in OpenMetadata.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdAssignmentTurnedIn"
|
||||
bold="Tests - YAML Config"
|
||||
href="/how-to-guides/data-quality-observability/quality/tests-yaml"%}
|
||||
Learn how to configure data quality tests in the YAML config file.
|
||||
{%/inlineCallout%}
|
||||
{%inlineCallout
|
||||
icon="MdOutlineDashboardCustomize"
|
||||
bold="Custom Tests"
|
||||
href="/how-to-guides/data-quality-observability/quality/custom-tests"%}
|
||||
Write your own data quality tests and test suites.
|
||||
{%/inlineCallout%}
|
||||
{%/inlineCalloutContainer%}
|
||||
@ -0,0 +1,32 @@
|
||||
---
|
||||
title: Data Quality Tab
|
||||
slug: /how-to-guides/data-quality-observability/quality/tab
|
||||
---
|
||||
|
||||
# Data Quality Tab
|
||||
|
||||
The Profiler & Data Quality tab is displayed only for Tables. It has three sub-tabs for **Table Profile, Column Profile, and Data Quality**.
|
||||
|
||||
Data quality tests can be run on the sample data. We can add tests at the table and column level. The Data Quality tab displays the total number of tests that were run, and also the number of tests that were successful, aborted, or failed. The list of test cases displays the details of the table or column on which the test was run.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/how-to-guides/quality/dq1.png"
|
||||
alt="Profiler & Data Quality"
|
||||
caption="Profiler & Data Quality"
|
||||
/%}
|
||||
|
||||
You can click on a Test Case to view further details. You can use a time filter on these reports. You can also edit these tests by clicking on the pencil icon next to each test.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/how-to-guides/quality/dq2.png"
|
||||
alt="Details of a Test Case"
|
||||
caption="Details of a Test Case"
|
||||
/%}
|
||||
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="How to Write and Deploy No-Code Test Cases from the UI"
|
||||
icon="MdArrowForward"
|
||||
href="/how-to-guides/data-quality-observability/quality/test"%}
|
||||
Verify your data quality with table and column level tests.
|
||||
{%/inlineCallout%}
|
||||
@ -1,6 +1,6 @@
|
||||
---
|
||||
title: How to Write and Deploy No-Code Test Cases
|
||||
slug: /how-to-guides/data-quality-profiler/test
|
||||
slug: /how-to-guides/data-quality-observability/quality/test
|
||||
---
|
||||
|
||||
# How to Write and Deploy No-Code Test Cases
|
||||
@ -142,12 +142,12 @@ alt="Resolved Status: Reason"
|
||||
caption="Resolved Status: Reason"
|
||||
/%}
|
||||
|
||||
Users can also set up [alerts](/how-to-guides/data-quality-profiler/alerts) to be notified when a test fails.
|
||||
Users can also set up [alerts](/how-to-guides/data-quality-observability/quality/alerts) to be notified when a test fails.
|
||||
|
||||
{%inlineCallout
|
||||
color="violet-70"
|
||||
bold="How to Set Alerts for Test Case Fails"
|
||||
icon="MdArrowForward"
|
||||
href="/how-to-guides/data-quality-profiler/alerts"%}
|
||||
href="/how-to-guides/data-quality-observability/quality/alerts"%}
|
||||
Get notified when a data quality test fails.
|
||||
{%/inlineCallout%}
|
||||
@ -1,9 +1,9 @@
|
||||
---
|
||||
title: Tests
|
||||
slug: /quality-and-observability/data-quality/tests
|
||||
title: Tests - YAML Config
|
||||
slug: /how-to-guides/data-quality-observability/quality/tests-yaml
|
||||
---
|
||||
|
||||
# Test
|
||||
# Tests in the YAML Config
|
||||
Here you can see all the supported tests definitions and how to configure them in the YAML config file.
|
||||
|
||||
A **Test Definition** is a generic definition of a test. This Test Definition then gets specified in a Test Case. This Test Case is where the parameter(s) of a Test Definition are specified.
|
||||
@ -1,33 +0,0 @@
|
||||
---
|
||||
title: How to Set Alerts for Test Case Fails
|
||||
slug: /how-to-guides/data-quality-profiler/alerts
|
||||
---
|
||||
|
||||
# How to Set Alerts for Test Case Fails
|
||||
|
||||
Users can set up alerts to be notified when data quality tests fail.
|
||||
|
||||
To set up an alert for test failures:
|
||||
- Navigate to **Settings >> Alerts**
|
||||
- Click on **Create Alert**
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/how-to-guides/quality/alert1.png"
|
||||
alt="Set up Alerts for Test Failure"
|
||||
caption="Set up Alerts for Test Failure"
|
||||
/%}
|
||||
|
||||
Enter the following details:
|
||||
- **Name:** Add a name for the alert.
|
||||
- **Description:** Describe what the laert is for.
|
||||
- **Trigger:** Uncheck the trigger for `All` and add a trigger for `Test Case`
|
||||
- **Filters:** Add filters to narrow down to the `Test Results` which `Failed`. You can also add another filter to specify the `FQN` to only include the tables that you want to consider.
|
||||
- **Destination:** Specify the destination where the test failed notification must be sent. The alerts can be sent to Email, Slack, MS Teams, Google Chat, and other Webhooks. Notifications can also be sent only to Admins, Owners and Followers of data assets.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/how-to-guides/quality/alert2.png"
|
||||
alt="Configure an Alert for Test Failure"
|
||||
caption="Configure an Alert for Test Failure"
|
||||
/%}
|
||||
|
||||
**Save** the details to create an Alert.
|
||||
@ -25,9 +25,9 @@ OpenMetadata is a complete package for data teams to break down team silos, shar
|
||||
icon="collaboration"
|
||||
/%}
|
||||
{% tile
|
||||
title="Data Quality & Profiler"
|
||||
description="Trust your data with quality tests that ensure freshness, & accuracy."
|
||||
link="/how-to-guides/data-quality-profiler"
|
||||
title="Data Quality and Observability"
|
||||
description="Trust your data with quality tests & monitor the health of your data systems."
|
||||
link="/how-to-guides/data-quality-observability"
|
||||
icon="quality"
|
||||
/%}
|
||||
{% tile
|
||||
@ -48,12 +48,6 @@ OpenMetadata is a complete package for data teams to break down team silos, shar
|
||||
link="/how-to-guides/data-governance"
|
||||
icon="governance"
|
||||
/%}
|
||||
{% tile
|
||||
title="Data Observability"
|
||||
description="Ensure the health and performance of your data systems with OpenMetadata."
|
||||
link="/how-to-guides/data-observability"
|
||||
icon="observability"
|
||||
/%}
|
||||
{% /tilesContainer %}
|
||||
|
||||
## Quick Start Guides
|
||||
@ -97,6 +91,6 @@ OpenMetadata is a complete package for data teams to break down team silos, shar
|
||||
|
||||
- Implement **[Data Governance](/how-to-guides/data-governance)** to maintain data integrity, security, and compliance.
|
||||
|
||||
- Implement **[Data Observability](/how-to-guides/data-observability)** to ensure the health and performance of your data systems.
|
||||
- Implement **[Data Observability](/how-to-guides/data-quality-observability)** to ensure the health and performance of your data systems.
|
||||
|
||||
{% /note %}
|
||||
|
||||
@ -300,14 +300,45 @@ site_menu:
|
||||
- category: How-to Guides / Data Collaboration / Overview of Knowledge Center
|
||||
url: /how-to-guides/data-collaboration/knowledge-center
|
||||
|
||||
- category: How-to Guides / Data Quality and Profiler
|
||||
url: /how-to-guides/data-quality-profiler
|
||||
- category: How-to Guides / Data Quality and Profiler / Profiler and Data Quality Tab
|
||||
url: /how-to-guides/data-quality-profiler/tab
|
||||
- category: How-to Guides / Data Quality and Profiler / How to Write and Deploy No-Code Test Cases
|
||||
url: /how-to-guides/data-quality-profiler/test
|
||||
- category: How-to Guides / Data Quality and Profiler / How to Set Alerts for Test Case Fails
|
||||
url: /how-to-guides/data-quality-profiler/alerts
|
||||
- category: How-to Guides / Data Quality and Observability
|
||||
url: /how-to-guides/data-quality-observability
|
||||
- category: How-to Guides / Data Quality and Observability / Data Quality
|
||||
url: /how-to-guides/data-quality-observability/quality
|
||||
- category: How-to Guides / Data Quality and Profiler / Data Quality / Data Quality Tab
|
||||
url: /how-to-guides/data-quality-observability/quality/tab
|
||||
- category: How-to Guides / Data Quality and Profiler / Data Quality / How to Write and Deploy No-Code Test Cases
|
||||
url: /how-to-guides/data-quality-observability/quality/test
|
||||
- category: How-to Guides / Data Quality and Observability / Data Quality / Configure Data Quality
|
||||
url: /how-to-guides/data-quality-observability/quality/configure
|
||||
- category: How-to Guides / Data Quality and Observability / Data Quality / Tests - YAML Config
|
||||
url: /how-to-guides/data-quality-observability/quality/tests-yaml
|
||||
- category: How-to Guides / Data Quality and Observability / Data Quality / Custom Tests
|
||||
url: /how-to-guides/data-quality-observability/quality/custom-tests
|
||||
- category: How-to Guides / Data Quality and Observability / Data Profiler
|
||||
url: /how-to-guides/data-quality-observability/profiler
|
||||
- category: How-to Guides / Data Quality and Profiler / Data Profiler/ Profiler Tab
|
||||
url: /how-to-guides/data-quality-observability/profiler/tab
|
||||
- category: How-to Guides / Data Quality and Observability / Data Profiler / Profiler Workflow
|
||||
url: /how-to-guides/data-quality-observability/profiler/workflow
|
||||
- category: How-to Guides / Data Quality and Observability / Data Profiler / Metrics
|
||||
url: /how-to-guides/data-quality-observability/profiler/metrics
|
||||
- category: How-to Guides / Data Quality and Observability / Data Profiler / Sample Data
|
||||
url: /how-to-guides/data-quality-observability/profiler/external-sample-data
|
||||
- category: How-to Guides / Data Quality and Observability / Data Profiler / External Workflow
|
||||
url: /how-to-guides/data-quality-observability/profiler/external-workflow
|
||||
- category: How-to Guides / Data Quality and Observability / Data Profiler / Auto PII Tagging
|
||||
url: /how-to-guides/data-quality-observability/profiler/auto-pii-tagging
|
||||
- category: How-to Guides / Data Quality and Observability / Data Observability
|
||||
url: /how-to-guides/data-quality-observability/observability
|
||||
- category: How-to Guides / Data Quality and Observability / Data Observability / Observability Alerts
|
||||
url: /how-to-guides/data-quality-observability/observability/alerts
|
||||
- category: How-to Guides / Data Quality and Observability / Incident Manager
|
||||
url: /how-to-guides/data-quality-observability/incident-manager
|
||||
- category: How-to Guides / Data Quality and Observability / Incident Manager / How to work with the Incident Manager
|
||||
url: /how-to-guides/data-quality-observability/incident-manager/workflow
|
||||
- category: How-to Guides / Data Quality and Observability / Incident Manager / Root Cause Analysis
|
||||
url: /how-to-guides/data-quality-observability/incident-manager/root-cause-analysis
|
||||
isCollateOnly: true
|
||||
|
||||
- category: How-to Guides / Data Lineage
|
||||
url: /how-to-guides/data-lineage
|
||||
@ -380,13 +411,6 @@ site_menu:
|
||||
- category: How-to Guides / Data Governance / Classification / Best Practices for Classification
|
||||
url: /how-to-guides/data-governance/classification/best-practices
|
||||
|
||||
- category: How-to Guides / Data Observability
|
||||
url: /how-to-guides/data-observability
|
||||
- category: How-to Guides / Data Observability / Incident Manager
|
||||
url: /how-to-guides/data-observability/incident-manager
|
||||
- category: How-to Guides / Data Observability / Incident Manager/ How to work with Incident Manager
|
||||
url: /how-to-guides/data-observability/incident-manager/workflow
|
||||
|
||||
- category: Releases
|
||||
url: /releases
|
||||
color: violet-70
|
||||
@ -904,37 +928,6 @@ site_menu:
|
||||
- category: Connectors / Ingestion / Best Practices
|
||||
url: /connectors/ingestion/best-practices
|
||||
|
||||
|
||||
- category: Quality & Observability
|
||||
url: /quality-and-observability
|
||||
color: violet-70
|
||||
icon: openmetadata
|
||||
|
||||
- category: Quality & Observability / Profiler
|
||||
url: /quality-and-observability/profiler
|
||||
- category: Quality & Observability / Profiler / Metrics
|
||||
url: /quality-and-observability/profiler/metrics
|
||||
- category: Quality & Observability / Profiler / Sample Data
|
||||
url: /quality-and-observability/profiler/external-sample-data
|
||||
- category: Quality & Observability / Profiler / External Workflow
|
||||
url: /quality-and-observability/profiler/external-workflow
|
||||
- category: Quality & Observability / Profiler / Auto PII Tagging
|
||||
url: /quality-and-observability/profiler/auto-pii-tagging
|
||||
- category: Quality & Observability / Data Quality
|
||||
url: /quality-and-observability/data-quality
|
||||
- category: Quality & Observability / Data Quality / Tests
|
||||
url: /quality-and-observability/data-quality/tests
|
||||
- category: Quality & Observability / Data Quality / Custom Tests
|
||||
url: /quality-and-observability/data-quality/custom-tests
|
||||
- category: Quality & Observability / Data Quality / Incident Manager
|
||||
url: /quality-and-observability/data-quality/incident-manager
|
||||
- category: Quality & Observability / Data Quality / Alerts
|
||||
url: /quality-and-observability/data-quality/alerts
|
||||
- category: Quality & Observability / Data Quality / Root Cause Analysis
|
||||
url: /quality-and-observability/data-quality/root-cause-analysis
|
||||
isCollateOnly: true
|
||||
|
||||
|
||||
- category: Main Concepts
|
||||
url: /main-concepts
|
||||
color: violet-70
|
||||
|
||||
@ -1,58 +0,0 @@
|
||||
---
|
||||
title: Incident Manager
|
||||
slug: /quality-and-observability/data-quality/incident-manager
|
||||
---
|
||||
|
||||
# Incident Manager
|
||||
|
||||
## Opening and Triagging Incidents
|
||||
In v1.1.0 we introduce the ability for user to manage and triagge incidents linked to failures. When a test case fail, it will automatically open a new incident and mark it as new. if enough information is available, OpenMetadata will automatically assign a severity to the incident - note that you can override this severity. It indicates that a new failure has happened.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-new.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
Once an incident has been open you will be able to triagge and manage it. You can perform different actions at this stage:
|
||||
- `ack`: the incident will be mark as acknoweldge, informing users that people are aware of the on going incident.
|
||||
- `assign`: the incident will be marked as assigned and a task will be opened for the assignee.
|
||||
- `resolved`: a new incident can directly be marked as resolved - see section below for more details
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-ack-form.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-ack.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
When resolving and incident a user will be required to specify the reason and add a comment. This provides context regarding the incident and helps users further understand what might have gone wrong
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-resolved-form.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/resolution-workflow-resolved.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
|
||||
|
||||
## Incidents Context & History
|
||||
|
||||
When clicking on an open incident you will different information:
|
||||
**Open Incident:** this section will show you open incidents with the timeline and any comments/collaboration that might have been happening.
|
||||
**Closed Incidents:** this section will show you incidents that have been resolved in the past with the timeline and any comments/collaboration that might have been happening and the resolution reason.
|
||||
|
||||
{% image
|
||||
src="/images/v1.4/features/ingestion/workflows/data-quality/incident-management-page.png"
|
||||
alt="Test suite results table"
|
||||
caption="Test suite results table"
|
||||
/%}
|
||||
@ -1,34 +0,0 @@
|
||||
---
|
||||
title: Quality & Observability
|
||||
slug: /quality-and-observability
|
||||
---
|
||||
|
||||
# Quality & Observability with OpenMetadata
|
||||
|
||||
OpenMetadata offers a simple and easy-to-use solution for quality and observability. With no code tests, observability metrics, incident management, and root cause analysis (Collate feature), you have a unified solution for discovery, governance, and observability.
|
||||
|
||||
## Observability Metrics
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
|
||||
{%inlineCallout
|
||||
icon="celebration"
|
||||
bold="Observability Metrics (Profiler)"
|
||||
href="/quality-and-observability/profiler" %}
|
||||
Deep dive into how to set up observability metrics in OpenMetadata!
|
||||
{%/inlineCallout%}
|
||||
|
||||
{%/inlineCalloutContainer%}
|
||||
|
||||
## Quality
|
||||
|
||||
{%inlineCalloutContainer%}
|
||||
|
||||
{%inlineCallout
|
||||
icon="celebration"
|
||||
bold="Quality"
|
||||
href="/quality-and-observability/data-quality" %}
|
||||
Deep dive into how to set up quality tests, alert and triagge and resolve incidents!
|
||||
{%/inlineCallout%}
|
||||
|
||||
{%/inlineCalloutContainer%}
|
||||
@ -17,7 +17,7 @@ version. To see what's coming in next releases, please check our [Roadmap](/rele
|
||||
# 1.4.2 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**June 10, 2024**
|
||||
**June 10th, 2024**
|
||||
{% /note %}
|
||||
|
||||
You can find the GitHub release [here](https://github.com/open-metadata/OpenMetadata/releases/tag/1.4.2-release).
|
||||
@ -55,7 +55,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.4.1 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**May 27, 2024**
|
||||
**May 27th, 2024**
|
||||
{% /note %}
|
||||
|
||||
You can find the GitHub release [here](https://github.com/open-metadata/OpenMetadata/releases/tag/1.4.1-release).
|
||||
@ -65,7 +65,7 @@ In 1.4.1, we provide migration fixes on top of the 1.4.0 release. Do check out t
|
||||
# 1.4.0 Release 🎉
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**May 21, 2024**
|
||||
**May 21th, 2024**
|
||||
|
||||
[OpenMetadata 1.4.0 Release](https://blog.open-metadata.org/openmetadata-release-1-4-0-f6fb11ec34d7)
|
||||
{% /note %}
|
||||
@ -231,7 +231,7 @@ https://www.youtube.com/watch?v=KZdVb8DiHJs - Video on Column Lineage Search
|
||||
# 1.3.4 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**May 12, 2024**
|
||||
**May 12th, 2024**
|
||||
{% /note %}
|
||||
|
||||
- Fixes reindex issues related to the `changeDescription` payload of some entities
|
||||
@ -240,7 +240,7 @@ https://www.youtube.com/watch?v=KZdVb8DiHJs - Video on Column Lineage Search
|
||||
# 1.3.3 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**April 19, 2024**
|
||||
**April 19th, 2024**
|
||||
{% /note %}
|
||||
|
||||
- Fix Application installation
|
||||
@ -306,7 +306,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.3.1 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**February 29, 2024**
|
||||
**February 29th, 2024**
|
||||
|
||||
You can find the GitHub release [here](https://github.com/open-metadata/OpenMetadata/releases/tag/1.3.1-release).
|
||||
{% /note %}
|
||||
@ -359,7 +359,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.3.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**February 5, 2024**
|
||||
**February 5th, 2024**
|
||||
|
||||
[OpenMetadata 1.3 Release - Intuitive Lineage UI, Data Observability Alerts, Data Quality Incident Manager, Custom Metrics for Profiler, Knowledge Center Improvements, and lots more](https://blog.open-metadata.org/openmetadata-release-1-3-ac801834ee80)
|
||||
{% /note %}
|
||||
@ -483,7 +483,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.2.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**October 26, 2023**
|
||||
**October 26th, 2023**
|
||||
|
||||
[OpenMetadata 1.2 Release - Domains, Data Products, Search Index, Stored Procedures, Glossary Approval Workflow, Customizable Landing Page, Applications, Knowledge Center, Cost Analysis, and lots more](https://blog.open-metadata.org/openmetadata-release-1-2-531f0e3c6d9a)
|
||||
{% /note %}
|
||||
@ -571,7 +571,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.1.2 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**August 24, 2023**
|
||||
**August 24th, 2023**
|
||||
{% /note %}
|
||||
|
||||
## Data Quality
|
||||
@ -604,7 +604,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.1.1 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**August 7, 2023**
|
||||
**August 7th, 2023**
|
||||
{% /note %}
|
||||
|
||||
## UI Improvements
|
||||
@ -641,7 +641,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.1.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**June 30, 2023**
|
||||
**June 30th, 2023**
|
||||
|
||||
[OpenMetadata 1.1.0 Release - UI Overhaul, New Connectors, Improved Lineage Parsing, PII Masking, and lots more](https://blog.open-metadata.org/openmetadata-1-1-0-release-97c1fb603bcf)
|
||||
{% /note %}
|
||||
@ -693,7 +693,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 1.0.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**April 25, 2023**
|
||||
**April 25th, 2023**
|
||||
|
||||
[OpenMetadata 1.0 Release - Improved Schemas & APIs, Ingestion Improvements, Storage Services, Dashboard Data Models, Auto PII Classification, Localization, and much more](https://blog.open-metadata.org/openmetadata-1-0-release-beb34762d916)
|
||||
{% /note %}
|
||||
@ -763,7 +763,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 0.13.3 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**March 30, 2023**
|
||||
**March 30th, 2023**
|
||||
{% /note %}
|
||||
|
||||
## Ingestion Framework
|
||||
@ -806,7 +806,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 0.13.2 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**January 30, 2023**
|
||||
**January 30th, 2023**
|
||||
|
||||
[OpenMetadata 0.13.2 Release - Improved SQL Lineage, Glossary Bulk Upload, Unified Tag Category API, Mutually Exclusive Tags, Chrome Extension, and lots more](https://blog.open-metadata.org/openmetadata-0-13-2-release-e32c0de93361)
|
||||
{% /note %}
|
||||
@ -846,7 +846,7 @@ You can find the GitHub release [here](https://github.com/open-metadata/OpenMeta
|
||||
# 0.13.1 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**December 20, 2022**
|
||||
**December 20th, 2022**
|
||||
{% /note %}
|
||||
|
||||
## Profiler and Data Quality
|
||||
@ -875,7 +875,7 @@ The logic for Notification Support has been improved. Users can define Alerts ba
|
||||
# 0.13.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**December 8, 2022**
|
||||
**December 8th, 2022**
|
||||
|
||||
[OpenMetadata 0.13.0 Release — Data Insights & KPIs, Lineage Traceability, Data Lake Profiler, Search Improvements, and lots more](https://blog.open-metadata.org/openmetadata-0-13-0-release-ac8ac5bd87c1)
|
||||
{% /note %}
|
||||
@ -913,7 +913,7 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan
|
||||
# 0.12.3 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**November 18, 2022**
|
||||
**November 18th, 2022**
|
||||
{% /note %}
|
||||
|
||||
## Bug Fixes
|
||||
@ -923,7 +923,7 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan
|
||||
# 0.12.2 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**October 20, 2022**
|
||||
**October 20th, 2022**
|
||||
{% /note %}
|
||||
|
||||
## Ingestion
|
||||
@ -935,7 +935,7 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan
|
||||
# 0.12.1 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**October 3, 2022**
|
||||
**October 3rd, 2022**
|
||||
{% /note %}
|
||||
|
||||
## Basic Authentication
|
||||
@ -967,7 +967,7 @@ Major enhancements have been made to how data is extracted from Kafka and Redpan
|
||||
# 0.12.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**September 7, 2022**
|
||||
**September 7th, 2022**
|
||||
|
||||
[OpenMetadata 0.12.0 Release](https://blog.open-metadata.org/openmetadata-0-12-0-release-1ac059700de4)
|
||||
{% /note %}
|
||||
@ -1047,7 +1047,7 @@ Manage Tab has been replaced with the manage button on the UI.
|
||||
# 0.11.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**July 1, 2022**
|
||||
**July 1st, 2022**
|
||||
|
||||
[OpenMetadata 0.11.0 Release](https://blog.open-metadata.org/openmetadata-0-11-release-8b82c85636a)
|
||||
{% /note %}
|
||||
@ -1127,7 +1127,7 @@ Manage Tab has been replaced with the manage button on the UI.
|
||||
# 0.10.1 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**May 17, 2022**
|
||||
**May 17th, 2022**
|
||||
{% /note %}
|
||||
|
||||
- Support for Postgres as OpenMetadata Store [#4601](https://github.com/open-metadata/OpenMetadata/issues/4601)
|
||||
@ -1139,7 +1139,7 @@ Manage Tab has been replaced with the manage button on the UI.
|
||||
# 0.10.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**April 27, 2022**
|
||||
**April 27th, 2022**
|
||||
|
||||
[OpenMetadata 0.10.0 Release](https://blog.open-metadata.org/openmetadata-0-10-0-release-82c4f5533c3f)
|
||||
{% /note %}
|
||||
@ -1210,7 +1210,7 @@ and prepares handy methods to help us test the connection to the source before c
|
||||
# 0.9.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**March 10, 2022**
|
||||
**March 10th, 2022**
|
||||
|
||||
[OpenMetadata 0.9.0 Release](https://blog.open-metadata.org/openmetadata-0-9-0-release-8e7b93ab1882)
|
||||
{% /note %}
|
||||
@ -1268,7 +1268,7 @@ and prepares handy methods to help us test the connection to the source before c
|
||||
# 0.8.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**January 22, 2022**
|
||||
**January 22nd, 2022**
|
||||
|
||||
[OpenMetadata 0.8.0 Release](https://blog.open-metadata.org/openmetadata-0-8-0-release-ca09bd2fbf54)
|
||||
{% /note %}
|
||||
@ -1292,7 +1292,7 @@ and prepares handy methods to help us test the connection to the source before c
|
||||
# 0.7.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**November 17, 2021**
|
||||
**November 17th, 2021**
|
||||
|
||||
[OpenMetadata 0.7.0 Release](https://blog.open-metadata.org/openmetadata-0-7-0-release-9f741b8d5089)
|
||||
{% /note %}
|
||||
@ -1325,7 +1325,7 @@ and prepares handy methods to help us test the connection to the source before c
|
||||
# 0.6.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**November 17, 2021**
|
||||
**November 17th, 2021**
|
||||
|
||||
[OpenMetadata 0.6.0 Release — Metadata Versioning, Events API, One-Click Ingestion, and more](https://blog.open-metadata.org/openmetadata-0-6-0-release-metadata-versioning-events-api-one-click-ingestion-and-more-4394c4f08e0b)
|
||||
{% /note %}
|
||||
@ -1350,7 +1350,7 @@ and prepares handy methods to help us test the connection to the source before c
|
||||
# 0.5.0 Release
|
||||
|
||||
{% note noteType="Tip" %}
|
||||
**October 19, 2021**
|
||||
**October 19th, 2021**
|
||||
|
||||
[OpenMetadata 0.5.0 Release is here — Lineage, Pipelines, Complex Types, Data Profiler and so much more](https://blog.open-metadata.org/openmetadata-0-5-0-1144a4000644)
|
||||
{% /note %}
|
||||
@ -1378,7 +1378,7 @@ and prepares handy methods to help us test the connection to the source before c
|
||||
|
||||
# 0.4.0 Release
|
||||
{% note noteType="Tip" %}
|
||||
**September 20, 2021**
|
||||
**September 20th, 2021**
|
||||
|
||||
[OpenMetadata 0.4.0 Release — Dashboards, Topics, Data Reliability](https://blog.open-metadata.org/openmetadata-0-4-0-release-dashboards-topics-data-reliability-14e8672ae0f5)
|
||||
{% /note %}
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user