diff --git a/openmetadata-docs/content/v1.6.x/collate-menu.md b/openmetadata-docs/content/v1.6.x/collate-menu.md index 13d4a626862..909b26efbaf 100644 --- a/openmetadata-docs/content/v1.6.x/collate-menu.md +++ b/openmetadata-docs/content/v1.6.x/collate-menu.md @@ -826,6 +826,8 @@ site_menu: url: /how-to-guides/data-governance/workflows/elements/nodes/set-glossary-term-status - category: How-to Guides / Data Governance / Workflows / How to Edit a Workflow url: /how-to-guides/data-governance/workflows/how-to-edit-workflow + - category: How-to Guides / Data Governance / Metrics + url: /how-to-guides/data-governance/metrics - category: Enable Security url: /security diff --git a/openmetadata-docs/content/v1.6.x/how-to-guides/data-governance/metrics.md b/openmetadata-docs/content/v1.6.x/how-to-guides/data-governance/metrics.md new file mode 100644 index 00000000000..b8acc32828e --- /dev/null +++ b/openmetadata-docs/content/v1.6.x/how-to-guides/data-governance/metrics.md @@ -0,0 +1,121 @@ +--- +title: Metrics for OpenMetadata +slug: /how-to-guides/data-governance/metrics +--- + +# Metrics in OpenMetadata + +The **Metrics** entity in OpenMetadata allows users to define, track, and manage key business and operational metrics. Metrics help organizations maintain consistency, traceability, and accuracy in data-driven decision-making. + +## Overview of Metrics + +Metrics represent calculated values based on data assets and are categorized under the **Governance** section in OpenMetadata. Each metric can be linked to glossary terms, data assets, and other metrics, providing comprehensive visibility into data quality and usage. + +## Key Properties of a Metric + +| **Property** | **Description** | **Example Value** | +|---------------------|---------------------------------------------------------------------------|-------------------------------| +| **Name** | Unique identifier for the metric, following camelCase naming conventions. | `customerRetentionRate` | +| **Display Name** | Human-readable name for the metric. | `Customer Retention Rate` | +| **Description** | Detailed explanation of what the metric represents. | `Percentage of retained users` | +| **Expression** | Formula or SQL query used to calculate the metric. | `COUNT(returning_customers)` | +| **Granularity** | Time scale for the metric, such as daily, weekly, or monthly. | `Daily` | +| **Metric Type** | Type of calculation applied to the metric (e.g., count, average, ratio). | `Percentage` | +| **Unit of Measurement** | Unit for interpreting metric values, such as count, dollars, or percentage. | `Percentage` | +| **SQL Query** | Optional SQL query defining the metric. | `SELECT COUNT(*) FROM sales` | +| **Owner** | Individual or team responsible for maintaining the metric. | `Data Governance Team` | + +## Metric Lineage and Dependencies + +OpenMetadata allows users to trace the source and dependencies of metrics using lineage. This ensures end-to-end traceability from raw data to metric reporting. A typical lineage might look like: + +```commandline +Database Table → Metric → Pipeline → Dashboard +``` + +Users can view associated tables, pipelines, and dashboards to understand how metrics are generated and utilized. + +## Creating Metrics Using the UI + +To create a new metric in OpenMetadata using the user interface, follow these steps: + +### 1. Navigate to the Metrics Section + +- Go to **Govern > Metrics** in the OpenMetadata UI. + +{% image +src="/images/v1.6/how-to-guides/governance/metrics/metrics-1.png" +alt="Navigate to the Metrics Section" +caption="Navigate to the Metrics Section" +/%} + +### 2. Add a New Metric + +- Click on **Add Metric** to initiate the metric creation process. + +{% image +src="/images/v1.6/how-to-guides/governance/metrics/metrics-2.png" +alt="Add a New Metric" +caption="Add a New Metric" +/%} + +### 3. Enter Metric Details + +Provide the required information, including: + +- **Metric Name** +- **Description** +- **Granularity** (time scale of the metric) +- **Metric Type** (e.g., count, average, ratio) +- **Computation Code** (SQL, Python, or Java) if applicable + +### 4. Create the Metric + +- After entering the details, click **Create** to finalize the metric. + +{% image +src="/images/v1.6/how-to-guides/governance/metrics/metrics-3.png" +alt="Create the Metric" +caption="Create the Metric" +/%} + +### 5. View the Created Metric + +- The newly created metric will now be available in the **Metrics** page for reference and further use. + +{% image +src="/images/v1.6/how-to-guides/governance/metrics/metrics-4.png" +alt="View the Created Metric" +caption="View the Created Metric" +/%} + +## Example JSON Schema for Metric + +```json +{ + "id": "123e4567-e89b-12d3-a456-426614174000", + "name": "customerRetentionRate", + "displayName": "Customer Retention Rate", + "description": "Percentage of customers retained over a given period.", + "formula": "COUNT(returning_customers) / COUNT(total_customers) * 100", + "sql": "SELECT COUNT(*) FROM customer_activity WHERE status='active'", + "granularity": "Monthly", + "metricType": "Percentage", + "unit": "Percentage", + "owner": "Data Governance Team", + "tags": ["Customer", "KPI", "Retention"] +} +``` + +## Managing Metrics in OpenMetadata + +- **Versioning**: Each update to a metric creates a new version, maintaining historical changes. +- **Linking**: Metrics can be linked to glossary terms, tables, dashboards, and pipelines for enriched context. +- **Monitoring**: Metrics can be monitored for value changes, enabling trend analysis over time + +## Best Practices for Metric Management + +- **Consistent Naming**: Use camelCase for metric names to ensure consistency across systems. +- **Clear Definitions**: Provide comprehensive descriptions and units for accurate interpretation. +- **Lineage Tracking**: Always associate metrics with source tables and pipelines for traceability. +- **Ownership**: Assign metric owners for accountability and maintenance. diff --git a/openmetadata-docs/content/v1.6.x/menu.md b/openmetadata-docs/content/v1.6.x/menu.md index 62b0f7647e9..69cd99018cf 100644 --- a/openmetadata-docs/content/v1.6.x/menu.md +++ b/openmetadata-docs/content/v1.6.x/menu.md @@ -951,6 +951,8 @@ site_menu: url: /how-to-guides/data-governance/domains-&-data-products/domains - category: How-to Guides / Data Governance / Domains & Data Product / How to Use Data Products url: /how-to-guides/data-governance/domains-&-data-products/data-products + - category: How-to Guides / Data Governance / Metrics + url: /how-to-guides/data-governance/metrics - category: Releases url: /releases diff --git a/openmetadata-docs/content/v1.7.x-SNAPSHOT/collate-menu.md b/openmetadata-docs/content/v1.7.x-SNAPSHOT/collate-menu.md index a9f03c4af18..bf3168667c8 100644 --- a/openmetadata-docs/content/v1.7.x-SNAPSHOT/collate-menu.md +++ b/openmetadata-docs/content/v1.7.x-SNAPSHOT/collate-menu.md @@ -804,6 +804,8 @@ site_menu: url: /how-to-guides/data-governance/domains-&-data-products/domains - category: How-to Guides / Data Governance / Domains & Data Product / How to Use Data Products url: /how-to-guides/data-governance/domains-&-data-products/data-products + - category: How-to Guides / Data Governance / Metrics + url: /how-to-guides/data-governance/metrics - category: Enable Security url: /security diff --git a/openmetadata-docs/content/v1.7.x-SNAPSHOT/how-to-guides/data-governance/metrics.md b/openmetadata-docs/content/v1.7.x-SNAPSHOT/how-to-guides/data-governance/metrics.md new file mode 100644 index 00000000000..25083ba5cf6 --- /dev/null +++ b/openmetadata-docs/content/v1.7.x-SNAPSHOT/how-to-guides/data-governance/metrics.md @@ -0,0 +1,121 @@ +--- +title: Metrics for OpenMetadata +slug: /how-to-guides/data-governance/metrics +--- + +# Metrics in OpenMetadata + +The **Metrics** entity in OpenMetadata allows users to define, track, and manage key business and operational metrics. Metrics help organizations maintain consistency, traceability, and accuracy in data-driven decision-making. + +## Overview of Metrics + +Metrics represent calculated values based on data assets and are categorized under the **Governance** section in OpenMetadata. Each metric can be linked to glossary terms, data assets, and other metrics, providing comprehensive visibility into data quality and usage. + +## Key Properties of a Metric + +| **Property** | **Description** | **Example Value** | +|---------------------|---------------------------------------------------------------------------|-------------------------------| +| **Name** | Unique identifier for the metric, following camelCase naming conventions. | `customerRetentionRate` | +| **Display Name** | Human-readable name for the metric. | `Customer Retention Rate` | +| **Description** | Detailed explanation of what the metric represents. | `Percentage of retained users` | +| **Expression** | Formula or SQL query used to calculate the metric. | `COUNT(returning_customers)` | +| **Granularity** | Time scale for the metric, such as daily, weekly, or monthly. | `Daily` | +| **Metric Type** | Type of calculation applied to the metric (e.g., count, average, ratio). | `Percentage` | +| **Unit of Measurement** | Unit for interpreting metric values, such as count, dollars, or percentage. | `Percentage` | +| **SQL Query** | Optional SQL query defining the metric. | `SELECT COUNT(*) FROM sales` | +| **Owner** | Individual or team responsible for maintaining the metric. | `Data Governance Team` | + +## Metric Lineage and Dependencies + +OpenMetadata allows users to trace the source and dependencies of metrics using lineage. This ensures end-to-end traceability from raw data to metric reporting. A typical lineage might look like: + +```commandline +Database Table → Metric → Pipeline → Dashboard +``` + +Users can view associated tables, pipelines, and dashboards to understand how metrics are generated and utilized. + +## Creating Metrics Using the UI + +To create a new metric in OpenMetadata using the user interface, follow these steps: + +### 1. Navigate to the Metrics Section + +- Go to **Govern > Metrics** in the OpenMetadata UI. + +{% image +src="/images/v1.7/how-to-guides/governance/metrics/metrics-1.png" +alt="Navigate to the Metrics Section" +caption="Navigate to the Metrics Section" +/%} + +### 2. Add a New Metric + +- Click on **Add Metric** to initiate the metric creation process. + +{% image +src="/images/v1.7/how-to-guides/governance/metrics/metrics-2.png" +alt="Add a New Metric" +caption="Add a New Metric" +/%} + +### 3. Enter Metric Details + +Provide the required information, including: + +- **Metric Name** +- **Description** +- **Granularity** (time scale of the metric) +- **Metric Type** (e.g., count, average, ratio) +- **Computation Code** (SQL, Python, or Java) if applicable + +### 4. Create the Metric + +- After entering the details, click **Create** to finalize the metric. + +{% image +src="/images/v1.7/how-to-guides/governance/metrics/metrics-3.png" +alt="Create the Metric" +caption="Create the Metric" +/%} + +### 5. View the Created Metric + +- The newly created metric will now be available in the **Metrics** page for reference and further use. + +{% image +src="/images/v1.7/how-to-guides/governance/metrics/metrics-4.png" +alt="View the Created Metric" +caption="View the Created Metric" +/%} + +## Example JSON Schema for Metric + +```json +{ + "id": "123e4567-e89b-12d3-a456-426614174000", + "name": "customerRetentionRate", + "displayName": "Customer Retention Rate", + "description": "Percentage of customers retained over a given period.", + "formula": "COUNT(returning_customers) / COUNT(total_customers) * 100", + "sql": "SELECT COUNT(*) FROM customer_activity WHERE status='active'", + "granularity": "Monthly", + "metricType": "Percentage", + "unit": "Percentage", + "owner": "Data Governance Team", + "tags": ["Customer", "KPI", "Retention"] +} +``` + +## Managing Metrics in OpenMetadata + +- **Versioning**: Each update to a metric creates a new version, maintaining historical changes. +- **Linking**: Metrics can be linked to glossary terms, tables, dashboards, and pipelines for enriched context. +- **Monitoring**: Metrics can be monitored for value changes, enabling trend analysis over time + +## Best Practices for Metric Management + +- **Consistent Naming**: Use camelCase for metric names to ensure consistency across systems. +- **Clear Definitions**: Provide comprehensive descriptions and units for accurate interpretation. +- **Lineage Tracking**: Always associate metrics with source tables and pipelines for traceability. +- **Ownership**: Assign metric owners for accountability and maintenance. diff --git a/openmetadata-docs/content/v1.7.x-SNAPSHOT/menu.md b/openmetadata-docs/content/v1.7.x-SNAPSHOT/menu.md index 664d18b91f0..b0548084280 100644 --- a/openmetadata-docs/content/v1.7.x-SNAPSHOT/menu.md +++ b/openmetadata-docs/content/v1.7.x-SNAPSHOT/menu.md @@ -959,6 +959,8 @@ site_menu: url: /how-to-guides/data-governance/domains-&-data-products/domains - category: How-to Guides / Data Governance / Domains & Data Product / How to Use Data Products url: /how-to-guides/data-governance/domains-&-data-products/data-products + - category: How-to Guides / Data Governance / Metrics + url: /how-to-guides/data-governance/metrics - category: Releases url: /releases diff --git a/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-1.png b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-1.png new file mode 100644 index 00000000000..4007a490c0f Binary files /dev/null and b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-1.png differ diff --git a/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-2.png b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-2.png new file mode 100644 index 00000000000..adea8e0e92a Binary files /dev/null and b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-2.png differ diff --git a/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-3.png b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-3.png new file mode 100644 index 00000000000..943b32d0392 Binary files /dev/null and b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-3.png differ diff --git a/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-4.png b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-4.png new file mode 100644 index 00000000000..968d5d6a6d3 Binary files /dev/null and b/openmetadata-docs/images/v1.6/how-to-guides/governance/metrics/metrics-4.png differ diff --git a/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-1.png b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-1.png new file mode 100644 index 00000000000..4007a490c0f Binary files /dev/null and b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-1.png differ diff --git a/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-2.png b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-2.png new file mode 100644 index 00000000000..adea8e0e92a Binary files /dev/null and b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-2.png differ diff --git a/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-3.png b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-3.png new file mode 100644 index 00000000000..943b32d0392 Binary files /dev/null and b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-3.png differ diff --git a/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-4.png b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-4.png new file mode 100644 index 00000000000..968d5d6a6d3 Binary files /dev/null and b/openmetadata-docs/images/v1.7/how-to-guides/governance/metrics/metrics-4.png differ