Data insights Docs (#13509)

* Data Insights Docs

* Data Insights Docs

openmetadata-docs/content/v1.1.x/how-to-guides/openmetadata/data-insights/data-culture.md

@@ -0,0 +1,86 @@
+---
+title: How to Transform the Data Culture of Your Company
+slug: /how-to-guides/openmetadata/data-insights/data-culture
+---
+
+# How to Transform the Data Culture of Your Company
+
+## What is Data Culture?
+
+Data Culture is a shared belief in the organization to use data to improve decision making and performance. It has essentially three important characteristics:
+- People are empowered to use data. 
+- Data is prioritized in decision making. 
+- Data assets are managed as products.
+
+## Why Data Culture is Important?
+
+It is observed that data-driven organizations experience above-market growth, leading to increased revenue, profitability, and operational efficiency. In order to fully realize the benefit of a data-driven organization, data culture plays a crucial role. Clear ownership of data, customer sensitivity and keeping the data high quality and reliable should be part of the data culture. It cannot be an adhoc reactive measure. Long term data quality solutions require a strong data culture.
+
+Data is a shared responsibility of the organization and requires an end-to-end approach. Data producers and consumers can together work towards better documentation and classification. Here, collaboration is key to solving issues and to improve data. Also, an organization needs to have a clear understanding of where they are with data. So they can set clear goals, achieve those goals, and measure success.
+
+## Key Aspects of Data Culture
+
+### 1. Data Needs Clear Ownership
+
+All important data must be owned. Individuals should not own important data assets. Team ownership is preffered over User ownership. It also pushes the data responsibility to a team instead of an individual user.
+
+### 2. Measure What Matters
+
+You cannot improve what you do not measure. It is important to set metrics to understand how your data is doing. Based on that, you can set goals towards improving your data.
+
+### 3. Treat Data as a Product
+
+- Maintain clear documentation of data with SLA and guarantees. 
+- Understand your user and use cases and build the data accordingly.
+- Continuously improve data with discipline.
+- Track user feedback using surveys.
+
+### 4 Reduce Toil with Integrated Tools and Automation
+
+Too many tools around specific use cases clutter the data landscape. It is better to use integrated tools that handle multiple scenarios like data discovery, collaboration, lineage, quality, etc. Automated data quality tools help to ensure updated and fresh data. Tools that identify the Tier 5 or least important data help to declutter with automted data deletion.
+
+### 5 Organize for Data
+
+Data does not come free. You need resources the teams to keep the data of high quality, reliable, and trustworthy. Decentralize data for end-to-end domain based data ownership.
+
+## How OpenMetadata Helps Enhance the Data Culture
+In order to enhance the data culture of a company, data need to be Trusted, Documented and Discoverable across the organization. OpenMetadata is an all-in-one platform for data discovery, collaboration, quality, governance, observability, lineage, glossary, and much more. Alongwith ensuring reliable quality of your data, you can use the collaborative features to maintain proper documentation, ownership, and appropriate tiering of your data assets.
+
+### 1. Centralize your Metadata in OpenMetadata
+
+OpenMetadata helps to understand your data landscape. It captures all your metadata in a single place. It is a collaborative tool for both technical and business users.
+
+### 2. Set KPIs to Drive Data Ownership
+
+The data insights feature allows you to set up KPIs using time-based goals to track ownership. Goal-based tasks can be set up for different teams. You can claim data asset ownership in OpenMetadata.
+
+### 3. Set KPIs to Drive Documentation
+
+Data without description is hard to use, resulting in the loss of productivity. Similarly, invalid or missing descriptions result in poor data outcomes. Good descriptions help to discover data assets quickly. You can set up KPIs with a specific goal to cover data documentation.
+
+### 4. Develop Data Vocabulary
+
+Data vocabulary helps in the consistent understanding of data. In OpenMetdata, using the [Glossary](/how-to-guides/openmetadata/data-governance/glossary-classification) feature, you can describe business terms and concepts in a single place. Also, the data assets can be labelled using these glossary terms in order to provide semantic meaning.
+
+### 5 Identify Important Data with Tiers
+
+Tiering is an important concept of data classification in OpenMetadata. Using [Tiers](/how-to-guides/openmetadata/data-governance/glossary-classification/tiers), data producers or owners can define the importance of data to an organization.
+
+In case of tiering, it is easiest to start with the most important (Tier 1) and the least important (Tier 5) data. Once the **Tier 1** or most important data is identified, organizations can focus on improving the descriptions and data quality. The Data Insights in OpenMetadata helps identify the unused datasets as **Tier 5**. The Tier 5 datasets can be deleted periodically to declutter.
+
+### 6 Provide Feedback to Teams
+
+OpenMetadata provides continuous feedback by way of weekly reports.  The detailed reports help to track progress over time. It keeps the leaders well informed. It helps to recognize the teams that are doing well.
+
+### 7 Use OpenMetadata Browser Extension
+
+By using the Chrome browser extension, users can consume the metadata in the tools of their choice. It provides consistent understanding of metadata at their fingertips, and helps improve productivity.
+
+### 8 Data as a Product in OpenMetadata
+
+OpenMetadata helps customers understand their data with a 360° view. Admins can set up sample data, table and column profiling for the important data assets. Data quality is important and it is a shared responsibility in the organization. Admins can set up data quality tests in OpenMetadata to detect and fix the issues early on. Both data producers and consumers can collaborate to capture assumptions about data and set up tests accordingly.
+
+Go ahead, leverage Data Insights to transform the data culture of your organization!
+Watch the video to learn more about proactively honing the data culture of your company by setting targets, monitoring, and boosting teams to accomplish data goals with OpenMetadata.
+
+{% youtube videoId="lOQepnTdA58" start="0:00" end="58:23" /%}

openmetadata-docs/content/v1.1.x/how-to-guides/openmetadata/data-insights/index.md

@@ -15,4 +15,42 @@ Watch the video to learn more about proactively honing the data culture of your
 
 Watch a demo of Data Insights in OpenMetadata
 
-{% youtube videoId="Epd9G6igLUM" start="0:00" end="21:58" /%}
+{% youtube videoId="Epd9G6igLUM" start="0:00" end="21:58" /%}
+
+{%inlineCalloutContainer%}
+ {%inlineCallout
+  color="violet-70"
+  bold="What is Tiering"
+  icon="MdInsights"
+  href="/how-to-guides/openmetadata/data-insights/tiering"%}
+  Set business importance of data using Tiers.
+ {%/inlineCallout%}
+ {%inlineCallout
+  color="violet-70"
+  bold="Set Up Data Insights Ingestion"
+  icon="MdInsights"
+  href="/how-to-guides/openmetadata/data-insights/ingestion"%}
+  Set up the ingestion pipeline right from the UI.
+ {%/inlineCallout%}
+ {%inlineCallout
+  color="violet-70"
+  bold="Key Performance Indicators (KPI)"
+  icon="MdInsights"
+  href="/how-to-guides/openmetadata/data-insights/kpi"%}
+  Define the KPIs and set goals for documentation, and ownership.
+ {%/inlineCallout%}
+ {%inlineCallout
+  color="violet-70"
+  bold="Data Insights Report"
+  icon="MdInsights"
+  href="/how-to-guides/openmetadata/data-insights/report"%}
+  Get a quick glance of data asset description, ownership, and tiering coverage.
+ {%/inlineCallout%}
+ {%inlineCallout
+  color="violet-70"
+  bold="How to Transform the Data Culture of Your Company"
+  icon="MdInsights"
+  href="/how-to-guides/openmetadata/data-insights/data-culture"%}
+  Improve your data culture for data-driven decision making.
+ {%/inlineCallout%}
+{%/inlineCalloutContainer%}

openmetadata-docs/content/v1.1.x/how-to-guides/openmetadata/data-insights/ingestion.md

@@ -0,0 +1,51 @@
+---
+title: Set Up Data Insights Ingestion 
+slug: /how-to-guides/openmetadata/data-insights/ingestion
+---
+
+# Set Up Data Insights Ingestion
+
+Admin users can set up a data insights ingestion pipeline right from the OpenMetadata UI.
+
+- Navigate to **Settings >> OpenMetadata >> Data Insights**.
+- Click on **Add Data Insights Ingestion**.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/di1.png"
+alt="Set Up Data Insights Ingestion"
+caption="Set Up Data Insights Ingestion"
+/%}
+
+- A default name is generated for the ingestion pipeline. You can leave it as it is or edit the name as required.
+- You can choose to enable the Debug Log.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/di2.png"
+alt="Set Up Data Insights Ingestion"
+caption="Set Up Data Insights Ingestion"
+/%}
+
+- Choose a schedule execution time for your workflow. The schedule time is displayed in UTC. We recommend running this workflow overnight or when activity on the platform is at its lowest to ensure accurate data. It is scheduled to run daily.
+- Click on **Add & Deploy**.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/di3.png"
+alt="Set Up Data Insights Ingestion Schedule"
+caption="Set Up Data Insights Ingestion Schedule"
+/%}
+
+{% image
+src="/images/v1.1/how-to-guides/insights/di4.png"
+alt="Data Insights Ingestion Created and Deployed"
+caption="Data Insights Ingestion Created and Deployed"
+/%}
+
+Navigate to the Insights page. You should see your [Data Insights Reports](/how-to-guides/openmetadata/data-insights/report). Note that if you have just deployed OpenMetadata, App Analytics data might not be present. App Analytics data is fetched from the previous day (UTC).
+
+{%inlineCallout
+  color="violet-70"
+  bold="Key Performance Indicators (KPI)"
+  icon="MdArrowForward"
+  href="/how-to-guides/openmetadata/data-insights/kpi"%}
+  Define the KPIs and set goals for documentation, and ownership.
+{%/inlineCallout%}

openmetadata-docs/content/v1.1.x/how-to-guides/openmetadata/data-insights/kpi.md

@@ -0,0 +1,61 @@
+---
+title: Key Performance Indicators (KPI)
+slug: /how-to-guides/openmetadata/data-insights/kpi
+---
+
+# Key Performance Indicators (KPI)
+
+Admins can define the Key Performance Indicators (KPIs) and set goals within OpenMetadata to work towards **better documentation, ownership, and tiering**. These goals are based on data assets and driven to achieve targets within a specified timeframe. For example, Admins can set goals to have at least 60% of the entities documented, owned and tiered by the end of Q4 2023.
+
+The data insights feature in OpenMetadata helps organizations to decentralize documentation and ownership of data assets. Organizations can drive the adoption of OpenMetadata by setting up company-wide KPIs to track the documentation, ownership, and tiering of data assets.
+
+## KPI Categories
+
+OpenMetadata currently supports the following KPI Categories.
+
+**Completed Description:** This KPI measures the description coverage of your data assets in OpenMetadata. You can choose an absolute (number) or a relative (percentage) value.
+
+**Completed Ownership:** This KPI measures the ownership coverage of your data assets in OpenMetadata. You can choose an absolute (number) or a relative (percentage) value.
+
+## How to Add KPIs
+
+When OpenMetadata is set up with data ingestion from third party sources, the details on the description, ownership, and tiering are also brought into OpenMetadata. You can track the existing documentation and ownership coverage and work towards a better data culture by setting up data insights. Configure the KPIs and set goals at an organizational level to encourage your team to get your data to a better state.
+
+To add KPIs:
+- Navigate to **Insights** and click on **Add KPI**.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/kpi1.png"
+alt="Add a KPI"
+caption="Add a KPI"
+/%}
+
+- Enter the following details on the KPI configuration page:
+  - **Select a Chart** among the available chart options.
+  - Enter a **Display Name**.
+  - Select the **Metric Type**, i.e., Percentage or Number. You can choose an absolute number or define a relative percentage of the data assets to be covered.
+  - Select a **Start and End Date** by which to achieve the KPI target.
+  - Add a **Description** to define what the KPI is about.
+- Click on **Submit**.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/kpi2.png"
+alt="Details of the KPI"
+caption="Details of the KPI"
+/%}
+
+{% image
+src="/images/v1.1/how-to-guides/insights/kpi3.png"
+alt="Ownership Coverage KPI Added"
+caption="Ownership Coverage KPI Added"
+/%}
+
+The line graph represents the progress made on a daily basis. It also displays the days left to achieve the target and the coverage so far.
+
+{%inlineCallout
+  color="violet-70"
+  bold="Data Insights Report"
+  icon="MdArrowForward"
+  href="/how-to-guides/openmetadata/data-insights/report"%}
+  Get a quick glance of data asset description, ownership, and tiering coverage.
+{%/inlineCallout%}

openmetadata-docs/content/v1.1.x/how-to-guides/openmetadata/data-insights/report.md

@@ -0,0 +1,143 @@
+---
+title: Data Insights Report
+slug: /how-to-guides/openmetadata/data-insights/report
+---
+
+# Data Insights Report
+
+The data insights report provides a quick glance at aspects like data ownership, description coverage, data tiering, and so on. Admins can view the aggregated user activity and get insights into user engagement and user growth. Admins can check for Daily active users and know how the tool is being used.
+
+OpenMetadata offers a suite of reports providing platform analytics around specific areas. The reports are available in three sections:
+- Data Assets
+- App Analytics
+- KPIs
+
+{% image
+src="/images/v1.1/how-to-guides/insights/insights1.png"
+alt="Data Insights Report"
+caption="Data Insights Report"
+/%}
+
+All the reports can be filtered by **Teams, Data Tiers, and a Time Filter**.
+{% image
+src="/images/v1.1/how-to-guides/insights/insights2.png"
+alt="Data Insights Report Filters: Team, Tier, Time"
+caption="Data Insights Report Filters: Team, Tier, Time"
+/%}
+
+## Data Assets Report
+The Data Asset reports display important metrics around your data assets in OpenMetadata. This report also displays the organizational health at a glance with details on the Total Data Assets, Data Assets with Description, Owners, and Tiers.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/ohg.png"
+alt="Organization Health at a Glance"
+caption="Organization Health at a Glance"
+/%}
+
+### Total Data Assets
+
+This chart represents the total number of data assets present in OpenMetadata. It offers a view of your data assets broken down by asset type (i.e. DatabaseSchema, Database, Dashboard, Chart, Topic, ML Model, etc.)
+
+{% image
+src="/images/v1.1/how-to-guides/insights/tda.png"
+alt="Total Data Assets"
+caption="Total Data Assets"
+/%}
+
+### Percentage of Data Assets with Description
+
+It displays the percentage of data assets with description by data asset type.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/pdad.png"
+alt="Percentage of Data Assets with Description"
+caption="Percentage of Data Assets with Description"
+/%}
+
+### Percentage of Data Assets with Owners
+
+This chart represents the percentage of data assets present in OpenMetadata with an owner assigned. Data assets that do not support assigning an owner will not be counted in this percentage. It allows you to quickly view the ownership coverage for your data assets in OpenMetadata.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/pdao.png"
+alt="Percentage of Data Assets with Owners"
+caption="Percentage of Data Assets with Owners"
+/%}
+
+### Total Data Assets by Tier
+
+It displays a broken down view of data assets by Tiers. Data Assets with no tiers assigned are not included in this. It allows you to quickly view the breakdown of data assets by tier.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/tdat.png"
+alt="Total Data Assets by Tier"
+caption="Total Data Assets by Tier"
+/%}
+
+## App Analytics
+
+App analytics helps to track user engagement. This report provides important metrics around the usage of OpenMetadata. This report also displays the organizational health at a glance with details on the Page Views by Data Assets, Daily Active Users on the Platform, and the Most Active User.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/ohg2.png"
+alt="Organization Health at a Glance"
+caption="Organization Health at a Glance"
+/%}
+
+### Most Viewed Data Assets
+
+Know the 10 most viewed data assets in your platform. It offers a quick view to identify the data assets of the most interest in your organization.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/mvda.png"
+alt="Most Viewed Data Assets"
+caption="Most Viewed Data Assets"
+/%}
+
+### Page Views by Data Assets
+
+It helps to understand the total number of page views by asset type. This allows you to understand which asset familly drives the most interest in your organization
+
+{% image
+src="/images/v1.1/how-to-guides/insights/pvda.png"
+alt="Page Views by Data Assets"
+caption="Page Views by Data Assets"
+/%}
+
+### Daily Active Users on the Platform
+
+Active users are users with at least one session. This report allows to understand the platform usage and see how your organization leverages OpenMetadata.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/daup.png"
+alt="Daily Active Users on the Platform"
+caption="Daily Active Users on the Platform"
+/%}
+
+### Most Active Users
+
+This report displays the most active users on the platform based on Page Views. They are the power users in your data team.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/mau.png"
+alt="Most Active Users"
+caption="Most Active Users"
+/%}
+
+## Key Performance Indicators (KPI)
+
+While data insights reports gives an analytical view of the OpenMetadata platform, KPIs are here to drive platform adoption. The below report displays the percentage coverage of description and ownership of the data assets.
+
+{% image
+src="/images/v1.1/how-to-guides/insights/kpi.png"
+alt="Key Performance Indicators (KPI)"
+caption="Key Performance Indicators (KPI)"
+/%}
+
+{%inlineCallout
+  color="violet-70"
+  bold="How to Transform the Data Culture of Your Company"
+  icon="MdArrowForward"
+  href="/how-to-guides/openmetadata/data-insights/data-culture"%}
+  Improve your data culture for data-driven decision making.
+{%/inlineCallout%}

openmetadata-docs/content/v1.1.x/how-to-guides/openmetadata/data-insights/tiering.md

@@ -0,0 +1,44 @@
+---
+title: What is Tiering
+slug: /how-to-guides/openmetadata/data-insights/tiering
+---
+
+# What is Tiering
+
+Tiering is an important concept of data classification in OpenMetadata. Data Producers and Consumers can set business importance of data by setting Tiers. `Tier 1` is the most important data of an organization.
+
+In OpenMetadata, Tiers are System Classification tags and can be accessed from **Govern > Classification > Tier**.
+
+{% image
+src="/images/v1.1/how-to-guides/governance/tier1.png"
+alt="Classification Tags: Tiers"
+caption="Classification Tags: Tiers"
+/%}
+
+In case of tiering, it is easiest to start with the most important (Tier 1) and the least important (Tier 5) data. Once the **Tier 1** or most important data is identified, organizations can focus on improving the descriptions and data quality. The Data Insights in OpenMetadata helps identify the unused datasets as **Tier 5**. The Tier 5 datasets can be deleted periodically to declutter. Other tiers can be added as per your organizational needs. **Tags** can be added to further mark the data assets.
+
+| **Tier** | **Impact** | **Used for** | **Type of Impact** | **Usage** |
+|--- | --- | --- | --- | --- |
+| **Tier 1** | High | External & Internal Decisions | Revenue, Regulatory, & Reputational | Highly used |
+| **Tier 2** | Moderate | Some External & Mostly Internal Decisions | Some Regulatory | Highly used |
+| **Tier 3** | Low | Internal Decisions | - | Highly used (Top N percentile) |
+| **Tier 4** | Low | Internal Team Decisions | - | - |
+| **Tier 5** | Individual owned | Unused Datasets | - | - |
+
+## How to Add Tiers
+
+From the **Explore** page, select a data asset and click on the edit icon for **Tier**. Select the appropriate tier. Clicking on the arrow next to the tier will provide a description of the tier.
+
+{% image
+src="/images/v1.1/how-to-guides/governance/tier2.png"
+alt="Add a Tier to Data Asset"
+caption="Add a Tier to Data Asset"
+/%}
+
+{%inlineCallout
+  color="violet-70"
+  bold="Set Up Data Insights Ingestion"
+  icon="MdArrowForward"
+  href="/how-to-guides/openmetadata/data-insights/ingestion"%}
+  Set up the ingestion pipeline right from the UI.
+{%/inlineCallout%}

openmetadata-docs/content/v1.1.x/menu.md

@@ -667,6 +667,16 @@ site_menu:
     url: /how-to-guides/openmetadata/data-lineage/manual
   - category: How to Guides / The Six Pillars of OpenMetadata / Data Insights
     url: /how-to-guides/openmetadata/data-insights
+  - category: How to Guides / The Six Pillars of OpenMetadata / Data Insights / What is Tiering
+    url: /how-to-guides/openmetadata/data-insights/tiering
+  - category: How to Guides / The Six Pillars of OpenMetadata / Data Insights / Set Up Data Insights Ingestion
+    url: /how-to-guides/openmetadata/data-insights/ingestion
+  - category: How to Guides / The Six Pillars of OpenMetadata / Data Insights / Key Performance Indicators (KPI)
+    url: /how-to-guides/openmetadata/data-insights/kpi
+  - category: How to Guides / The Six Pillars of OpenMetadata / Data Insights / Data Insights Report
+    url: /how-to-guides/openmetadata/data-insights/report
+  - category: How to Guides / The Six Pillars of OpenMetadata / Data Insights / How to Transform the Data Culture of Your Company
+    url: /how-to-guides/openmetadata/data-insights/data-culture
   - category: How to Guides / The Six Pillars of OpenMetadata / Data Governance
     url: /how-to-guides/openmetadata/data-governance
   - category: How to Guides / The Six Pillars of OpenMetadata / Data Governance / Glossary and Classification