Added dbt description toggle docs (#11870)

* Added dbt desc toggle docs

* Added source of truth

* added desc for catalog
This commit is contained in:
Onkar Ravgan 2023-06-06 14:46:13 +05:30 committed by GitHub
parent 721869428e
commit 89995665b0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 111 additions and 6 deletions

View File

@ -79,9 +79,9 @@ Please refer [here](/connectors/ingestion/workflows/dbt/ingest-dbt-owner) for ad
### 5. dbt Descriptions
Descriptions from dbt models can be imported and assigned to respective tables and columns.
Descriptions from dbt `manifest.json` and `catalog.json` can be imported and assigned to respective tables and columns.
By default descriptions from `manifest.json` will be imported. Descriptions from `catalog.json` will only be updated if catalog file is passed.
For more information and to control how the table and column descriptions are updated from dbt please take a look [here](/connectors/ingestion/workflows/dbt/ingest-dbt-descriptions)
{% image
src="/images/v0.13.3/openmetadata/ingestion/workflows/dbt/dbt-features/dbt-descriptions.png"

View File

@ -0,0 +1,33 @@
---
title: Ingest Descriptions from dbt
slug: /connectors/ingestion/workflows/dbt/ingest-dbt-descriptions
---
# Ingest Descriptions from dbt
Ingest the descriptions from dbt `manifest.json` or `catalog.json` file into OpenMetadata tables and columns.
{% note %}
By default descriptions from `manifest.json` will be imported. Descriptions from `catalog.json` will only be updated if `catalog.json` file is passed.
If only the `manifest.json` file is passed, descriptions from the `manifest.json` will be updated.
If the `manifest.json` and `catalog.json` both are passed, descriptions from the `catalog.json` will be updated.
{% /note %}
## Overriding the existing table and column descriptions
To establish a unified and reliable system for descriptions, a single source of truth is necessary. It either is directly OpenMetadata, if individuals want to go there and keep updating, or if they prefer to keep it centralized in dbt, then we can always rely on that directly.
When the `Update Descriptions` toggle is enabled during the configuration of dbt ingestion, existing descriptions of tables and columns will be overwritten with the dbt descriptions.
If toggle is disabled during the configuration of dbt ingestion, dbt descriptions will only be updated for tables and columns in OpenMetadata that currently have no descriptions. Existing descriptions will remain unchanged and will not be overwritten with dbt descriptions.
{% image
src="/images/v0.13.3/openmetadata/ingestion/workflows/dbt/dbt-features/dbt-update-descriptions.png"
alt="update-dbt-descriptions"
caption="Update dbt Descriptions"
/%}

View File

@ -542,6 +542,8 @@ site_menu:
url: /connectors/ingestion/workflows/dbt/ingest-dbt-cli
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Owner
url: /connectors/ingestion/workflows/dbt/ingest-dbt-owner
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Descriptions
url: /connectors/ingestion/workflows/dbt/ingest-dbt-descriptions
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Tags
url: /connectors/ingestion/workflows/dbt/ingest-dbt-tags
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Lineage

View File

@ -96,9 +96,9 @@ Please refer [here](/connectors/ingestion/workflows/dbt/ingest-dbt-owner) for ad
### 5. dbt Descriptions
Descriptions from dbt models can be imported and assigned to respective tables and columns.
Descriptions from dbt `manifest.json` and `catalog.json` can be imported and assigned to respective tables and columns.
By default descriptions from `manifest.json` will be imported. Descriptions from `catalog.json` will only be updated if catalog file is passed.
For more information and to control how the table and column descriptions are updated from dbt please take a look [here](/connectors/ingestion/workflows/dbt/ingest-dbt-descriptions)
{% image
src="/images/v1.0.0/features/ingestion/workflows/dbt/dbt-features/dbt-descriptions.png"

View File

@ -0,0 +1,33 @@
---
title: Ingest Descriptions from dbt
slug: /connectors/ingestion/workflows/dbt/ingest-dbt-descriptions
---
# Ingest Descriptions from dbt
Ingest the descriptions from dbt `manifest.json` or `catalog.json` file into OpenMetadata tables and columns.
{% note %}
By default descriptions from `manifest.json` will be imported. Descriptions from `catalog.json` will only be updated if `catalog.json` file is passed.
If only the `manifest.json` file is passed, descriptions from the `manifest.json` will be updated.
If the `manifest.json` and `catalog.json` both are passed, descriptions from the `catalog.json` will be updated.
{% /note %}
## Overriding the existing table and column descriptions
To establish a unified and reliable system for descriptions, a single source of truth is necessary. It either is directly OpenMetadata, if individuals want to go there and keep updating, or if they prefer to keep it centralized in dbt, then we can always rely on that directly.
When the `Update Descriptions` toggle is enabled during the configuration of dbt ingestion, existing descriptions of tables and columns will be overwritten with the dbt descriptions.
If toggle is disabled during the configuration of dbt ingestion, dbt descriptions will only be updated for tables and columns in OpenMetadata that currently have no descriptions. Existing descriptions will remain unchanged and will not be overwritten with dbt descriptions.
{% image
src="/images/v1.0.0/features/ingestion/workflows/dbt/dbt-features/dbt-update-descriptions.png"
alt="update-dbt-descriptions"
caption="Update dbt Descriptions"
/%}

View File

@ -601,6 +601,8 @@ site_menu:
url: /connectors/ingestion/workflows/dbt/ingest-dbt-cli
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Owner
url: /connectors/ingestion/workflows/dbt/ingest-dbt-owner
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Descriptions
url: /connectors/ingestion/workflows/dbt/ingest-dbt-descriptions
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Tags
url: /connectors/ingestion/workflows/dbt/ingest-dbt-tags
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Lineage

View File

@ -96,9 +96,9 @@ Please refer [here](/connectors/ingestion/workflows/dbt/ingest-dbt-owner) for ad
### 5. dbt Descriptions
Descriptions from dbt models can be imported and assigned to respective tables and columns.
Descriptions from dbt `manifest.json` and `catalog.json` can be imported and assigned to respective tables and columns.
By default descriptions from `manifest.json` will be imported. Descriptions from `catalog.json` will only be updated if catalog file is passed.
For more information and to control how the table and column descriptions are updated from dbt please take a look [here](/connectors/ingestion/workflows/dbt/ingest-dbt-descriptions)
{% image
src="/images/v1.0.0/features/ingestion/workflows/dbt/dbt-features/dbt-descriptions.png"

View File

@ -0,0 +1,33 @@
---
title: Ingest Descriptions from dbt
slug: /connectors/ingestion/workflows/dbt/ingest-dbt-descriptions
---
# Ingest Descriptions from dbt
Ingest the descriptions from dbt `manifest.json` or `catalog.json` file into OpenMetadata tables and columns.
{% note %}
By default descriptions from `manifest.json` will be imported. Descriptions from `catalog.json` will only be updated if `catalog.json` file is passed.
If only the `manifest.json` file is passed, descriptions from the `manifest.json` will be updated.
If the `manifest.json` and `catalog.json` both are passed, descriptions from the `catalog.json` will be updated.
{% /note %}
## Overriding the existing table and column descriptions
To establish a unified and reliable system for descriptions, a single source of truth is necessary. It either is directly OpenMetadata, if individuals want to go there and keep updating, or if they prefer to keep it centralized in dbt, then we can always rely on that directly.
When the `Update Descriptions` toggle is enabled during the configuration of dbt ingestion, existing descriptions of tables and columns will be overwritten with the dbt descriptions.
If toggle is disabled during the configuration of dbt ingestion, dbt descriptions will only be updated for tables and columns in OpenMetadata that currently have no descriptions. Existing descriptions will remain unchanged and will not be overwritten with dbt descriptions.
{% image
src="/images/v1.1.0-snapshot/features/ingestion/workflows/dbt/dbt-features/dbt-update-descriptions.png"
alt="update-dbt-descriptions"
caption="Update dbt Descriptions"
/%}

View File

@ -613,6 +613,8 @@ site_menu:
url: /connectors/ingestion/workflows/dbt/ingest-dbt-cli
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Owner
url: /connectors/ingestion/workflows/dbt/ingest-dbt-owner
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Descriptions
url: /connectors/ingestion/workflows/dbt/ingest-dbt-descriptions
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Tags
url: /connectors/ingestion/workflows/dbt/ingest-dbt-tags
- category: Connectors / Ingestion / Workflows / dbt / Ingest dbt Lineage

Binary file not shown.

After

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 103 KiB