yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-23 16:38:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

MINOR: [DOCS] dbt cloud docs update (#17321)

Browse Source 
* dbt cloud docs update

* fix token part
This commit is contained in:
Onkar Ravgan 2024-08-07 20:50:31 +05:30 committed by GitHub
parent 5ea7f8cc55
commit af9f45b00a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 62 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
openmetadata-docs/content/v1.4.x/connectors/ingestion/workflows/dbt/ingest-dbt-ui.md
View File

@ -127,11 +127,20 @@ File server path of the `manifest.json`, `catalog.json` and `run_results.json` f
#### 6. dbt Cloud
Click on the the link [here](https://docs.getdbt.com/guides/getting-started) for getting started with dbt cloud account setup if not done already.
OpenMetadata uses dbt cloud APIs to fetch the `run artifacts` (manifest.json, catalog.json and run_results.json) from the most recent dbt run.
The APIs need to be authenticated using an Authentication Token. Follow the link [here](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens) to generate an authentication token for your dbt cloud account.
The `Account Viewer` permission is the minimum requirement for the dbt cloud token.
{% note %}
The dbt Cloud workflow leverages the [dbt Cloud v2](https://docs.getdbt.com/dbt-cloud/api-v2#/) APIs to retrieve dbt run artifacts (manifest.json, catalog.json, and run_results.json) and ingest the dbt metadata.
It uses the [/runs](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) API to obtain the most recent successful dbt run, filtering by account_id, project_id and job_id if specified. The artifacts from this run are then collected using the [/artifacts](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Run%20Artifacts) API.
Refer to the code [here](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/dbt/dbt_config.py#L142)
{% /note %}
{% image
src="/images/v1.4/features/ingestion/workflows/dbt/dbt-cloud.webp"
alt="dbt-cloud"
@ -142,6 +151,8 @@ The `Account Viewer` permission is the minimum requirement for the dbt cloud tok
The fields for `Dbt Cloud Account Id`, `Dbt Cloud Project Id` and `Dbt Cloud Job Id` should be numeric values.
To know how to get the values for `Dbt Cloud Account Id`, `Dbt Cloud Project Id` and `Dbt Cloud Job Id` fields check [here](/connectors/ingestion/workflows/dbt/ingest-dbt-yaml).
{% /note %}

17
openmetadata-docs/content/v1.4.x/connectors/ingestion/workflows/dbt/ingest-dbt-yaml.md
View File

@ -495,6 +495,16 @@ In this configuration we will be fetching the dbt `manifest.json`, `catalog.json
The `Account Viewer` permission is the minimum requirement for the dbt cloud token.
{% note %}
The dbt Cloud workflow leverages the [dbt Cloud v2](https://docs.getdbt.com/dbt-cloud/api-v2#/) APIs to retrieve dbt run artifacts (manifest.json, catalog.json, and run_results.json) and ingest the dbt metadata.
It uses the [/runs](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) API to obtain the most recent successful dbt run, filtering by account_id, project_id and job_id if specified. The artifacts from this run are then collected using the [/artifacts](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Run%20Artifacts) API.
Refer to the code [here](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/dbt/dbt_config.py#L142)
{% /note %}
{% codePreview %}
{% codeInfoContainer %}
@ -507,6 +517,7 @@ The `Account Viewer` permission is the minimum requirement for the dbt cloud tok
{% codeInfo srNumber=61 %}
- **dbtCloudAuthToken**: Please follow the instructions in [dbt Cloud's API](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens) documentation to create a dbt Cloud authentication token.
The `Account Viewer` permission is the minimum requirement for the dbt cloud token.
{% /codeInfo %}
{% codeInfo srNumber=62 %}
@ -518,12 +529,18 @@ For example, if the URL is `https://cloud.getdbt.com/#/accounts/1234/projects/67
{% codeInfo srNumber=63 %}
- **dbtCloudJobId**: In case of multiple jobs in a dbt cloud account, specify the job's ID from which you want to extract the dbt run artifacts.
If left empty, the dbt artifacts will be fetched from the most recent run on dbt cloud.
After creating a dbt job, take note of the url which will be similar to `https://cloud.getdbt.com/#/accounts/1234/projects/6789/jobs/553344/`. The job ID is `553344`.
The value entered should be a `numeric` value.
{% /codeInfo %}
{% codeInfo srNumber=64 %}
- **dbtCloudProjectId**: In case of multiple projects in a dbt cloud account, specify the project's ID from which you want to extract the dbt run artifacts.
If left empty, the dbt artifacts will be fetched from the most recent run on dbt cloud.
To find your project ID, sign in to your dbt cloud account and choose a specific project. Take note of the url which will be similar to `https://cloud.getdbt.com/#/accounts/1234/settings/projects/6789/`, the project ID is `6789`.
The value entered should be a `numeric` value.
{% /codeInfo %}

13
openmetadata-docs/content/v1.5.x-SNAPSHOT/connectors/ingestion/workflows/dbt/ingest-dbt-ui.md
View File

@ -127,11 +127,20 @@ File server path of the `manifest.json`, `catalog.json` and `run_results.json` f
#### 6. dbt Cloud
Click on the the link [here](https://docs.getdbt.com/guides/getting-started) for getting started with dbt cloud account setup if not done already.
OpenMetadata uses dbt cloud APIs to fetch the `run artifacts` (manifest.json, catalog.json and run_results.json) from the most recent dbt run.
The APIs need to be authenticated using an Authentication Token. Follow the link [here](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens) to generate an authentication token for your dbt cloud account.
The `Account Viewer` permission is the minimum requirement for the dbt cloud token.
{% note %}
The dbt Cloud workflow leverages the [dbt Cloud v2](https://docs.getdbt.com/dbt-cloud/api-v2#/) APIs to retrieve dbt run artifacts (manifest.json, catalog.json, and run_results.json) and ingest the dbt metadata.
It uses the [/runs](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) API to obtain the most recent successful dbt run, filtering by account_id, project_id and job_id if specified. The artifacts from this run are then collected using the [/artifacts](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Run%20Artifacts) API.
Refer to the code [here](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/dbt/dbt_config.py#L142)
{% /note %}
{% image
src="/images/v1.5/features/ingestion/workflows/dbt/dbt-cloud.webp"
alt="dbt-cloud"
@ -142,6 +151,8 @@ The `Account Viewer` permission is the minimum requirement for the dbt cloud tok
The fields for `Dbt Cloud Account Id`, `Dbt Cloud Project Id` and `Dbt Cloud Job Id` should be numeric values.
To know how to get the values for `Dbt Cloud Account Id`, `Dbt Cloud Project Id` and `Dbt Cloud Job Id` fields check [here](/connectors/ingestion/workflows/dbt/ingest-dbt-yaml).
{% /note %}

17
openmetadata-docs/content/v1.5.x-SNAPSHOT/connectors/ingestion/workflows/dbt/ingest-dbt-yaml.md
View File

@ -495,6 +495,16 @@ In this configuration we will be fetching the dbt `manifest.json`, `catalog.json
The `Account Viewer` permission is the minimum requirement for the dbt cloud token.
{% note %}
The dbt Cloud workflow leverages the [dbt Cloud v2](https://docs.getdbt.com/dbt-cloud/api-v2#/) APIs to retrieve dbt run artifacts (manifest.json, catalog.json, and run_results.json) and ingest the dbt metadata.
It uses the [/runs](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) API to obtain the most recent successful dbt run, filtering by account_id, project_id and job_id if specified. The artifacts from this run are then collected using the [/artifacts](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Run%20Artifacts) API.
Refer to the code [here](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/dbt/dbt_config.py#L142)
{% /note %}
{% codePreview %}
{% codeInfoContainer %}
@ -507,6 +517,7 @@ The `Account Viewer` permission is the minimum requirement for the dbt cloud tok
{% codeInfo srNumber=61 %}
- **dbtCloudAuthToken**: Please follow the instructions in [dbt Cloud's API](https://docs.getdbt.com/docs/dbt-cloud-apis/service-tokens) documentation to create a dbt Cloud authentication token.
The `Account Viewer` permission is the minimum requirement for the dbt cloud token.
{% /codeInfo %}
{% codeInfo srNumber=62 %}
@ -518,12 +529,18 @@ For example, if the URL is `https://cloud.getdbt.com/#/accounts/1234/projects/67
{% codeInfo srNumber=63 %}
- **dbtCloudJobId**: In case of multiple jobs in a dbt cloud account, specify the job's ID from which you want to extract the dbt run artifacts.
If left empty, the dbt artifacts will be fetched from the most recent run on dbt cloud.
After creating a dbt job, take note of the url which will be similar to `https://cloud.getdbt.com/#/accounts/1234/projects/6789/jobs/553344/`. The job ID is `553344`.
The value entered should be a `numeric` value.
{% /codeInfo %}
{% codeInfo srNumber=64 %}
- **dbtCloudProjectId**: In case of multiple projects in a dbt cloud account, specify the project's ID from which you want to extract the dbt run artifacts.
If left empty, the dbt artifacts will be fetched from the most recent run on dbt cloud.
To find your project ID, sign in to your dbt cloud account and choose a specific project. Take note of the url which will be similar to `https://cloud.getdbt.com/#/accounts/1234/settings/projects/6789/`, the project ID is `6789`.
The value entered should be a `numeric` value.
{% /codeInfo %}

4
openmetadata-ui/src/main/resources/ui/public/locales/en-US/Database/workflows/dbt.md
View File

@ -104,6 +104,8 @@ In case of multiple projects in a dbt cloud account, specify the project's ID fr
If left empty, the dbt artifacts will be fetched from the most recent run on dbt cloud.
To find your project ID, sign in to your dbt cloud account and choose a specific project. Take note of the url which will be similar to `https://cloud.getdbt.com/#/accounts/1234/settings/projects/6789/`, the project ID is `6789`.
The value entered should be a `numeric` value.
$$
@ -114,6 +116,8 @@ In case of multiple jobs in a dbt cloud account, specify the job's ID from which
If left empty, the dbt artifacts will be fetched from the most recent run on dbt cloud.
After creating a dbt job, take note of the url which will be similar to `https://cloud.getdbt.com/#/accounts/1234/projects/6789/jobs/553344/`. The job ID is `553344`.
The value entered should be a `numeric` value.
$$