1.8 KiB

Ingesting metadata from dbt requires either using the dbt module or the dbt-cloud module.

Concept Mapping

Source Concept DataHub Concept Notes
Source Dataset Subtype Source
Seed Dataset Subtype Seed
Model Dataset Subtype Model
Snapshot Dataset Subtype Snapshot
Test Assertion
Test Result Assertion Run Result
Model Runs DataProcessInstance

Note:

  1. You must run ingestion for both dbt and your data warehouse (target platform). They can be run in any order.
  2. It generates column lineage between the dbt nodes (e.g. when a model/snapshot depends on a dbt source or ephemeral model) as well as lineage between the dbt nodes and the underlying target platform nodes (e.g. BigQuery Table -> dbt source, dbt model -> BigQuery table/view).
  3. It automatically generates "sibling" relationships between the dbt nodes and the target / data warehouse nodes. These nodes will show up in the UI with both platform logos.
  4. We also support automated actions (like add a tag, term or owner) based on properties defined in dbt meta.