mirror of
https://github.com/datahub-project/datahub.git
synced 2025-07-03 15:17:58 +00:00
1.3 KiB
1.3 KiB
Concept Mapping
| Source Concept | DataHub Concept | Notes |
|---|---|---|
"glue" |
Data Platform | |
| Glue Database | Container | Subtype Database |
| Glue Table | Dataset | Subtype Table |
| Glue Job | Data Flow | |
| Glue Job Transform | Data Job | |
| Glue Job Data source | Dataset | |
| Glue Job Data sink | Dataset |
Compatibility
To capture lineage across Glue jobs and databases, a requirements must be met – otherwise the AWS API is unable to report any lineage. The job must be created in Glue Studio with the "Generate classic script" option turned on (this option can be accessed in the "Script" tab). Any custom scripts that do not have the proper annotations will not have reported lineage.