yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-19 07:30:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-ingestion/docs/sources/databricks/unity-catalog_post.md
Tamas Nemeth 0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292) 
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00

828 B
Raw Blame History

Troubleshooting

No data lineage captured or missing lineage

Check that you meet the Unity Catalog lineage requirements.

Also check the Unity Catalog limitations to make sure that lineage would be expected to exist in this case.

Lineage extraction is too slow

Currently, there is no way to get table or column lineage in bulk from the Databricks Unity Catalog REST api. Table lineage calls require one API call per table, and column lineage calls require one API call per column. If you find metadata extraction taking too long, you can turn off column level lineage extraction via the include_column_lineage config flag.