15 Commits

Author SHA1 Message Date
Hyejin Yoon
f986315582
doc: Acryl to DataHub, datahubproject.io to datahub.com (#13252)
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-04-28 10:34:33 -04:00
Andrew Sikowitz
d138a64a6a
ci(graphql,workflows): Format .md, .graphql, and workflow .yml files via prettier (#13220) 2025-04-16 16:55:51 -07:00
Tamas Nemeth
f4fb89e799
feat(ingest/spark): Promote beta plugin (#10881)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-07-25 14:46:32 +02:00
Tamas Nemeth
d204d5654a
fix(ingestion/spark): Platform instance and column level lineage fix (#10843)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-07-09 13:54:06 +02:00
Mayuri Nehate
529710ab9d
fix(ingest/databricks): handle and report config parse failure, updat… (#10261) 2024-04-17 12:14:16 -07:00
Mayuri Nehate
9928d70c1d
fix(ingest/databricks): support hive metastore schemas with special char (#10049) 2024-03-27 09:41:46 -07:00
dushayntAW
3a4bdef44a
fix(ingest/unity): generate sibling and lineage (#9894) 2024-03-18 20:22:17 +05:30
Mayuri Nehate
f2e78db92e
feat(ingest/databricks): ingest hive metastore by default, more docs (#9601)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-18 14:26:33 -08:00
Andrew Sikowitz
27f23ecdd5
feat(ingest/unity): GE Profiling (#8951) 2023-12-06 13:59:23 -05:00
Andrew Sikowitz
d3346a04e4
feat(ingest/unity): Ingest notebooks and their lineage (#8940) 2023-10-04 10:22:45 -04:00
Hyejin Yoon
04ecf4f75a
docs(docs): add native versioning (#8714) 2023-08-25 14:10:13 -07:00
Andrew Sikowitz
8a198cd615
fix(ingest/unity): Pin databricks-sdk and update docs (#8293) 2023-06-27 13:38:55 -04:00
Andrew Sikowitz
d3cd4dbb0c
feat(ingest/unity): Allow ingestion without metastore admin role (#8091)
- Adds more detailed docs and connection test
- Fixes empty username queries
2023-05-24 15:36:22 -07:00
Andrew Sikowitz
5b290c9bc5
feat(ingest/unity): Add usage extraction; add TableReference (#7910)
- Adds usage extraction to the unity catalog source and a TableReference object to handle references to tables
Also makes the following refactors:
- Creates UsageAggregator class to usage_common, as I've seen this same logic multiple times.
- Allows customizable user_urn_builder in usage_common as not all unity users are emails. We create emails with a default email_domain config in other connectors like redshift and snowflake, which seems unnecessary now?
- Creates TableReference for unity catalog and adds it to the Table dataclass, for managing string references to tables. Replaces logic, especially in lineage extraction, with these references
- Creates gen_dataset_urn and gen_user_urn on unity source to reduce duplicate code
Breaks up proxy.py into implementation and types
2023-05-01 11:30:09 -07:00
Tamas Nemeth
0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00