14 Commits

Author SHA1 Message Date
Anush Kumar
40b51ac2da
feat(ingestion): Added Databricks support to Fivetran source (#14897) 2025-10-06 10:06:37 -07:00
Harshal Sheth
a17fc4e0a8
chore(python): drop pydantic v1 support (#14014)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
Co-authored-by: Piotr Skrydalewicz <piotr.skrydalewicz@acryl.io>
2025-09-23 07:40:29 +00:00
Tamas Nemeth
9ec58e3876
fix(ingest/unity): Backport proxy fix for unity catalog sql library (#14571) 2025-09-03 17:37:45 +02:00
Hyejin Yoon
584f6ce3d0
feat(ingest/unity) : add mlmodel / mlmodel version support (#14594) 2025-09-02 15:22:17 +09:00
Tamas Nemeth
6fa4f3e590
fix(ingest/unity): Add option to set databricks api page size (#14430) 2025-08-12 13:38:32 +02:00
Mayuri Nehate
529710ab9d
fix(ingest/databricks): handle and report config parse failure, updat… (#10261) 2024-04-17 12:14:16 -07:00
Mayuri Nehate
f2e78db92e
feat(ingest/databricks): ingest hive metastore by default, more docs (#9601)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-18 14:26:33 -08:00
Mayuri Nehate
aac1c55a14
feat(ingest/unity): enable hive metastore ingestion (#9416) 2023-12-14 10:35:06 -05:00
Andrew Sikowitz
27f23ecdd5
feat(ingest/unity): GE Profiling (#8951) 2023-12-06 13:59:23 -05:00
Andrew Sikowitz
66806a805e
feat(ingest/unity): Set external url for containers and datasets (#8238) 2023-06-15 09:05:49 +02:00
Andrew Sikowitz
d3cd4dbb0c
feat(ingest/unity): Allow ingestion without metastore admin role (#8091)
- Adds more detailed docs and connection test
- Fixes empty username queries
2023-05-24 15:36:22 -07:00
Andrew Sikowitz
afcf462cb1
feat(ingest/unity): Add profiling support (#7976)
- Also adds a new databricks sdk
2023-05-11 10:00:50 -07:00
Mayuri Nehate
0131aeefb1
fix(ingest/unity): improve error message if no scheme in workspace_url (#7951)
Co-authored-by: John Joyce <john@acryl.io>
2023-05-08 10:13:53 -07:00
Andrew Sikowitz
5b290c9bc5
feat(ingest/unity): Add usage extraction; add TableReference (#7910)
- Adds usage extraction to the unity catalog source and a TableReference object to handle references to tables
Also makes the following refactors:
- Creates UsageAggregator class to usage_common, as I've seen this same logic multiple times.
- Allows customizable user_urn_builder in usage_common as not all unity users are emails. We create emails with a default email_domain config in other connectors like redshift and snowflake, which seems unnecessary now?
- Creates TableReference for unity catalog and adds it to the Table dataclass, for managing string references to tables. Replaces logic, especially in lineage extraction, with these references
- Creates gen_dataset_urn and gen_user_urn on unity source to reduce duplicate code
Breaks up proxy.py into implementation and types
2023-05-01 11:30:09 -07:00