13 Commits

Author SHA1 Message Date
Andrew Sikowitz
16fe22aafa
feat(ui/last-updated): Calculate last updated time as max(properties time, operation time) (#9242) 2023-12-06 15:21:56 -05:00
Andrew Sikowitz
5cb04981b2
test(ingest/unity): Update goldens (#9254) 2023-11-16 17:18:17 -05:00
Andrew Sikowitz
3a21c27f06
feat(ingest): Turn on browse path v2 creation (#8342)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 16:43:42 -04:00
Andrew Sikowitz
584366771d
refactor(unity): Remove databricks_cli and cleanup (#8249)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-23 18:01:05 +05:30
Andrew Sikowitz
66806a805e
feat(ingest/unity): Set external url for containers and datasets (#8238) 2023-06-15 09:05:49 +02:00
Andrew Sikowitz
3022c2d12e
feat(ingest/unity): Add qualified name to dataset properties (#8164) 2023-06-05 11:20:13 -07:00
Andrew Sikowitz
d3cd4dbb0c
feat(ingest/unity): Allow ingestion without metastore admin role (#8091)
- Adds more detailed docs and connection test
- Fixes empty username queries
2023-05-24 15:36:22 -07:00
Andrew Sikowitz
5b290c9bc5
feat(ingest/unity): Add usage extraction; add TableReference (#7910)
- Adds usage extraction to the unity catalog source and a TableReference object to handle references to tables
Also makes the following refactors:
- Creates UsageAggregator class to usage_common, as I've seen this same logic multiple times.
- Allows customizable user_urn_builder in usage_common as not all unity users are emails. We create emails with a default email_domain config in other connectors like redshift and snowflake, which seems unnecessary now?
- Creates TableReference for unity catalog and adds it to the Table dataclass, for managing string references to tables. Replaces logic, especially in lineage extraction, with these references
- Creates gen_dataset_urn and gen_user_urn on unity source to reduce duplicate code
Breaks up proxy.py into implementation and types
2023-05-01 11:30:09 -07:00
Andrew Sikowitz
e9c2f9afcc
feat(ingest/unity): Ingest ownership for containers; lookup service principal display names (#7869) 2023-04-21 11:02:39 -07:00
Harshal Sheth
e461d03d94
feat(ingest/unity): capture create/lastModified timestamps (#7819)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-04-17 12:18:21 -07:00
Harshal Sheth
204727a6ee
feat(ingest/unity): support extracting ownership (#7801) 2023-04-12 19:45:41 -07:00
Tamas Nemeth
62e33e03a3
fix(ingest/unity): Use assigned metastore if not metastore listed in unity catalog (#7446) 2023-02-28 08:06:28 +01:00
Tamas Nemeth
0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00