1604 Commits

Author SHA1 Message Date
Tamas Nemeth
0ea6145a9d
fix(ingest/profiling): Add option to enable external table profiling (#9463) 2023-12-15 00:12:45 +01:00
Shubham Jagtap
1741c07d76
feat(ingestion): Add test_connection methods for important sources (#9334) 2023-12-14 12:31:51 -05:00
Shirshanka Das
ecef50f8fc
fix(ingest): make user_urn and group_urn generation consider user and… (#9026)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-12-14 21:33:36 +05:30
siddiquebagwan-gslab
0d6a5e5df2
feat(ingestion/transformer): create tag if not exist (#9076) 2023-12-14 10:36:28 -05:00
Mayuri Nehate
aac1c55a14
feat(ingest/unity): enable hive metastore ingestion (#9416) 2023-12-14 10:35:06 -05:00
Aseem Bansal
b0de1dc0ce
fix(powerbi): add access token refresh (#9405)
Co-authored-by: elish7lapid <elisheva@foundational.io>
Co-authored-by: treff7es <treff7es@gmail.com>
2023-12-14 18:41:50 +05:30
skrydal
a495d652e0
feat(ingestion): Add typeUrn handling to ownership transformers (#9370) 2023-12-13 14:34:20 -05:00
RyanHolstien
8a1122049c
feat(patch): support fine grained lineage patches (#9408)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-12-11 16:25:43 -06:00
Harshal Sheth
08a9b9b6de
fix(ingest): bug fixes and docs updates (#9422) 2023-12-08 16:18:34 -05:00
Teppo Naakka
0e40d38f4c
fix(ingest/powerbi): fix powerbi chart input handling (#9415) 2023-12-08 13:02:03 +01:00
Harshal Sheth
f03c66ca1f
fix(ingest/lookml): make deploy key optional (#9378) 2023-12-07 17:18:16 -05:00
Andrew Sikowitz
16fe22aafa
feat(ui/last-updated): Calculate last updated time as max(properties time, operation time) (#9242) 2023-12-06 15:21:56 -05:00
Andrew Sikowitz
27f23ecdd5
feat(ingest/unity): GE Profiling (#8951) 2023-12-06 13:59:23 -05:00
Andrew Sikowitz
806f09ae23
test(ingest/unity): Unity catalog data generation (#8949) 2023-12-05 12:33:00 -05:00
Harshal Sheth
7517c77ffd
fix(ingest): resolve issue with caplog and asyncio (#9377) 2023-12-04 20:00:11 -05:00
Harshal Sheth
4ec3208918
feat(ingest): support stdin in datahub put (#9359) 2023-12-04 14:31:58 -05:00
Andrew Sikowitz
c0ef728868
fix(ingest/powerbi): Allow old parser to parse [db].[schema].[table] table references (#9360) 2023-12-04 13:21:42 -05:00
Harshal Sheth
4d9eb12cba
feat(ingest/dbt): support custom ownership types in dbt meta (#9332) 2023-12-01 14:33:10 +05:30
Harshal Sheth
f9fd9467ef
feat(ingest): clean up DataHubRestEmitter return type (#9286)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-11-30 21:00:43 -05:00
Harshal Sheth
a7dc9c9d22
feat(sdk): autogenerate urn types (#9257) 2023-11-30 18:11:36 -05:00
Adriano Vega Llobell
3d7962cf17
fix(ingest/sql): improve handling of views with dots in their names (#9183) 2023-11-28 17:58:42 -05:00
Mayuri Nehate
08fb730676
fix(ingest): drop deprecated database_alias from sql sources (#9299)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-11-28 15:49:49 -05:00
terratrue-daniel
ff9876f5a2
fix(ingest/mssql): Add MONEY and SMALLMONEY data types as Number (#9313) 2023-11-28 14:43:52 -05:00
Mayuri Nehate
a34fdfd8b7
fix(ingest/lookml): emit all views with same name and different file path (#9279) 2023-11-24 17:59:24 -05:00
Tamas Nemeth
f794a90515
fix(ingest/snowflake): Apply email filter on all usage metrics (#9269) 2023-11-23 09:45:25 +01:00
RyanHolstien
a62d52e6b1
feat(mysql): upgrade to version 8.2 for quickstart (#9241)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-11-22 13:54:12 -06:00
Tim
01874808af
fix(ingest/athena): detect decimal type correctly (#9270) 2023-11-20 16:57:48 +01:00
Andrew Sikowitz
5cb04981b2
test(ingest/unity): Update goldens (#9254) 2023-11-16 17:18:17 -05:00
Adriano Vega Llobell
d0fa5de9f0
fix(sql-parser): convert platform instance to lowercase when building table urns (#9181) 2023-11-16 13:58:49 -05:00
Harshal Sheth
19aa215068
feat(ingest/dbt): dbt column-level lineage (#8991) 2023-11-13 19:00:30 -05:00
Shubham Jagtap
a187127ac5
feat(ingestion): file-based state checkpoint provider (#9029) 2023-11-10 14:36:00 -08:00
Teppo Naakka
107713846f
fix(ingest/powerbi): use dataset workspace id as key for parent container (#8994) 2023-11-09 16:48:06 -08:00
Shubham Jagtap
e73e926999
feat(integration/fivetran): Fivetran connector integration (#9018)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-11-07 23:02:41 -08:00
Andrew Sikowitz
2c58c63780
fix(ingest/datahub-source): Order by version in memory (#9185) 2023-11-06 15:58:57 -05:00
Harshal Sheth
ec9725026d
chore(ingest): remove legacy memory_leak_detector (#9158) 2023-11-02 09:39:08 -07:00
Harshal Sheth
bab9d1c931
fix(ingest): fully support MCPs in urn_iter primitive (#9157) 2023-11-01 20:47:18 -07:00
Tony Ouyang
876de214c9
feat(ingest/mongodb): support stateful ingestion (#9118) 2023-11-01 12:43:17 +05:30
Harshal Sheth
55f14530a3
feat(ingest): bump sqlglot parser (#9155) 2023-11-01 12:42:52 +05:30
Harshal Sheth
73514ad9c5
fix(ingest): cleanup large images in CI (#9153) 2023-10-31 21:28:38 -07:00
Harshal Sheth
ea1273281e
feat(ingest): sql parser perf + asyncio fixes (#9119) 2023-10-30 20:57:59 -07:00
Harshal Sheth
ce0f36b8bc
feat(sqlparser): support more update syntaxes + fix bug with subqueries (#9105) 2023-10-30 14:23:19 -07:00
Harshal Sheth
0bd2d9a36c
feat(ingest): add entity_supports_aspect helper (#9120) 2023-10-30 14:22:05 -07:00
Mayuri Nehate
e02b9096bd
fix(ingest/snowflake): missing view downstream cll if platform instance is set (#8966) 2023-10-27 12:03:43 -07:00
Mayuri Nehate
5166d90433
fix(ingest/sql-server): update queries to use escaped procedure name (#9127) 2023-10-27 10:19:51 -07:00
nicholas.fwang
852267972c
refactor(ingestion/mongodb): Add platform_instance to mongodb (#8663)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-10-26 09:57:43 -07:00
Mayuri Nehate
f402090c1e
feat(ingest): support view lineage for all sqlalchemy sources (#9039) 2023-10-26 09:14:32 -07:00
Andrew Sikowitz
6c932e8afe
feat(ingest/bigquery): Attempt to support raw dataset pattern (#9109) 2023-10-25 13:17:09 -07:00
Andrew Sikowitz
8a80e858a7
build(ingest): Bump avro pin: security vulnerability (#9042) 2023-10-25 13:06:12 -04:00
siddiquebagwan-gslab
b612545220
feat(ingestion/redshift): support auto_incremental_lineage (#9010) 2023-10-25 15:26:06 +05:30
skrydal
378d84a346
fix(ui): Fixes handling of resources filters in UI (#9087) 2023-10-24 21:12:11 +01:00