1604 Commits

Author SHA1 Message Date
Harshal Sheth
af06f95c5e
fix(ingest/dbt): fix config validator for skip_sources_in_lineage (#10098) 2024-03-21 15:18:37 -07:00
alexs-101
e6e5c091ed
feat(tableau): ability to force extraction of table/column level linage from SQL queries (#9838) 2024-03-21 09:27:22 +01:00
Harshal Sheth
c480b59b0b
feat(ingest/powerbi): add chart subtypes (#10076) 2024-03-21 09:20:40 +01:00
Harshal Sheth
8c21b178df
feat(ingest): support incremental column-level lineage (#10090) 2024-03-21 09:18:12 +01:00
Harshal Sheth
6c3834b38c
feat(ingest/dbt): add option to skip sources (#10077) 2024-03-21 09:10:24 +01:00
siladitya
43ac405c57
fix(metadata-ingestion)glue connector failure when Optional field Type of PartitionKey is absent for a Table (#10052) 2024-03-20 11:02:28 +01:00
Harshal Sheth
1f9ce9d7d8
feat(ingest/dbt): point dbt assertions at dbt nodes (#10055) 2024-03-18 18:13:01 -07:00
Harshal Sheth
630f2a13f6
feat(ingest): check for private import path usages (#10059) 2024-03-18 14:59:18 -07:00
Harshal Sheth
05930560cc
feat(ingest/s3): set default spark version (#10057) 2024-03-18 14:27:01 -07:00
dushayntAW
3a4bdef44a
fix(ingest/unity): generate sibling and lineage (#9894) 2024-03-18 20:22:17 +05:30
Harshal Sheth
d4d175da1a
fix(ingest/redshift): tweak lineage v2 queries (#10045) 2024-03-15 12:16:18 -07:00
Mayuri Nehate
77c72dad01
feat(ingest): add classification to bigquery, redshift (#10031) 2024-03-13 22:45:28 -07:00
Harshal Sheth
b0163c4885
feat(ingest): utilities for query logs (#10036) 2024-03-12 23:20:46 -07:00
Tamas Nemeth
4535f2adfd
fix(ingest/redshift): Temp table lineage fix (#10008)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-12 14:31:00 -07:00
Harshal Sheth
4155a621a6
feat(ingest/dbt): capture both raw and compiled code (#10026) 2024-03-12 12:54:03 -07:00
Mayuri Nehate
2de0e62ac4
feat(ingest): add classification for sql sources (#10013)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-12 09:23:20 -07:00
Harshal Sheth
7e2076e852
feat(ingest): add query formatting to sql aggregator (#10025) 2024-03-11 15:34:44 -07:00
Harshal Sheth
92a3ac6f11
fix(ingest): use contextvar for cooperative timeout (#10021) 2024-03-11 14:14:39 -07:00
ethan-cartwright
7540e642a5
fix(ingest/redshift): patch instead of replace redshift custom properties (#9293)
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-03-11 11:16:44 -07:00
Harshal Sheth
b6956f9a5c
feat(ingest): update sqlglot fork (#10022) 2024-03-11 15:22:30 +01:00
Mayuri Nehate
8047d0b514
feat(ingest/snowflake): generate query nodes for snowflake (#9966)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-06 15:11:37 -08:00
dushayntAW
68a26b47b2
fix(ingest/delta-lake): support parsing nested types correctly (#9862) 2024-03-06 19:22:01 +05:30
Shubham Jagtap
fda5eb89f7
feat(ingest): enable stateful_ingestion by default for DataHub rest sink (#9934) 2024-03-05 11:18:03 -08:00
Harshal Sheth
4189ca9532
feat(ingest/dbt): generate CLL for all node types (#9964) 2024-03-05 10:07:03 -08:00
Shubham Jagtap
4d053f1ad3
fix(ingestion/fivetran): Fix fivetran get connector jobs bug (#9975) 2024-03-05 17:31:28 +01:00
AvaniSiddhapuraAPT
782d33db99
fix(ingest/bigquery): escape special characters for table descriptions (#9932) 2024-03-05 21:00:09 +05:30
Harshal Sheth
5bee25fa20
feat(ingest): add Looks subtype + stop reemitting browsePathV2 (#9978) 2024-03-05 14:27:41 +01:00
Harshal Sheth
9dd0c37830
feat(ingest): fix fspath lint error (#9976) 2024-03-04 17:09:22 -08:00
AvaniSiddhapuraAPT
1b4f31bcfe
fix(ingest/json-schema): adding support descriptions for array (#9757) 2024-03-01 13:03:47 -08:00
david-leifker
ed10a8d8cc
feat(search): search access controls (#9892)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2024-02-28 16:57:26 -06:00
Harshal Sheth
1736edf8f5
feat(ingest): fix bugs in SqlParsingAggregator (#9926) 2024-02-28 15:06:33 +01:00
Aditya Malik
92b1cfa194
feat(ingest): Support for JSONL in s3 source with max_rows support (#9921)
Co-authored-by: Aditya <aditya.malik@quillbot.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-28 15:05:30 +01:00
Tamas Nemeth
f561efee83
fix(ingest/mysql): Adding support for bit type (#9950) 2024-02-28 15:00:14 +01:00
RyanHolstien
ddf0b7d2cd
feat(patch): refactor cll patch (#9922)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-27 12:00:44 -06:00
Aseem Bansal
336d0543c1
feat(ingest/csv-enricher): add support for ownership type urn (#9935) 2024-02-27 17:47:04 +00:00
Tamas Nemeth
d2d9661a1e
fix(ingest/patch): Adding proper json patch path quoting (#9872) 2024-02-26 18:46:21 -08:00
Harshal Sheth
ca613ba0e2
feat(ingest/tableau): add non-embedded custom sql to project container (#9875) 2024-02-26 15:08:17 -08:00
Shubham Jagtap
a1f2216da7
feat(ingest/qlik): Qlik cloud connector integration (#9682)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-26 11:27:37 -08:00
Shubham Jagtap
5921a334d3
feat(ingestion/trino): Add sibling support in ingestion (#9853) 2024-02-26 13:55:11 +05:30
Harshal Sheth
392158809d
feat(ingest/redshift): redshift lineage v2 (#9904) 2024-02-23 16:32:51 -08:00
Harshal Sheth
b01416c11b
feat(ingest): add timeout to sqlglot parser (#9868) 2024-02-23 11:11:02 -08:00
Tony Ouyang
943edc3462
feat(ingest/mongodb): improve sorting when downsampling collection schema (#9856) 2024-02-21 15:25:29 -08:00
Harshal Sheth
ef3a8148ec
fix(ingest): bump sqlglot + add some debug info to tests (#9867) 2024-02-21 13:07:59 -08:00
Harshal Sheth
b15b352a00
fix(ingest): better logging line attribution (#9876) 2024-02-21 11:37:57 -08:00
Harshal Sheth
f13ae77966
feat(ingest): add escape hatch methods to SqlParsingAggregator (#9860) 2024-02-21 11:33:05 -08:00
Jay Feldman
cda34b50fc
fix(ingest/looker): update for Looker query API breaking change (#9865) 2024-02-15 16:08:01 -08:00
Tony Ouyang
ae1806fefa
feat(ingest/dynamoDB): flatten struct fields (#9852)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-02-15 13:40:03 -08:00
sleeperdeep
7d73c41989
feat(ingest/oracle): support changing data dictionary (ALL_ or DBA_) (#8873)
Co-authored-by: sleeperdeep <--global>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-09 18:56:52 -05:00
Harshal Sheth
24da7a573b
test(ingest): use pytest-random-order for unit tests (#9753) 2024-02-09 18:20:05 -05:00
Sergio Gómez Villamor
93a7afbf0f
feat(ingest/glossary): handle ownership type in business glossary connector (#9805) 2024-02-09 16:30:51 -05:00