1732 Commits

Author SHA1 Message Date
Shubham Jagtap
fda5eb89f7
feat(ingest): enable stateful_ingestion by default for DataHub rest sink (#9934) 2024-03-05 11:18:03 -08:00
Harshal Sheth
4189ca9532
feat(ingest/dbt): generate CLL for all node types (#9964) 2024-03-05 10:07:03 -08:00
Shubham Jagtap
4d053f1ad3
fix(ingestion/fivetran): Fix fivetran get connector jobs bug (#9975) 2024-03-05 17:31:28 +01:00
AvaniSiddhapuraAPT
782d33db99
fix(ingest/bigquery): escape special characters for table descriptions (#9932) 2024-03-05 21:00:09 +05:30
Harshal Sheth
5bee25fa20
feat(ingest): add Looks subtype + stop reemitting browsePathV2 (#9978) 2024-03-05 14:27:41 +01:00
Harshal Sheth
9dd0c37830
feat(ingest): fix fspath lint error (#9976) 2024-03-04 17:09:22 -08:00
AvaniSiddhapuraAPT
1b4f31bcfe
fix(ingest/json-schema): adding support descriptions for array (#9757) 2024-03-01 13:03:47 -08:00
david-leifker
ed10a8d8cc
feat(search): search access controls (#9892)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2024-02-28 16:57:26 -06:00
Harshal Sheth
1736edf8f5
feat(ingest): fix bugs in SqlParsingAggregator (#9926) 2024-02-28 15:06:33 +01:00
Aditya Malik
92b1cfa194
feat(ingest): Support for JSONL in s3 source with max_rows support (#9921)
Co-authored-by: Aditya <aditya.malik@quillbot.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-28 15:05:30 +01:00
Tamas Nemeth
f561efee83
fix(ingest/mysql): Adding support for bit type (#9950) 2024-02-28 15:00:14 +01:00
RyanHolstien
ddf0b7d2cd
feat(patch): refactor cll patch (#9922)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-27 12:00:44 -06:00
Aseem Bansal
336d0543c1
feat(ingest/csv-enricher): add support for ownership type urn (#9935) 2024-02-27 17:47:04 +00:00
Tamas Nemeth
d2d9661a1e
fix(ingest/patch): Adding proper json patch path quoting (#9872) 2024-02-26 18:46:21 -08:00
Harshal Sheth
ca613ba0e2
feat(ingest/tableau): add non-embedded custom sql to project container (#9875) 2024-02-26 15:08:17 -08:00
Shubham Jagtap
a1f2216da7
feat(ingest/qlik): Qlik cloud connector integration (#9682)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-26 11:27:37 -08:00
Shubham Jagtap
5921a334d3
feat(ingestion/trino): Add sibling support in ingestion (#9853) 2024-02-26 13:55:11 +05:30
Harshal Sheth
392158809d
feat(ingest/redshift): redshift lineage v2 (#9904) 2024-02-23 16:32:51 -08:00
Harshal Sheth
b01416c11b
feat(ingest): add timeout to sqlglot parser (#9868) 2024-02-23 11:11:02 -08:00
Tony Ouyang
943edc3462
feat(ingest/mongodb): improve sorting when downsampling collection schema (#9856) 2024-02-21 15:25:29 -08:00
Harshal Sheth
ef3a8148ec
fix(ingest): bump sqlglot + add some debug info to tests (#9867) 2024-02-21 13:07:59 -08:00
Harshal Sheth
b15b352a00
fix(ingest): better logging line attribution (#9876) 2024-02-21 11:37:57 -08:00
Harshal Sheth
f13ae77966
feat(ingest): add escape hatch methods to SqlParsingAggregator (#9860) 2024-02-21 11:33:05 -08:00
Jay Feldman
cda34b50fc
fix(ingest/looker): update for Looker query API breaking change (#9865) 2024-02-15 16:08:01 -08:00
Tony Ouyang
ae1806fefa
feat(ingest/dynamoDB): flatten struct fields (#9852)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-02-15 13:40:03 -08:00
sleeperdeep
7d73c41989
feat(ingest/oracle): support changing data dictionary (ALL_ or DBA_) (#8873)
Co-authored-by: sleeperdeep <--global>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-09 18:56:52 -05:00
Harshal Sheth
24da7a573b
test(ingest): use pytest-random-order for unit tests (#9753) 2024-02-09 18:20:05 -05:00
Sergio Gómez Villamor
93a7afbf0f
feat(ingest/glossary): handle ownership type in business glossary connector (#9805) 2024-02-09 16:30:51 -05:00
Harshal Sheth
0d780e5f8f
feat(ingest): sql parsing aggregator (#9786) 2024-02-09 16:27:45 -05:00
Harshal Sheth
f8f1af2735
feat(ingest/dbt): speed up test result only ingestion (#9791) 2024-02-08 23:17:32 +01:00
Tamas Nemeth
3bfc1e46a7
fix(ingest/fivetran): Fixing issue when connector user is empty (#9774) 2024-02-05 11:51:57 -08:00
skrydal
bf1ce96ae6
feat(cli): Add support for custom ownership types to dataproduct CLI (#9762)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-05 08:29:14 -08:00
Andrea Scarpino
f743d3c044
feat(ingest/metabase): Allow to configure a different url for links (#9727) 2024-02-05 17:36:50 +05:30
Harshal Sheth
0e418b527e
fix(ingest): upgrade pytest-docker (#9765) 2024-02-01 16:33:15 -08:00
sid-acryl
533130408a
feat(ingestion/redshift): collapse lineage to permanent table (#9704)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: treff7es <treff7es@gmail.com>
2024-02-01 12:47:09 -08:00
sid-acryl
874c683f2a
fix(ingest/transformer): generate ownership aspect from handle_end_of_stream (#9720) 2024-01-31 14:25:08 -08:00
sid-acryl
ad2df22d78
feat(ingestion/dbt): multiple node owner separated by comma (#9740)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-01-31 16:38:26 +01:00
Aseem Bansal
3f9490d901
feat(cli): option to init via username password, add lint for smoke-test (#9675)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-31 14:42:40 +05:30
Ellie O'Neil
d1604b39cc
feat(ingestion/tableau) Ingest tags from datasource (#9730)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-29 17:17:07 -08:00
Harshal Sheth
1498c36875
chore(cli): drop support for python 3.7 (#9731) 2024-01-29 10:50:47 -08:00
Tamas Nemeth
90c88082b1
fix(ingest/s3): Not sorting schema fields to keep original order (#9349) 2024-01-29 14:14:34 +01:00
tom
f7f0b14f37
fix(ingestion/metabase): Fetch Dashboards through Collections (#9631)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-25 17:51:41 -08:00
Shubham Jagtap
f83a2fab44
fix(ingestion/bigquery): Table-view-snapshot Lineage Bug fix (#9579)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-25 18:48:41 +05:30
Andrew Sikowitz
53c7790f9a
feat(ingest/metabase): Use new sql parser; reduce error reporting levels (#9714) 2024-01-25 10:36:59 +01:00
Harshal Sheth
9b051e38d6
feat(ingest/dbt): support aws config without region (#9650)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-01-24 14:29:41 -08:00
Aseem Bansal
c4dec931a3
feat(ingest): handling for const in json schema (#9694)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-24 06:44:14 +00:00
Harshal Sheth
19b76c3307
feat(ingest/sql-parser): test redshift temp tables (#9653) 2024-01-23 08:37:53 +01:00
Harshal Sheth
f14dc4ded2
feat(ingest): improve sql tests + dbt reporting (#9649) 2024-01-23 08:37:23 +01:00
sleeperdeep
720296ea46
fix(ingestion/mssql): use platform_instance for mssql dataflow/datajob (#9527)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: sleeperdeep <--global>
2024-01-22 20:25:01 -08:00
Harshal Sheth
b94d463fe0
feat(ingest/lookml): support complex lookml manifests (#9688) 2024-01-22 16:23:56 -08:00