935 Commits

Author SHA1 Message Date
sid-acryl
c97fd1f8c0
fix(ingest/tableau): honor the key projectNameWithin in pagination (#12107) 2024-12-16 11:32:05 -08:00
Jonny Dixon
06edf23a33
fix(ingestion/dremio): Ignore filtered containers in schema allowdeny pattern (#11959)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-12-13 14:55:31 +05:30
Harshal Sheth
e730afdb68
feat(ingest): improve query fingerprinting (#12104) 2024-12-12 13:51:18 -08:00
sid-acryl
2ec9cb0536
fix(ingestion/lookml): resolve CLL issue caused by column name casing. (#11876) 2024-12-12 15:32:56 +05:30
Harshal Sheth
93c8ae2267
fix(ingest/snowflake): handle dots in snowflake table names (#12105) 2024-12-12 15:31:32 +05:30
skrydal
b091e4615d
feat(ingest/kafka): Flag for optional schemas ingestion (#12077) 2024-12-11 16:02:31 +00:00
Aseem Bansal
ff7ac48021
fix(cli): don't use /api in gms url (#12083) 2024-12-11 16:11:08 +05:30
Harshal Sheth
d953718ab7
feat(ingest): allow max_workers=1 with ASYNC_BATCH rest sink (#12088) 2024-12-10 18:32:52 -05:00
sagar-salvi-apptware
57b12bd9cb
fix(ingest): replace sqllineage/sqlparse with our SQL parser (#12020) 2024-12-10 08:36:01 -08:00
Shirshanka Das
e4ea993df1
fix(py-sdk): DataJobPatchBuilder handling timestamps, output edges (#12067)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-12-09 19:40:31 -05:00
Tamas Nemeth
70bec48088
fix(ingest/gc): Additional dataprocess cleanup fixes (#12049) 2024-12-07 13:45:14 +01:00
hwmarkcheng
46aa962bad
feat(ingest/superset): initial support for superset datasets (#11972) 2024-12-06 13:48:00 -08:00
Gabe Lyons
1ed55f4176
feat(snowflake): adding oauth token bypass to snowflake (#12048) 2024-12-05 17:50:56 -08:00
Harshal Sheth
48b5a6221c
feat(ingest): add urn validation test files (#12036) 2024-12-05 08:32:31 -08:00
Tamas Nemeth
3c388a56a5
fix(ingest/gc): Adding test and more checks to gc source (#12027) 2024-12-05 14:19:44 +05:30
Tamas Nemeth
16a02411c3
fix(ingest/sagemaker): Gracefully handle missing model group (#12000) 2024-12-03 10:48:04 +01:00
Harshal Sheth
ce6474df5a
chore(ingest): remove deprecated calls to Urn.create_from_string (#11983) 2024-12-02 09:53:13 -08:00
k-bartlett
dc87b51369
feat(ingest): connector for Neo4j (#11526)
Co-authored-by: kbartlett <keith.bartlett@fullsight.org>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
Co-authored-by: Jay Feldman <8128360+feldjay@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: deepgarg-visa <149145061+deepgarg-visa@users.noreply.github.com>
Co-authored-by: Felix Lüdin <13187726+Masterchen09@users.noreply.github.com>
2024-12-02 14:53:28 +05:30
Harshal Sheth
a46de1ecf9
feat(ingest/athena): handle partition fetching errors (#11966) 2024-11-28 21:42:55 -05:00
Harshal Sheth
a92c6b2bb0
feat(ingest): add tests for colon characters in urns (#11976) 2024-11-28 21:42:40 -05:00
Harshal Sheth
189f8cefa7
feat(ingest): standardize sql type mappings (#11982) 2024-11-28 14:53:46 -05:00
sid-acryl
7bf7673735
refactor(ingest/powerbi): organize code within the module based on responsibilities (#11924) 2024-11-27 09:32:24 -08:00
Mayuri Nehate
b5fb691f0d
feat(ingest/kafka): improve error handling of oauth_cb config (#11929) 2024-11-25 10:31:35 +05:30
Mayuri Nehate
c3f9a9206d
feat(ingest/mssql): include stored procedure lineage (#11912)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-22 20:32:24 +05:30
sid-acryl
86b8175627
fix(ingestion/kafka): OAuth callback execution (#11900) 2024-11-22 13:08:23 +05:30
Harshal Sheth
1bfd4ee1d4
feat(ingest): handle mssql casing issues in lineage (#11920) 2024-11-21 17:16:04 -08:00
Harshal Sheth
7bdd0a8016
chore(ingest): always use urn creation helpers (#11911) 2024-11-21 13:49:41 +05:30
Harshal Sheth
42bb07a35e
fix(ingest/bigquery): increase logging in bigquery-queries extractor (#11774) 2024-11-20 13:35:01 -08:00
Harshal Sheth
3b415cde69
refactor(ingest/snowflake): move oauth config into snowflake dir (#11888) 2024-11-20 13:34:47 -08:00
Harshal Sheth
5519a330e2
chore(ingest): bump black (#11898) 2024-11-20 13:33:54 -08:00
Harshal Sheth
7dbb3e60cb
chore(ingest): start using explicit exports (#11899) 2024-11-20 13:33:30 -08:00
Harshal Sheth
85c8e605be
fix(ingest): consider sql parsing fallback as failure (#11896) 2024-11-19 15:06:16 -08:00
Andrew Sikowitz
94f1f39667
fix(ingest/partitionExecutor): Fetch ready items for non-empty batch when _pending is empty (#11885) 2024-11-18 17:25:43 -08:00
skrydal
2527f54972
feat(ingest/iceberg): Iceberg performance improvement (multi-threading) (#11182) 2024-11-18 19:41:45 +01:00
sagar-salvi-apptware
fd2da83ff4
feat(ingest/cassandra): Add support for Cassandra as a source (#11822) 2024-11-15 20:41:21 +05:30
Andrew Sikowitz
5ff6295b0f
fix(ingest/partition-executor): Fix deadlock by recomputing ready items (#11853) 2024-11-14 08:48:30 +01:00
Mayuri Nehate
383a70ac0a
fix(ingest/oracle): fix scheme for sqlalchemy < 2 (#11829) 2024-11-14 12:46:27 +05:30
sid-acryl
6454ff30ab
feat(ingest/powerbi): DatabricksMultiCloud native query support (#11756)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-11-12 19:26:06 +05:30
Mayuri Nehate
84c677629d
feat(ingest): add stateful ingestion support for file source (#11804) 2024-11-08 16:11:30 +05:30
Andrew Sikowitz
a1e16fc22a
fix(ingest/browsePathsV2): Emit Container aspect first, to avoid BrowsePathsV2 generation race condition (#11813) 2024-11-06 23:07:33 -08:00
Harshal Sheth
e609ff810d
feat(ingest/powerbi): improve reporting around m-query parser (#11763) 2024-10-31 16:27:45 -07:00
Harshal Sheth
143fc011fa
feat(ingest/powerbi): add timeouts for m-query parsing (#11753) 2024-10-30 19:40:45 +01:00
Tamas Nemeth
b33ad0a788
feat(ingest/datahub): Add way to filter soft deleted entities (#11738) 2024-10-30 17:41:45 +01:00
Harshal Sheth
6316e10d48
feat(ingest): check ordering in SqlParsingAggregator tests (#11735) 2024-10-29 17:50:37 +01:00
Aseem Bansal
02f0a3dee7
feat(ingest/transform): extend ownership transformer to other entities (#11700) 2024-10-29 15:28:41 +05:30
sagar-salvi-apptware
bb63cbd9db
fix(ingestion/bigquery): Add lineage extraction for BigQuery with GCS source (#11442) 2024-10-29 09:18:08 +01:00
Mayuri Nehate
87fa5b89e8
feat: multi-query lineage for temp upstreams (#11708) 2024-10-25 16:56:55 +05:30
Jonny Dixon
8b062eb8bd
feat(ingest/oracle): retire deprecated cx_oracle library (#11607)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-23 14:39:57 -07:00
Mayuri Nehate
eab2ac7a2e
feat(ingest/snowflake): support lineage via rename and swap using que… (#11600) 2024-10-23 14:02:08 +05:30
Julien Jehannet
326afc6308
fix(ingestion/glue): manage table names from resource_links from nearest catalog correctly (#11578) 2024-10-23 11:39:23 +05:30