3904 Commits

Author SHA1 Message Date
Andrew Sikowitz
5ff6295b0f
fix(ingest/partition-executor): Fix deadlock by recomputing ready items (#11853) 2024-11-14 08:48:30 +01:00
Mayuri Nehate
383a70ac0a
fix(ingest/oracle): fix scheme for sqlalchemy < 2 (#11829) 2024-11-14 12:46:27 +05:30
Aseem Bansal
8bbfa8ccdc
fix(ingest/dbt): handle multiple owners (#11848) 2024-11-13 18:42:11 +05:30
sid-acryl
6454ff30ab
feat(ingest/powerbi): DatabricksMultiCloud native query support (#11756)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-11-12 19:26:06 +05:30
Mayuri Nehate
84c677629d
feat(ingest): add stateful ingestion support for file source (#11804) 2024-11-08 16:11:30 +05:30
Mayuri Nehate
8ca8fd98ce
fix(ingest/sql-parsing): ignore processed query_id from temp upstream (#11798)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-08 07:42:03 +01:00
sid-acryl
35a7a2a708
fix(ingestion/powerbi): object has no attribute startswith (#11814) 2024-11-07 14:55:47 +05:30
Jonny Dixon
0d57fbd428
fix(ingest/dremio): update dremio sql query to retrieve queried datasets in sql jobs (#11801) 2024-11-07 14:54:10 +05:30
Aseem Bansal
f13d544152
fix(ingest/fivetran): do not materialise upstream (#11806) 2024-11-07 14:37:43 +05:30
Andrew Sikowitz
a1e16fc22a
fix(ingest/browsePathsV2): Emit Container aspect first, to avoid BrowsePathsV2 generation race condition (#11813) 2024-11-06 23:07:33 -08:00
Aseem Bansal
32878ab2e1
fix(ingest/gc): add limit, add actual loop for iterating over batches (#11809)
Co-authored-by: treff7es <treff7es@gmail.com>
2024-11-06 17:08:13 -06:00
Harshal Sheth
d5d4810457
fix(ingest/powerbi): change default for use_powerbi_email (#11742)
Co-authored-by: sid-acryl <155424659+sid-acryl@users.noreply.github.com>
2024-11-06 14:28:47 +05:30
sagar-salvi-apptware
5d17ecb290
feat(ingest/dremio): Dremio Source Ingestion (#11598)
Co-authored-by: Jonny Dixon <jonny.dixon@acryl.io>
Co-authored-by: Jonny Dixon <45681293+acrylJonny@users.noreply.github.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-11-05 22:06:35 +05:30
Tamas Nemeth
83ec73b7d1
fix(ingest/openapi): Fix openapi tests (#11789) 2024-11-05 15:25:35 +05:30
Harshal Sheth
a67981beda
fix(ingest/redshift): fix unload lineage in lineage_v2 (#11620)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-11-05 09:06:07 +01:00
Aseem Bansal
1f02c84480
feat(ingest/cli): add undo soft delete command (#11740) 2024-11-04 16:16:23 +05:30
Harshal Sheth
e609ff810d
feat(ingest/powerbi): improve reporting around m-query parser (#11763) 2024-10-31 16:27:45 -07:00
Felix Lüdin
8b72076e73
feat(forms/cli): add additional filters to the forms CLI (subtypes, platform instances, owners, tags and glossary terms) (#10979) 2024-10-31 15:55:17 -07:00
chriscc2
9329916239
feat(ingest): unpin looker-sdk dependency (#11755)
Co-authored-by: Chris Chan <chriscchan@google.com>
2024-10-31 15:50:41 -07:00
noggi
94bcb79b84
feat(ingestion): Add execution request cleanup job (#11765) 2024-10-31 10:59:21 -07:00
Harshal Sheth
143fc011fa
feat(ingest/powerbi): add timeouts for m-query parsing (#11753) 2024-10-30 19:40:45 +01:00
Harshal Sheth
799c452056
fix(ingest): reduce asyncio in check_upgrade (#11734) 2024-10-30 19:36:38 +01:00
Harshal Sheth
93f76def1f
fix(ingest): pin teradata dep (#11760) 2024-10-30 10:38:33 -07:00
Tamas Nemeth
b33ad0a788
feat(ingest/datahub): Add way to filter soft deleted entities (#11738) 2024-10-30 17:41:45 +01:00
Harshal Sheth
c8704509a4
fix(ingest/bigquery): add missing path spec deps (#11748) 2024-10-30 11:13:15 +01:00
sid-acryl
0e62c699fc
fix(ingestion/powerbi): handle double quotes in M-query (#11743) 2024-10-29 21:47:19 -07:00
Harshal Sheth
cd1ad16852
docs(dbt-cloud): reference service accounts in docs (#11750) 2024-10-29 21:25:00 -07:00
Harshal Sheth
6316e10d48
feat(ingest): check ordering in SqlParsingAggregator tests (#11735) 2024-10-29 17:50:37 +01:00
Harshal Sheth
91fbd12f84
fix(ingest/unity): remove redundant check (#11732) 2024-10-29 17:49:37 +01:00
Harshal Sheth
b26da57928
feat(ingest): remove dep on termcolor (#11733) 2024-10-29 17:48:55 +01:00
Aseem Bansal
02f0a3dee7
feat(ingest/transform): extend ownership transformer to other entities (#11700) 2024-10-29 15:28:41 +05:30
Harshal Sheth
bea253a064
feat(ingest): unpin traitlets (#11731) 2024-10-29 09:21:20 +01:00
Mayuri Nehate
20d0999601
feat(ingest/databricks): report unique query count from usage (#11576) 2024-10-29 09:20:55 +01:00
sid-acryl
8c53c54081
feat(ingestion/powerbi): ingest powerbi app (#11629) 2024-10-29 09:19:37 +01:00
sagar-salvi-apptware
bb63cbd9db
fix(ingestion/bigquery): Add lineage extraction for BigQuery with GCS source (#11442) 2024-10-29 09:18:08 +01:00
Harshal Sheth
a11ac8d104
feat(ingest/fivetran): avoid duplicate table lineage entries (#11712) 2024-10-29 09:13:34 +01:00
Mayuri Nehate
e90d744cbd
feat(bigquery): support config for region qualifiers to fetch jobs (#11728) 2024-10-28 14:16:34 -07:00
Mayuri Nehate
87fa5b89e8
feat: multi-query lineage for temp upstreams (#11708) 2024-10-25 16:56:55 +05:30
haeniya
7c8dba4bd9
feat(ingest/tableau): support ingestion of access roles (#11157)
Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-24 11:56:00 -07:00
Harshal Sheth
b89ca3f081
feat(ingest/fivetran): support overriding destination db (#11701) 2024-10-24 19:31:34 +02:00
Tamas Nemeth
4f6f69d076
fet(ingest/bigquery): Add support ingesting foreign keys and primary keys for BigQuery tables (#11686) 2024-10-24 19:24:44 +02:00
Tamas Nemeth
f6760436b7
feat(ingestion/dagster): Dagster assetless ingestion (#11262)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
2024-10-24 16:06:00 +02:00
Semion Sidorenko
6787db7124
fix(ingest/superset): Don't set schema/db for druid (#11682) 2024-10-23 14:42:27 -07:00
Jonny Dixon
8b062eb8bd
feat(ingest/oracle): retire deprecated cx_oracle library (#11607)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-23 14:39:57 -07:00
pankajmahato-visa
e569fbc8a6
fix(ingest/mongodb): Add Collection Name as Dataset Name in MongoDB (#11698) 2024-10-23 18:10:20 +02:00
Mayuri Nehate
eab2ac7a2e
feat(ingest/snowflake): support lineage via rename and swap using que… (#11600) 2024-10-23 14:02:08 +05:30
Harshal Sheth
e96323a2a2
feat(ingest/fivetran): show connector filter reason (#11695) 2024-10-23 13:49:49 +05:30
Julien Jehannet
326afc6308
fix(ingestion/glue): manage table names from resource_links from nearest catalog correctly (#11578) 2024-10-23 11:39:23 +05:30
Harshal Sheth
581a39ad40
fix(ingest): add logging for mcp diff (#11683) 2024-10-22 20:22:40 -07:00
Harshal Sheth
35f30b7d3c
feat(ingest): use mainline sqlglot (#11693) 2024-10-22 19:57:46 -07:00