3089 Commits

Author SHA1 Message Date
sagar-salvi-apptware
b357f87f94
fix(ingest/sigma): migrate sigma workbooks from container to dashboard (#11939) 2024-11-26 18:56:30 +05:30
Mayuri Nehate
3423e34899
feat(ingest): add more logs for kafka polling (#11954) 2024-11-26 16:22:06 +05:30
Harshal Sheth
9f9a8b1006
fix(ingest/tableau): handle none database field (#11950) 2024-11-26 11:30:56 +05:30
Mayuri Nehate
90fe14aadb
docs(ingest/bigquery): add partition support capability (#11940) 2024-11-25 12:14:10 -08:00
Mayuri Nehate
a1c71a48f2
feat(ingest/oracle): support profile limits for large tables (#11827) 2024-11-25 10:36:58 +05:30
Mayuri Nehate
b5fb691f0d
feat(ingest/kafka): improve error handling of oauth_cb config (#11929) 2024-11-25 10:31:35 +05:30
Mayuri Nehate
dd892dfbb1
fix(ingest/bigquery): ignore include constraints for biglake datasets (#11874) 2024-11-25 10:31:17 +05:30
Mayuri Nehate
c3f9a9206d
feat(ingest/mssql): include stored procedure lineage (#11912)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-22 20:32:24 +05:30
sid-acryl
86b8175627
fix(ingestion/kafka): OAuth callback execution (#11900) 2024-11-22 13:08:23 +05:30
Andrew Sikowitz
80ef5418cf
docs(ingest): Raise error on unsupported sqlite version (#11921)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-21 21:23:24 -08:00
Harshal Sheth
1bfd4ee1d4
feat(ingest): handle mssql casing issues in lineage (#11920) 2024-11-21 17:16:04 -08:00
Harshal Sheth
7bdd0a8016
chore(ingest): always use urn creation helpers (#11911) 2024-11-21 13:49:41 +05:30
Harshal Sheth
37bb0c9eae
fix(ingest/sql): disable patch checker (#11910) 2024-11-20 13:48:56 -08:00
Harshal Sheth
42bb07a35e
fix(ingest/bigquery): increase logging in bigquery-queries extractor (#11774) 2024-11-20 13:35:01 -08:00
Harshal Sheth
3b415cde69
refactor(ingest/snowflake): move oauth config into snowflake dir (#11888) 2024-11-20 13:34:47 -08:00
Harshal Sheth
5519a330e2
chore(ingest): bump black (#11898) 2024-11-20 13:33:54 -08:00
Harshal Sheth
7dbb3e60cb
chore(ingest): start using explicit exports (#11899) 2024-11-20 13:33:30 -08:00
sid-acryl
3f267afc97
fix(ingest/powerbi): m-query fixes (#11906)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-11-20 11:31:54 -08:00
Aseem Bansal
05d362a94e
fix(ingest): add error handling (#11905) 2024-11-20 20:47:39 +05:30
Harshal Sheth
524ef8c6d0
perf(ingest/redshift): limit copy lineage (#11662)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-11-20 13:59:05 +05:30
Harshal Sheth
85c8e605be
fix(ingest): consider sql parsing fallback as failure (#11896) 2024-11-19 15:06:16 -08:00
Andrew Sikowitz
94f1f39667
fix(ingest/partitionExecutor): Fetch ready items for non-empty batch when _pending is empty (#11885) 2024-11-18 17:25:43 -08:00
Raudzis Sebastian
19702c8225
fix(ingest/lookml): replace class variable with instance variable for improved encapsulation (#11881) 2024-11-18 15:45:05 -08:00
skrydal
2527f54972
feat(ingest/iceberg): Iceberg performance improvement (multi-threading) (#11182) 2024-11-18 19:41:45 +01:00
Tamas Nemeth
d660db11de
fix(ingest/airflow): Remove seems like not intended force debug mode which caused plugin fail on EMR (#11877) 2024-11-18 13:36:39 +01:00
Hendrik Richert
37fa07694f
feat(py-sdk): add cli version to ingestion headers (#11847)
Co-authored-by: Hendrik Richert <hendrik.richert@swisscom.com>
Co-authored-by: Shirshanka Das <shirshanka+github@gmail.com>
2024-11-18 16:55:40 +05:30
Shirshanka Das
3e128f4286
chore(structured-properties): add cli validation for entity types (#11863) 2024-11-16 13:45:39 -08:00
sagar-salvi-apptware
fd2da83ff4
feat(ingest/cassandra): Add support for Cassandra as a source (#11822) 2024-11-15 20:41:21 +05:30
Jonny Dixon
ab0b0a2eb9
fix(ingest/dremio): Dremio software jobs retrieval SQL query fix query error (#11817)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-11-14 14:47:50 +05:30
Andrew Sikowitz
5ff6295b0f
fix(ingest/partition-executor): Fix deadlock by recomputing ready items (#11853) 2024-11-14 08:48:30 +01:00
Mayuri Nehate
383a70ac0a
fix(ingest/oracle): fix scheme for sqlalchemy < 2 (#11829) 2024-11-14 12:46:27 +05:30
Aseem Bansal
8bbfa8ccdc
fix(ingest/dbt): handle multiple owners (#11848) 2024-11-13 18:42:11 +05:30
sid-acryl
6454ff30ab
feat(ingest/powerbi): DatabricksMultiCloud native query support (#11756)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-11-12 19:26:06 +05:30
Mayuri Nehate
84c677629d
feat(ingest): add stateful ingestion support for file source (#11804) 2024-11-08 16:11:30 +05:30
Mayuri Nehate
8ca8fd98ce
fix(ingest/sql-parsing): ignore processed query_id from temp upstream (#11798)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-08 07:42:03 +01:00
sid-acryl
35a7a2a708
fix(ingestion/powerbi): object has no attribute startswith (#11814) 2024-11-07 14:55:47 +05:30
Jonny Dixon
0d57fbd428
fix(ingest/dremio): update dremio sql query to retrieve queried datasets in sql jobs (#11801) 2024-11-07 14:54:10 +05:30
Aseem Bansal
f13d544152
fix(ingest/fivetran): do not materialise upstream (#11806) 2024-11-07 14:37:43 +05:30
Andrew Sikowitz
a1e16fc22a
fix(ingest/browsePathsV2): Emit Container aspect first, to avoid BrowsePathsV2 generation race condition (#11813) 2024-11-06 23:07:33 -08:00
Aseem Bansal
32878ab2e1
fix(ingest/gc): add limit, add actual loop for iterating over batches (#11809)
Co-authored-by: treff7es <treff7es@gmail.com>
2024-11-06 17:08:13 -06:00
Harshal Sheth
d5d4810457
fix(ingest/powerbi): change default for use_powerbi_email (#11742)
Co-authored-by: sid-acryl <155424659+sid-acryl@users.noreply.github.com>
2024-11-06 14:28:47 +05:30
sagar-salvi-apptware
5d17ecb290
feat(ingest/dremio): Dremio Source Ingestion (#11598)
Co-authored-by: Jonny Dixon <jonny.dixon@acryl.io>
Co-authored-by: Jonny Dixon <45681293+acrylJonny@users.noreply.github.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-11-05 22:06:35 +05:30
Harshal Sheth
a67981beda
fix(ingest/redshift): fix unload lineage in lineage_v2 (#11620)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-11-05 09:06:07 +01:00
Aseem Bansal
1f02c84480
feat(ingest/cli): add undo soft delete command (#11740) 2024-11-04 16:16:23 +05:30
Harshal Sheth
e609ff810d
feat(ingest/powerbi): improve reporting around m-query parser (#11763) 2024-10-31 16:27:45 -07:00
Felix Lüdin
8b72076e73
feat(forms/cli): add additional filters to the forms CLI (subtypes, platform instances, owners, tags and glossary terms) (#10979) 2024-10-31 15:55:17 -07:00
noggi
94bcb79b84
feat(ingestion): Add execution request cleanup job (#11765) 2024-10-31 10:59:21 -07:00
Harshal Sheth
143fc011fa
feat(ingest/powerbi): add timeouts for m-query parsing (#11753) 2024-10-30 19:40:45 +01:00
Harshal Sheth
799c452056
fix(ingest): reduce asyncio in check_upgrade (#11734) 2024-10-30 19:36:38 +01:00
Tamas Nemeth
b33ad0a788
feat(ingest/datahub): Add way to filter soft deleted entities (#11738) 2024-10-30 17:41:45 +01:00