911 Commits

Author SHA1 Message Date
skrydal
b091e4615d
feat(ingest/kafka): Flag for optional schemas ingestion (#12077) 2024-12-11 16:02:31 +00:00
Mayuri Nehate
355a7e69aa
feat(ingest/snowflake): allow option for incremental properties (#12080) 2024-12-11 16:08:39 +05:30
Shirshanka Das
e4ea993df1
fix(py-sdk): DataJobPatchBuilder handling timestamps, output edges (#12067)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-12-09 19:40:31 -05:00
haeniya
4811de1c1d
feat(ingestion/tableau): hidden asset handling (#11559)
Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>
2024-12-09 14:29:11 +05:30
hwmarkcheng
46aa962bad
feat(ingest/superset): initial support for superset datasets (#11972) 2024-12-06 13:48:00 -08:00
Mayuri Nehate
2fe21329fa
fix(ingest): avoid shell entities during view lineage generation (#12044) 2024-12-06 12:43:05 +05:30
Mayuri Nehate
8d15df0c11
feat(ingest/kafka): additional validation for oauth_db signature (#11996) 2024-12-05 12:41:43 +05:30
Harshal Sheth
2b42b29d2f
fix(ingest/tableau): make sites.get_by_id call optional (#12024) 2024-12-04 14:37:09 +05:30
Tamas Nemeth
230bd2674b
fix(ingest/looker): Don't fail on unknown liquid filters (#12014) 2024-12-03 20:01:12 +01:00
sagar-salvi-apptware
b538d4bc73
fix(ingestion/dremio): Fixed lineage view for dremio EE (#11990) 2024-12-02 15:02:47 +05:30
Harshal Sheth
189f8cefa7
feat(ingest): standardize sql type mappings (#11982) 2024-11-28 14:53:46 -05:00
Mayuri Nehate
ecba2244f0
fix(ingest/kafka):add poll for admin client for oauth_cb (#11985)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-11-28 19:23:30 +05:30
Margarida Fernandes
f3eda31326
feat(ingestion): extend feast plugin to ingest tags and owners (#11784)
Co-authored-by: John Joyce <john@acryl.io>
2024-11-27 15:41:04 -08:00
sid-acryl
7bf7673735
refactor(ingest/powerbi): organize code within the module based on responsibilities (#11924) 2024-11-27 09:32:24 -08:00
Mayuri Nehate
2e3b4294bb
feat(ingest/mssql): allow filtering by procedure_pattern (#11953) 2024-11-26 11:16:42 -05:00
sagar-salvi-apptware
b357f87f94
fix(ingest/sigma): migrate sigma workbooks from container to dashboard (#11939) 2024-11-26 18:56:30 +05:30
Mayuri Nehate
c3f9a9206d
feat(ingest/mssql): include stored procedure lineage (#11912)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-22 20:32:24 +05:30
sid-acryl
86b8175627
fix(ingestion/kafka): OAuth callback execution (#11900) 2024-11-22 13:08:23 +05:30
Harshal Sheth
1bfd4ee1d4
feat(ingest): handle mssql casing issues in lineage (#11920) 2024-11-21 17:16:04 -08:00
Harshal Sheth
5519a330e2
chore(ingest): bump black (#11898) 2024-11-20 13:33:54 -08:00
Harshal Sheth
7dbb3e60cb
chore(ingest): start using explicit exports (#11899) 2024-11-20 13:33:30 -08:00
sid-acryl
3f267afc97
fix(ingest/powerbi): m-query fixes (#11906)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-11-20 11:31:54 -08:00
skrydal
2527f54972
feat(ingest/iceberg): Iceberg performance improvement (multi-threading) (#11182) 2024-11-18 19:41:45 +01:00
sagar-salvi-apptware
fd2da83ff4
feat(ingest/cassandra): Add support for Cassandra as a source (#11822) 2024-11-15 20:41:21 +05:30
sid-acryl
6454ff30ab
feat(ingest/powerbi): DatabricksMultiCloud native query support (#11756)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-11-12 19:26:06 +05:30
Mayuri Nehate
84c677629d
feat(ingest): add stateful ingestion support for file source (#11804) 2024-11-08 16:11:30 +05:30
Aseem Bansal
f13d544152
fix(ingest/fivetran): do not materialise upstream (#11806) 2024-11-07 14:37:43 +05:30
Harshal Sheth
d5d4810457
fix(ingest/powerbi): change default for use_powerbi_email (#11742)
Co-authored-by: sid-acryl <155424659+sid-acryl@users.noreply.github.com>
2024-11-06 14:28:47 +05:30
sagar-salvi-apptware
5d17ecb290
feat(ingest/dremio): Dremio Source Ingestion (#11598)
Co-authored-by: Jonny Dixon <jonny.dixon@acryl.io>
Co-authored-by: Jonny Dixon <45681293+acrylJonny@users.noreply.github.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-11-05 22:06:35 +05:30
Tamas Nemeth
83ec73b7d1
fix(ingest/openapi): Fix openapi tests (#11789) 2024-11-05 15:25:35 +05:30
Harshal Sheth
143fc011fa
feat(ingest/powerbi): add timeouts for m-query parsing (#11753) 2024-10-30 19:40:45 +01:00
sid-acryl
0e62c699fc
fix(ingestion/powerbi): handle double quotes in M-query (#11743) 2024-10-29 21:47:19 -07:00
sid-acryl
8c53c54081
feat(ingestion/powerbi): ingest powerbi app (#11629) 2024-10-29 09:19:37 +01:00
Harshal Sheth
a11ac8d104
feat(ingest/fivetran): avoid duplicate table lineage entries (#11712) 2024-10-29 09:13:34 +01:00
Mayuri Nehate
87fa5b89e8
feat: multi-query lineage for temp upstreams (#11708) 2024-10-25 16:56:55 +05:30
haeniya
7c8dba4bd9
feat(ingest/tableau): support ingestion of access roles (#11157)
Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-24 11:56:00 -07:00
Harshal Sheth
b89ca3f081
feat(ingest/fivetran): support overriding destination db (#11701) 2024-10-24 19:31:34 +02:00
pankajmahato-visa
e569fbc8a6
fix(ingest/mongodb): Add Collection Name as Dataset Name in MongoDB (#11698) 2024-10-23 18:10:20 +02:00
Harshal Sheth
e96323a2a2
feat(ingest/fivetran): show connector filter reason (#11695) 2024-10-23 13:49:49 +05:30
skrydal
b7958c4a91
fix(ingest/bigquery): Fix tags urn/name ingestion for BigQuery (#11691) 2024-10-22 21:54:30 +02:00
Harshal Sheth
dd1a06fb55
feat(ingest/fivetran): add safeguards on table/column lineage (#11674) 2024-10-21 12:29:45 +05:30
Harshal Sheth
8f7f2c1724
feat(ingest/fivetran): protect against high sync volume (#11589) 2024-10-18 14:29:03 -07:00
Harshal Sheth
dfd7293f8d
feat(ingest/looker): include project name in model/explore properties (#11664)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-10-18 13:05:06 -07:00
Mayuri Nehate
179a6714a6
feat(ingest): ensure sqlite file delete on clean exit (#11612) 2024-10-18 10:33:28 +05:30
Harshal Sheth
6e3724b2da
perf(ingest): streamline CLL generation (#11645) 2024-10-17 17:50:59 -07:00
Tamas Nemeth
cf3634a6a9
fix(ingest/bigquery): Not setting platform instance for bigquery platform resources (#11659) 2024-10-17 19:57:02 +02:00
Harshal Sheth
8b42ac8cde
fix(ingest): run sqllineage in process by default (#11650) 2024-10-16 20:47:48 -07:00
Harshal Sheth
d34717fd82
fix(ingest): remove default value from DatahubClientConfig.server (#11570) 2024-10-16 13:50:33 -07:00
sid-acryl
ce909c8b8b
feat(ingest/powerbi): link cross workspace dataset into assets (#11560) 2024-10-16 08:28:05 -07:00
Tamas Nemeth
e76647dd7a
feat(ingest/bigquery): Generate platform resource entities for BigQuery labels (#11602)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2024-10-16 10:04:01 +02:00