927 Commits

Author SHA1 Message Date
Mayuri Nehate
87fa5b89e8
feat: multi-query lineage for temp upstreams (#11708) 2024-10-25 16:56:55 +05:30
haeniya
7c8dba4bd9
feat(ingest/tableau): support ingestion of access roles (#11157)
Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-24 11:56:00 -07:00
Harshal Sheth
b89ca3f081
feat(ingest/fivetran): support overriding destination db (#11701) 2024-10-24 19:31:34 +02:00
pankajmahato-visa
e569fbc8a6
fix(ingest/mongodb): Add Collection Name as Dataset Name in MongoDB (#11698) 2024-10-23 18:10:20 +02:00
Harshal Sheth
e96323a2a2
feat(ingest/fivetran): show connector filter reason (#11695) 2024-10-23 13:49:49 +05:30
skrydal
b7958c4a91
fix(ingest/bigquery): Fix tags urn/name ingestion for BigQuery (#11691) 2024-10-22 21:54:30 +02:00
Harshal Sheth
dd1a06fb55
feat(ingest/fivetran): add safeguards on table/column lineage (#11674) 2024-10-21 12:29:45 +05:30
Harshal Sheth
8f7f2c1724
feat(ingest/fivetran): protect against high sync volume (#11589) 2024-10-18 14:29:03 -07:00
Harshal Sheth
dfd7293f8d
feat(ingest/looker): include project name in model/explore properties (#11664)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-10-18 13:05:06 -07:00
Mayuri Nehate
179a6714a6
feat(ingest): ensure sqlite file delete on clean exit (#11612) 2024-10-18 10:33:28 +05:30
Harshal Sheth
6e3724b2da
perf(ingest): streamline CLL generation (#11645) 2024-10-17 17:50:59 -07:00
Tamas Nemeth
cf3634a6a9
fix(ingest/bigquery): Not setting platform instance for bigquery platform resources (#11659) 2024-10-17 19:57:02 +02:00
Harshal Sheth
8b42ac8cde
fix(ingest): run sqllineage in process by default (#11650) 2024-10-16 20:47:48 -07:00
Harshal Sheth
d34717fd82
fix(ingest): remove default value from DatahubClientConfig.server (#11570) 2024-10-16 13:50:33 -07:00
sid-acryl
ce909c8b8b
feat(ingest/powerbi): link cross workspace dataset into assets (#11560) 2024-10-16 08:28:05 -07:00
Tamas Nemeth
e76647dd7a
feat(ingest/bigquery): Generate platform resource entities for BigQuery labels (#11602)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2024-10-16 10:04:01 +02:00
skrydal
ebcce1f87b
fix(ingest/iceberg): Iceberg table name (#11599) 2024-10-11 18:48:25 +02:00
sid-acryl
59a2f70858
feat(ingest/powerbi): fix subTypes and add workspace_type_filter (#11523)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-11 08:54:50 -07:00
Shuixi Li
f147b51fc8
feat(ingest): add preset source (#10954)
Co-authored-by: MARK CHENG <hcheng@wealthsimple.com>
Co-authored-by: hwmarkcheng <94201005+hwmarkcheng@users.noreply.github.com>
2024-10-09 20:27:31 -07:00
Tamas Nemeth
3cb1678ecc
feat(ingest/bigquery): Add way to reference existing DataHub Tag from a bigquery label (#11544) 2024-10-09 17:01:34 +02:00
sid-acryl
6b961133e9
fix(ingestion/powerbi): fix for databricks lineage m-query pattern (#11462) 2024-10-07 14:42:44 -07:00
Mayuri Nehate
364496c3d7
fix(ingest/sigma): handle members api paginated response (#11535) 2024-10-04 11:32:57 -07:00
sid-acryl
67d7116055
fix(ingest/lookml): missing lineage for looker template -- if prod (#11426) 2024-10-01 11:26:00 -07:00
Jonny Dixon
a8e6a06ea4
docs(ingest/dbt): update run result paths examples (#11138) 2024-09-27 10:11:06 -07:00
haeniya
99bfcefb72
feat(ingestion/tableau): introduce project_path_pattern (#10855)
Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>
2024-09-27 09:46:39 -07:00
Mayuri Nehate
6a58493011
fix(ingest/bq): do not query PARTITIONS for biglake tables (#11463) 2024-09-27 16:46:37 +05:30
Harshal Sheth
d754650875
feat(ingest/dbt): produce multiple assertions for multi-table dbt tests (#11451) 2024-09-25 13:10:11 -07:00
sid-acryl
59b21ac24f
fix(ingestion/looker): skip personal folder independent looks (#11415) 2024-09-24 20:31:06 +05:30
sagar-salvi-apptware
d696dbef10
fix(ingest/mode): add connection timeouts to avoid RemoteDisconnected errors (#11245) 2024-09-23 12:28:30 -07:00
sid-acryl
9fb2df11f3
fix(ingest): sort by last modified not working in the UI (#11343) 2024-09-23 10:06:05 -07:00
Sergio Gómez Villamor
31edb46dbc
feat(ingestion): adds env property in ContainerProperties (#11214)
Co-authored-by: siladitya2 <siladitya2@gmail.com>
2024-09-18 14:56:52 +05:30
Harshal Sheth
3755731f0e
chore(ingest): improve code formatting (#11326) 2024-09-11 10:48:57 -07:00
sagar-salvi-apptware
852a23b845
feat(mode/ingest): Add support for missing Mode datasets in lineage (#11290) 2024-09-10 10:24:55 -07:00
Mayuri Nehate
cf49f80e77
feat(ingest/sql): auto extract and use mode query user metadata (#11307)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-09 12:38:24 -07:00
sid-acryl
3150d90bd1
fix(ingestion/tableau): restructure the tableau graphql datasource query (#11230)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-09 10:45:06 -07:00
Harshal Sheth
9589adebb3
feat(ingest): enable query usage stats by default (#11281) 2024-09-06 10:02:40 -07:00
Mayuri Nehate
08a5956e26
fix(bigquery): followups on bigquery queries v2 integration (#11291) 2024-09-04 07:42:35 -07:00
sleeperdeep
e34a628afe
fix(ingest/mssql): remove lower() method from sql_common get_db_name (#10773) 2024-09-03 12:41:23 -07:00
Mayuri Nehate
1f3688a1ed
feat(ingest/databricks): include metadata for browse only tables (#10766)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-02 19:09:45 +05:30
Mayuri Nehate
a7fc7f519a
feat(ingest/bq): integrate bigquery-queries into main source (#11247) 2024-08-30 18:16:45 -07:00
matthew-coudert-cko
c513e17dbb
feat(ingest/fivetran): support filtering on destination ids (#11277) 2024-08-30 13:25:18 -07:00
Andrew Sikowitz
fa1164aa63
feat(ingest/s3): Support reading S3 file type (#11177)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-08-30 12:15:12 +02:00
Alisa Aylward
675b225c5b
feat(ingest/snowflake): Add cluster formula to dataset properties (#11254) 2024-08-29 16:11:35 -07:00
Alisa Aylward
ca2e134d34
fix(ingest/snowflake): exclude snowflake excluded tags (#11250) 2024-08-27 08:48:59 -07:00
Felix Lüdin
ce99bc4f22
feat(ingest): add ingestion source for SAP Analytics Cloud (#10958)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-26 11:29:15 -07:00
Mayuri Nehate
223650dd7a
feat(ingest): add bigquery-queries source (#10994)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00
Tamas Nemeth
ef6a410091
feat(ingest/s3): Partition support improvements (#11083)
- Partition autodetection
- Option to find min/max/min-max partition of a dataset
- Generating Partition aspects
2024-08-22 17:55:43 +02:00
sid-acryl
dc30c0a0b7
feat(ingestion/looker): filter Looker dashboards by folder (#11205)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-22 14:10:00 +05:30
Alisa Aylward
88b3893b5c
feat(ingest/snowflake): allow iceberg tables in lineage and access metadata (#10961) 2024-08-22 01:20:51 +00:00
Andrew Sikowitz
30c4fa91ce
feat(ingest/mongodb): Ingest databases as containers (#11178)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-08-21 14:53:47 -04:00