3089 Commits

Author SHA1 Message Date
Harshal Sheth
c9b9afc530
feat(ingest/dbt): enable model performance and compiled code by default (#10164) 2024-04-02 09:29:27 -05:00
RyanHolstien
ef637ccb37
fix(docker): fix versioning for compose file post release (#10176) 2024-04-01 15:01:09 -05:00
Christian Groll
14bbc0b590
<fix>[oracle ingestion]: get database name when using service (#10158) 2024-04-01 12:57:52 -07:00
Valerii
3e39129f7b
fix(ingest/tableau) Fix Tableau lineage ingestion from Clickhouse (#10167) 2024-04-01 11:22:47 -07:00
Harshal Sheth
61c21e1a73
feat(ingest): bump sqlglot dep (#10144)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-03-28 17:53:28 -07:00
Shubham Jagtap
9f2c5d36f3
feat(ingestion/bigquery): BigQuery Owner Label to Datahub Ownership (#10047) 2024-03-28 15:50:25 -07:00
Mayuri Nehate
4e328c38a7
feat(ingest/looker): update browse paths to align with looker UI (#10147) 2024-03-28 11:57:43 -07:00
Robert Espinoza
4d69cea472
add row type for athena types (#10131)
Co-authored-by: Robert Espinoza <robert.espinoza@agilebits.com>
2024-03-28 13:15:03 +01:00
Harshal Sheth
e043587ac2
feat(ingest/bigquery): improve debug logs (#10101) 2024-03-27 15:22:26 -07:00
Harshal Sheth
25d9d6656c
feat(ingest): fix validators (#10115)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-03-27 15:20:55 -07:00
Huanjie Guo
654d991753
chore(ingest): update doc & log detail (#10139) 2024-03-27 13:39:06 -07:00
Mayuri Nehate
0361f2463d
feat(ingest/dynamodb): add support for classification (#10138) 2024-03-27 11:28:58 -07:00
Mayuri Nehate
9928d70c1d
fix(ingest/databricks): support hive metastore schemas with special char (#10049) 2024-03-27 09:41:46 -07:00
Harshal Sheth
f0bdc24fc9
feat(ingest/dbt): dbt model performance (#9992) 2024-03-26 17:18:54 -07:00
Harshal Sheth
1febe68b49
fix(ingest/dbt): respect convert_column_urns_to_lowercase in mapping CLL (#10132) 2024-03-26 14:42:47 -07:00
Harshal Sheth
a70e775a12
feat(ingest): emit platform for query entities (#10103) 2024-03-26 11:22:53 -07:00
Alexander
e4ebf34b6f
feat(ingest/bigquery): Respect dataset and table patterns when ingesting lineage via catalog api (#10080) 2024-03-26 10:03:28 -07:00
Tamas Nemeth
7e5610f358
feat(ingest/dagster): Dagster source (#10071)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
2024-03-25 13:28:35 +01:00
k7ragav
9de15a273a
fix(ingest/looker): use external_base_url for explore url generation (#10093) 2024-03-24 03:01:28 -04:00
dushayntAW
dd502ae662
fix(ingest): added new transformer to cleanup suffix/prefix in owner URN (#10067) 2024-03-22 15:23:03 +05:30
dushayntAW
22487376de
fix(ingestion/unity-catalog): patch owners and properties (#10086) 2024-03-22 15:22:38 +05:30
Harshal Sheth
af06f95c5e
fix(ingest/dbt): fix config validator for skip_sources_in_lineage (#10098) 2024-03-21 15:18:37 -07:00
Aseem Bansal
9659d60867
feat(ingest/datahub-gc): gc source to cleanup things (#10085) 2024-03-21 15:21:17 +05:30
alexs-101
e6e5c091ed
feat(tableau): ability to force extraction of table/column level linage from SQL queries (#9838) 2024-03-21 09:27:22 +01:00
Diego Monti
7a2d61d424
fix(ingest/metabase): Use connect_uri instead of display_uri to query Metabase API (#9996) 2024-03-21 09:23:33 +01:00
Harshal Sheth
c480b59b0b
feat(ingest/powerbi): add chart subtypes (#10076) 2024-03-21 09:20:40 +01:00
Harshal Sheth
8c21b178df
feat(ingest): support incremental column-level lineage (#10090) 2024-03-21 09:18:12 +01:00
Harshal Sheth
6c3834b38c
feat(ingest/dbt): add option to skip sources (#10077) 2024-03-21 09:10:24 +01:00
Ellie O'Neil
87169baf96
Clean up logic for dataset.py yaml loader (#10089) 2024-03-20 14:41:49 -07:00
Sergio Gómez Villamor
70ab759cea
feat(redshift): adds flag to skip all external tables (#10040) 2024-03-20 11:04:17 +01:00
siladitya
43ac405c57
fix(metadata-ingestion)glue connector failure when Optional field Type of PartitionKey is absent for a Table (#10052) 2024-03-20 11:02:28 +01:00
Ellie O'Neil
446fbe5784
fix(ingest/slack): add rate limiting to more places in slack source (#10082) 2024-03-20 13:56:06 +05:30
Ellie O'Neil
cc0552ac80
Add owners, tags, glossary terms to dataset yaml loader (#9859) 2024-03-19 17:27:04 -07:00
skrydal
9aa099f4b0
fix(ingestion): Handle Redshift string length limit in Serverless mode (#10051) 2024-03-19 18:54:41 +01:00
Harshal Sheth
1f9ce9d7d8
feat(ingest/dbt): point dbt assertions at dbt nodes (#10055) 2024-03-18 18:13:01 -07:00
Harshal Sheth
11eda4bc84
feat(ingest): refactor LineageConfig class (#10074) 2024-03-18 17:34:53 -07:00
Harshal Sheth
f4d4e791b0
feat(ingest): add sql formatter utility (#10064) 2024-03-18 17:28:32 -07:00
Harshal Sheth
630f2a13f6
feat(ingest): check for private import path usages (#10059) 2024-03-18 14:59:18 -07:00
Harshal Sheth
656f94b63c
feat(ingest): better rest emitter error message (#10073) 2024-03-18 14:27:43 -07:00
Harshal Sheth
05930560cc
feat(ingest/s3): set default spark version (#10057) 2024-03-18 14:27:01 -07:00
Harshal Sheth
104e78776d
fix(ingest): only auto-enable stateful ingestion if pipeline name is set (#10075) 2024-03-18 13:59:01 -07:00
dushayntAW
3a4bdef44a
fix(ingest/unity): generate sibling and lineage (#9894) 2024-03-18 20:22:17 +05:30
Harshal Sheth
d4d175da1a
fix(ingest/redshift): tweak lineage v2 queries (#10045) 2024-03-15 12:16:18 -07:00
Mayuri Nehate
77c72dad01
feat(ingest): add classification to bigquery, redshift (#10031) 2024-03-13 22:45:28 -07:00
Harshal Sheth
b0163c4885
feat(ingest): utilities for query logs (#10036) 2024-03-12 23:20:46 -07:00
Tamas Nemeth
4535f2adfd
fix(ingest/redshift): Temp table lineage fix (#10008)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-12 14:31:00 -07:00
Harshal Sheth
4155a621a6
feat(ingest/dbt): capture both raw and compiled code (#10026) 2024-03-12 12:54:03 -07:00
Mayuri Nehate
2de0e62ac4
feat(ingest): add classification for sql sources (#10013)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-12 09:23:20 -07:00
Tamas Nemeth
28f16aabb3
fix(ingest/teradata): small teradata improvements (#9953) 2024-03-12 14:22:17 +01:00
skrydal
2265ae9257
feat(ingestion): Support for Server-less Redshift (#9998)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-03-12 09:32:20 +01:00