1065 Commits

Author SHA1 Message Date
Anush Kumar
40b51ac2da
feat(ingestion): Added Databricks support to Fivetran source (#14897) 2025-10-06 10:06:37 -07:00
Aseem Bansal
cecb91b615
feat(ingest): add high level stage for ingestion (#14862) 2025-10-03 18:29:04 +05:30
Tamas Nemeth
78d258383f
fix(ingest/snowflake): Skip sql parsing if all the features disable in config where it is needed (#14908) 2025-10-03 11:08:26 +02:00
Tamas Nemeth
5b4a082c03
fix(ingest/snowflake): Fixed the Snowflake external URL generation issue for privatelink connections. (#14905) 2025-10-02 17:49:59 +02:00
Michael Maltese
5da54bf14d
feat(s3/ingest): performance improvements for get_dir_to_process and get_folder_info (#14709) 2025-10-02 15:51:02 +02:00
Anush Kumar
00caa38adf
feat(ingestion/sqlglot): preserve CTEs when extracting SELECT from INSERT statements and add corresponding unit test (#14898) 2025-10-01 13:52:35 +02:00
Sergio Gómez Villamor
e9e18e4705
feat(secret): FileSecretStore and EnvironmentSecretStore (#14882) 2025-09-30 07:30:56 +00:00
Mayuri Nehate
e698f0bf1d
feat(sdk/search): add tags, glossary terms filter (#14873)
Co-authored-by: Mayuri N <mayuri.nehate@datahub.com>
2025-09-29 06:31:39 +00:00
Mayuri Nehate
8d13b03e85
feat(sdk/search): add owner filter (#14649)
Co-authored-by: Mayuri N <mayuri.nehate@datahub.com>
2025-09-29 04:35:29 +00:00
Anush Kumar
c18b125a05
feat(ingestion): Enhanced column lineage extraction for Looker/LookML (#14826) 2025-09-26 09:27:18 -07:00
Tamas Nemeth
7e9c525448
fix(ingestion): Fix for module level variable caching in sqllite check (#14861) 2025-09-26 14:24:51 +02:00
Michael Maltese
55d714e0cd
fix(ingest/mssql): don't split_statements on keywords inside bracketed identifiers (#14863) 2025-09-25 12:29:38 -04:00
skrydal
b0c9662be7
feature(transformers): Introduce Set browsePathsV2 transformer (#14825) 2025-09-23 19:20:01 +00:00
Anush Kumar
0f69e96078
feat(sdk): Added support for Change Audit Stamps in Dashboard and Chart entities (#14815) 2025-09-23 07:07:52 -07:00
Harshal Sheth
a17fc4e0a8
chore(python): drop pydantic v1 support (#14014)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
Co-authored-by: Piotr Skrydalewicz <piotr.skrydalewicz@acryl.io>
2025-09-23 07:40:29 +00:00
Anush Kumar
7c1200c704
refactor(ingestion): looker source migration to use SDKv2 entities (#14693) 2025-09-18 13:26:50 -07:00
Jonny Dixon
5be17c6444
feat(ingestion/tableau): parameter to have entity owners as email address of owner (#14724) 2025-09-18 15:25:14 +00:00
Kevin Karch
002cc398d0
fix(ingest): change redash sql parse error to warnining (#14785) 2025-09-17 08:06:15 -04:00
skrydal
667b7cb12c
fix(sdk_v2/lineage): Fix handling of null platform (#14784) 2025-09-17 09:11:03 +02:00
Sergio Gómez Villamor
d82ae8014e
feat(bigquery): add created and modified timestamps to dataset containers (#14716)
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-15 18:24:58 +02:00
Sergio Gómez Villamor
492e28a938
feat(ingest/neo4j): migrate Neo4j source to DataHub Python SDK v2 (#14591)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-15 08:42:54 +00:00
Tamas Nemeth
01932d3f87
fix(ingest/pipeline): Fix for slow ingestion and incomplete ingestion report metrics (#14735) 2025-09-11 16:07:47 +02:00
Aseem Bansal
137ffb7d48
fix(ingest): only add to samples where platform match (#14722) 2025-09-11 13:26:39 +05:30
skrydal
5f23652fd3
fix(ingestion/iceberg): Improve iceberg source resiliency to server errors (#14731) 2025-09-11 00:57:03 +02:00
Tamas Nemeth
a82d4e0647
fix(ingest/athena): Fix Athena partition extraction and CONCAT function type issues (#14712) 2025-09-10 12:33:54 +02:00
Tamas Nemeth
4ea758da19
chore(ingest/sqlparser): Bump sqlglot to 27.12.0 (#14673) 2025-09-09 19:57:52 +02:00
skrydal
cc8e87143e
fix(cli): Fix to the deletion command (#14667)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-09-05 11:37:55 +00:00
Harshal Sheth
188e5af79e
feat(ingest): respect user email pattern in usage aggregator (#14562)
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-03 23:34:42 +00:00
Harshal Sheth
ceb8dd2e11
feat(sdk): add container support for charts and dashboards (#14641) 2025-09-03 12:36:07 -07:00
Tamas Nemeth
9ec58e3876
fix(ingest/unity): Backport proxy fix for unity catalog sql library (#14571) 2025-09-03 17:37:45 +02:00
Michael Maltese
117a9e91d0
fix(ingest/databricks): fix upstream external path lineage when using system tables (#14633) 2025-09-02 22:54:52 +02:00
Hyejin Yoon
584f6ce3d0
feat(ingest/unity) : add mlmodel / mlmodel version support (#14594) 2025-09-02 15:22:17 +09:00
Anush Kumar
be8c684b35
refactor(ingestion): renamed redshift lineage_v2 to lineage and other v2 nomenclatures (#14603) 2025-08-29 11:37:26 -07:00
Anush Kumar
b64f2a1533
refactor(ingestion): Updated Redshift lineage_v1 refs and removed v1 implementation (#14580) 2025-08-29 10:09:10 -07:00
Sergio Gómez Villamor
ea0677b918
feat(snowflake): add China region support (#14434)
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-29 10:01:52 +02:00
Sergio Gómez Villamor
67a441f312
fix(tool_meta_extractor): relax hex query detection to search entire query text (#14582) 2025-08-28 13:22:40 +02:00
Mayuri Nehate
fe8f108746
fix(sdk): make Filter type permissive of implicit and dict (#14569) 2025-08-28 15:22:00 +05:30
Michael Minichino
340b1bf930
feat(ingest/excel): Add Excel Source (#13261)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-08-28 08:03:37 +00:00
Michael Minichino
0252818bd0
feat(ingest/powerbi): Add ODBC SQL query parsing with DSN-to-database/schema mapping (#13752)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-27 15:25:23 +05:30
Michael Maltese
da127b92df
feat(s3): support wildcards in bucket name component of path_specs (#14549) 2025-08-26 16:29:52 -04:00
Jonny Dixon
0462415095
feat(ingestion/sql-queries): support incremental lineage (#14548) 2025-08-26 10:02:40 +01:00
Sergio Gómez Villamor
b3f20ee437
test(ingestion/json-schema): add test for JSON Schema $ref loop in definitions (#14536) 2025-08-25 07:21:50 +02:00
Sergio Gómez Villamor
b3fafc38be
feat(snowflake): ingest views from information schema (#14444)
Co-authored-by: Claude <noreply@anthropic.com>
2025-08-22 11:13:04 +02:00
Jonny Dixon
d7a64258d1
feat(ingestion/dbt): allow for dbt resources to be secondaries in sibling relationship on switch (#13230)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-08-22 07:32:18 +01:00
Harshal Sheth
e6ac57f465
chore(ingest): remove pickle from stateful ingestion (#14531) 2025-08-21 16:12:47 -07:00
Anush Kumar
bf09c5eebf
fix(ingestion): updated json_extract function for postgres db #12604 (#14533) 2025-08-21 15:42:51 -07:00
Benjamin Maquet
952807eb13
fix(cli): skip version warning message for servers (#14470) 2025-08-19 21:07:46 +05:30
Aseem Bansal
9a741dc92f
fix(ingest): exclude soft deleted entities from counts (#14490) 2025-08-19 17:10:53 +05:30
Sergio Gómez Villamor
22bce85410
feat(snowflake): enhance pushdown_deny_usernames by allowing patterns and add pushdown_allow_usernames (#14428) 2025-08-13 11:58:23 +02:00
Michael Maltese
738f16ce91
fix(ingest/hex): use viewer external URL for published projects/components (#14353) 2025-08-12 12:43:19 -04:00