4298 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
e847b58472
feat(ingest): ensure payload size constraints for queryProperties, querySubjects and upstreamLineage aspects (#14919)
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-06 18:10:59 +00:00
Anush Kumar
40b51ac2da
feat(ingestion): Added Databricks support to Fivetran source (#14897) 2025-10-06 10:06:37 -07:00
Deepak Garg
0613f8d471
feat(quickstart): bump min docker req and add validation (#14927) 2025-10-06 15:35:46 +05:30
skrydal
eea46700f6
chore(doc): Fix json schema generation after pydantic v2 move (#14926) 2025-10-04 13:32:45 +00:00
Aseem Bansal
1d0e3778a5
fix(ingest/grafana): add exception handling (#14921) 2025-10-03 20:30:59 +05:30
Aseem Bansal
cecb91b615
feat(ingest): add high level stage for ingestion (#14862) 2025-10-03 18:29:04 +05:30
Tamas Nemeth
78d258383f
fix(ingest/snowflake): Skip sql parsing if all the features disable in config where it is needed (#14908) 2025-10-03 11:08:26 +02:00
Tamas Nemeth
5b4a082c03
fix(ingest/snowflake): Fixed the Snowflake external URL generation issue for privatelink connections. (#14905) 2025-10-02 17:49:59 +02:00
Sergio Gómez Villamor
28b866a721
feat: ConnectionModel and DataHubGraph:get_urns_by_filter and StructuredProperties from saas (#14912) 2025-10-02 15:23:06 +00:00
Michael Maltese
5da54bf14d
feat(s3/ingest): performance improvements for get_dir_to_process and get_folder_info (#14709) 2025-10-02 15:51:02 +02:00
david-leifker
04d0a50118
feat(): Basepath support (#14866)
Co-authored-by: Esteban Gutierrez <esteban.gutierrez@acryl.io>
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2025-10-01 11:08:38 -05:00
Anush Kumar
00caa38adf
feat(ingestion/sqlglot): preserve CTEs when extracting SELECT from INSERT statements and add corresponding unit test (#14898) 2025-10-01 13:52:35 +02:00
sabdul
af12c2be37
docs(snaplogic): Add snaplogic to integration page (#14881) 2025-09-30 17:56:59 +09:00
Sergio Gómez Villamor
e9e18e4705
feat(secret): FileSecretStore and EnvironmentSecretStore (#14882) 2025-09-30 07:30:56 +00:00
Aseem Bansal
1e749d0c55
docs(ingest): decode strings for easier getting started (#14830) 2025-09-29 16:43:20 +05:30
Mayuri Nehate
e698f0bf1d
feat(sdk/search): add tags, glossary terms filter (#14873)
Co-authored-by: Mayuri N <mayuri.nehate@datahub.com>
2025-09-29 06:31:39 +00:00
Mayuri Nehate
8d13b03e85
feat(sdk/search): add owner filter (#14649)
Co-authored-by: Mayuri N <mayuri.nehate@datahub.com>
2025-09-29 04:35:29 +00:00
Sergio Gómez Villamor
900d7fe244
docs: hide pydantic_removed_field marked fields from documentation (#14829)
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-28 09:36:37 +00:00
Anush Kumar
c18b125a05
feat(ingestion): Enhanced column lineage extraction for Looker/LookML (#14826) 2025-09-26 09:27:18 -07:00
Tamas Nemeth
7e9c525448
fix(ingestion): Fix for module level variable caching in sqllite check (#14861) 2025-09-26 14:24:51 +02:00
Michael Maltese
eb066dcf1e
fix(ingest/gcs): fix a number of issues and add integration tests (#14857) 2025-09-25 14:53:33 -04:00
Michael Maltese
55d714e0cd
fix(ingest/mssql): don't split_statements on keywords inside bracketed identifiers (#14863) 2025-09-25 12:29:38 -04:00
Kevin Karch
941b3c725a
feat(ingest): add lowercase urn config option to metabase source (#14850) 2025-09-25 08:50:56 -04:00
Sergio Gómez Villamor
0fa4639857
fix(sdk): fixes imports for some SaaS classes (#14843)
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-09-25 14:32:55 +02:00
Tamas Nemeth
7e868493fe
fix(ingest/redshift): Fix for missing schema containers during ingestion (#14845) 2025-09-24 14:41:39 +02:00
skrydal
b0c9662be7
feature(transformers): Introduce Set browsePathsV2 transformer (#14825) 2025-09-23 19:20:01 +00:00
Sergio Gómez Villamor
ec166abade
tests(snaplogic): fix tests (#14848) 2025-09-23 18:18:32 +00:00
Anush Kumar
0f69e96078
feat(sdk): Added support for Change Audit Stamps in Dashboard and Chart entities (#14815) 2025-09-23 07:07:52 -07:00
Michael Maltese
9be65bd971
feat(ingest/tableau): enable extract_lineage_from_unsupported_custom_sql_queries by default (#14717) 2025-09-23 09:39:56 -04:00
sabdul
6a97baeee4
feat(ingestion/snaplogic): Add snaplogic as a source for metadata ingestion (#14231) 2025-09-23 20:26:24 +09:00
Sergio Gómez Villamor
77fa969b76
fix(ingestion): avoid pyarrow CVE-2023-47248 (#14819)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Piotr Skrydalewicz <piotr.skrydalewicz@acryl.io>
2025-09-23 08:19:03 +00:00
Harshal Sheth
a17fc4e0a8
chore(python): drop pydantic v1 support (#14014)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
Co-authored-by: Piotr Skrydalewicz <piotr.skrydalewicz@acryl.io>
2025-09-23 07:40:29 +00:00
Deepak Garg
00bc0db68c
feat(quickstart): bump min docker req (#14827) 2025-09-23 10:27:49 +05:30
Sergio Gómez Villamor
082719b197
docs(bigquery): improve docs about strategies for lineage/usage extraction (#14806) 2025-09-22 12:36:31 +00:00
Sergio Gómez Villamor
5c79114cb2
feat(ingestion/superset): add HTTP retry configuration to prevent infinite loops (#14818) 2025-09-19 13:13:37 +00:00
Anush Kumar
6bf511d8a8
fix(ingestion/looker): handle potential None values in explore dataset entity (#14813) 2025-09-18 15:08:07 -07:00
Anush Kumar
7c1200c704
refactor(ingestion): looker source migration to use SDKv2 entities (#14693) 2025-09-18 13:26:50 -07:00
Jonny Dixon
5be17c6444
feat(ingestion/tableau): parameter to have entity owners as email address of owner (#14724) 2025-09-18 15:25:14 +00:00
Anush Kumar
da885c6196
refactor(ingestion): lookml source migration to use SDKv2 entities (#14710) 2025-09-17 19:37:42 -07:00
Benjamin Maquet
5c07dc6e5a
feat(superset/preset): propagate chart & dashboard tags to DataHub (#14538) 2025-09-17 17:19:43 -04:00
Michael Maltese
e2c5767e39
fix(ge_profiler): support nonnull_count for complex types (#14631) 2025-09-17 15:04:31 +00:00
Abdullah
acffdce986
feat(dbt): add filtering for materialized nodes based on their physical location (#14689)
Co-authored-by: Abdullah Tariq <abdullah.tariq@adevinta.com>
Co-authored-by: skrydal <piotr.skrydalewicz@gmail.com>
2025-09-17 13:18:16 +00:00
Kevin Karch
002cc398d0
fix(ingest): change redash sql parse error to warnining (#14785) 2025-09-17 08:06:15 -04:00
skrydal
667b7cb12c
fix(sdk_v2/lineage): Fix handling of null platform (#14784) 2025-09-17 09:11:03 +02:00
Sergio Gómez Villamor
d82ae8014e
feat(bigquery): add created and modified timestamps to dataset containers (#14716)
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-15 18:24:58 +02:00
skrydal
36cf767d2d
Revert "chore(ingestion/iceberg): Safe-guard pyiceberg version before pydantic 1->2 transition (#14736)" (#14767) 2025-09-15 18:20:56 +02:00
Tamas Nemeth
2788b867c5
fix(ingest/pipeline): Fix BatchPartitionExecutor Shutdown Race Condition (#14750) 2025-09-15 14:59:16 +02:00
Jonny Dixon
92bcccd2bc
docs(ingestion/redshift): add required permissions for table and view discovery in pre-requisites documentation (#14765) 2025-09-15 12:55:20 +01:00
skrydal
b568eac9bc
chore(ingestion/iceberg): Include explicit extras in dependencies (#14766) 2025-09-15 11:47:04 +00:00
Sergio Gómez Villamor
492e28a938
feat(ingest/neo4j): migrate Neo4j source to DataHub Python SDK v2 (#14591)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-15 08:42:54 +00:00