1041 Commits

Author SHA1 Message Date
Jonny Dixon
f38fa686c7 integration tests 2025-11-20 17:36:24 +00:00
Jonny Dixon
bbd1ad4e43 updates 2025-11-20 17:03:39 +00:00
Jonny Dixon
628bb454ce
Merge branch 'master' into implement-tableau-virtual-connections 2025-11-20 16:09:58 +00:00
kyungsoo-datahub
28da4476d8
feat(config): Hide input data in validation errors when DATAHUB_DEBUG is disabled (#15345) 2025-11-19 10:34:30 -08:00
Esteban Gutierrez
646c51279f
feat(ingestion): Make PySpark optional for S3 (#15123)
Co-authored-by: Piotr Skrydalewicz <piotr.skrydalewicz@acryl.io>
2025-11-19 12:11:40 +00:00
Esteban Gutierrez
2ec8c8e983
fix(ingestion): Fix confluent cloud test health check (#15346) 2025-11-19 08:02:12 +01:00
Tamas Nemeth
b686d2b501
feat(ingest/kafka-connect): Add Confluent Cloud connector and transform pipeline support (#14575) 2025-11-18 12:17:43 +01:00
Anush Kumar
88091ec2ed
refactor(lookml): Fixed log levels for source reporters (#15318) 2025-11-18 08:10:35 +00:00
bryanprosser-acryl
95ed3ea8b7
feat(openapi): rework to pull schemas from openapi v2 and v3 definitions (#14045) 2025-11-14 17:07:26 +00:00
Anush Kumar
474a880591
feat(lookml): Use Looker API to get fields and generate SQL using field chunks with ability to emit partial results (#15060) 2025-11-14 05:12:50 +00:00
Anush Kumar
2f31635fdb
feat(dbt): enhance DBTCloud integration with bulk job ingestion (#15264) 2025-11-13 11:55:08 -08:00
kyungsoo-datahub
8d18f30fac
feat(ingestion): Add secret masking framework (#15188) 2025-11-12 22:22:37 -08:00
kyungsoo-datahub
ae1406a3a7
fix(ingest/ci): fix integration test failures (#15284) 2025-11-12 21:56:17 -08:00
Sergio Gómez Villamor
f3dd874824
refactor(pydantic): remove Pydantic v1 legacy code and migrate fully to v2 (#15261) 2025-11-12 09:57:06 +00:00
kyungsoo-datahub
a526b877fa
feat(ingest/snowflake): add Streamlit apps ingestion support (#15272) 2025-11-11 11:43:35 -08:00
Tamas Nemeth
3728e0cc81
Revert "fix(ingest/dremio): Fix platform_instance URN generation" (#15236) 2025-11-07 19:18:09 +01:00
Anush Kumar
1c11ec033d
bug(unity-catalog): update parameter key in UnityCatalogApiProxy for MLFlow Model files (#15220) 2025-11-06 02:23:39 +00:00
Sergio Gómez Villamor
f1c95c79d1
chore: Remove Pydantic V1 deprecation warnings (#15057)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-05 14:31:43 +00:00
Anush Kumar
6048a8255e
feat(unity/mlmodel): add model signature and run details support in Unity Catalog Source (#15177) 2025-11-04 21:25:34 -08:00
Brock Griffey
680d16fa84
feat(ingestion/sql-queries): add performance optimizations, S3 support, temp table patterns (#14757)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-10-27 14:55:37 -04:00
Sergio Gómez Villamor
f844ea7b2d
chore(metabase): use non deprecated fields in ChartInfo and DashboardInfo aspects (#15102) 2025-10-27 09:17:08 +00:00
Jonny Dixon
2b7624321d integration tests 2025-10-26 21:06:47 +00:00
Tamas Nemeth
a865b6ba63
fix(ingest/dremio): Fix platform_instance URN generation (#15076) 2025-10-24 14:32:21 +02:00
Sergio Gómez Villamor
12cf28186c
fix(druid): fixes docker setup for postgres in druid integration tests (#15090) 2025-10-23 10:28:12 +00:00
Tamas Nemeth
e9becdd11c
feat(ingest/snowfle,bigquery): Stateful time window ingestion for queries v2 with bucket alignment (#15040) 2025-10-22 11:16:50 +02:00
Sergio Gómez Villamor
b3d354bf89
chore: migrate Unity catalog from deprecated SqlParsingBuilder to SqlParsingAggregator and remove SqlParsingBuilder (#15005) 2025-10-16 14:27:23 +00:00
Michael Maltese
4ce4718729
fix(ingest/mssql): ODBC connection always needs a database name (#14994) 2025-10-14 09:18:07 -04:00
Tamas Nemeth
e8c144362e
fix(ingest/dremio): Dremio sql parsing fix (#14974) 2025-10-13 11:03:19 +02:00
Anush Kumar
40b51ac2da
feat(ingestion): Added Databricks support to Fivetran source (#14897) 2025-10-06 10:06:37 -07:00
Aseem Bansal
cecb91b615
feat(ingest): add high level stage for ingestion (#14862) 2025-10-03 18:29:04 +05:30
Tamas Nemeth
5b4a082c03
fix(ingest/snowflake): Fixed the Snowflake external URL generation issue for privatelink connections. (#14905) 2025-10-02 17:49:59 +02:00
Michael Maltese
5da54bf14d
feat(s3/ingest): performance improvements for get_dir_to_process and get_folder_info (#14709) 2025-10-02 15:51:02 +02:00
Anush Kumar
c18b125a05
feat(ingestion): Enhanced column lineage extraction for Looker/LookML (#14826) 2025-09-26 09:27:18 -07:00
Michael Maltese
eb066dcf1e
fix(ingest/gcs): fix a number of issues and add integration tests (#14857) 2025-09-25 14:53:33 -04:00
Sergio Gómez Villamor
ec166abade
tests(snaplogic): fix tests (#14848) 2025-09-23 18:18:32 +00:00
Michael Maltese
9be65bd971
feat(ingest/tableau): enable extract_lineage_from_unsupported_custom_sql_queries by default (#14717) 2025-09-23 09:39:56 -04:00
sabdul
6a97baeee4
feat(ingestion/snaplogic): Add snaplogic as a source for metadata ingestion (#14231) 2025-09-23 20:26:24 +09:00
Harshal Sheth
a17fc4e0a8
chore(python): drop pydantic v1 support (#14014)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
Co-authored-by: Piotr Skrydalewicz <piotr.skrydalewicz@acryl.io>
2025-09-23 07:40:29 +00:00
Anush Kumar
7c1200c704
refactor(ingestion): looker source migration to use SDKv2 entities (#14693) 2025-09-18 13:26:50 -07:00
Anush Kumar
da885c6196
refactor(ingestion): lookml source migration to use SDKv2 entities (#14710) 2025-09-17 19:37:42 -07:00
Benjamin Maquet
5c07dc6e5a
feat(superset/preset): propagate chart & dashboard tags to DataHub (#14538) 2025-09-17 17:19:43 -04:00
Abdullah
acffdce986
feat(dbt): add filtering for materialized nodes based on their physical location (#14689)
Co-authored-by: Abdullah Tariq <abdullah.tariq@adevinta.com>
Co-authored-by: skrydal <piotr.skrydalewicz@gmail.com>
2025-09-17 13:18:16 +00:00
Sergio Gómez Villamor
492e28a938
feat(ingest/neo4j): migrate Neo4j source to DataHub Python SDK v2 (#14591)
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-15 08:42:54 +00:00
Brock Griffey
4244620e7a
feat(cassandra): Add optional SSL configuration (#14726) 2025-09-11 15:37:30 +00:00
Benjamin Maquet
9105241bfd
feat(superset/preset): add dataset and column description (#14426) 2025-09-08 16:35:43 +09:00
Tamas Nemeth
742409fddc
fix(ingestion/nifi): Change Zookeeper image to bitnamilegacy version in test (#14626) 2025-09-02 18:15:33 +05:30
Sergio Gómez Villamor
3401821f60
test(snowflake): add test for include_queries=False behavior (#14617)
Co-authored-by: Tim Drahn <tkdrahn@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-09-01 16:37:00 +00:00
skrydal
9698850f08
fix(ingestion/snowflake): Fix views schema extraction for corner cases (#14601) 2025-08-29 21:01:54 +00:00
sleeperdeep
73a351f92f
feat(mssql): quote_schemas & odbc_connect fields with related logic were added to SQLServerConfig (#13771)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-08-29 18:16:19 +00:00
Michael Minichino
340b1bf930
feat(ingest/excel): Add Excel Source (#13261)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-08-28 08:03:37 +00:00