1732 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
5da40ec91e
fix(kafka-connect): handle SQL Server 3-level topic patterns and add Debezium integration tests (#13970)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-08 13:09:19 +02:00
Sergio Gómez Villamor
d85e2e5f28
fix(kafka-connect): escape dots in topic.prefix for regex patterns (#13955) 2025-07-08 09:59:03 +02:00
Harshal Sheth
6d2796a1c1
feat(ingest/snowflake): support TLL for stored procs (#13890) 2025-07-07 14:59:42 -07:00
Tamas Nemeth
5b8d4bad7c
feat(ingest/athena): Iceberg partition columns extraction (#13607) 2025-07-07 14:46:40 +02:00
Tamas Nemeth
e331e807c6
fix(ingest/s3): Fix ingestion when path_spec had a wildcard character in the path (#13940) 2025-07-07 13:12:21 +02:00
Aseem Bansal
a7c5895d98
feat(ingest): add aspects by subtype in report, telemetry (#13921) 2025-07-03 17:07:39 +05:30
Sergio Gómez Villamor
1561a6c8ca
feat(hex): add retry logic with exponential backoff for 429 rate limiting (#13905)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-03 11:12:31 +02:00
Tamas Nemeth
eaf2bf6dec
feat(ingest/kafka-connect): Add more connectors the regexp transformation support (#13748) 2025-07-03 08:57:50 +02:00
Hyejin Yoon
fedbfa3f7e
feat: update fivetran connector with new sdk (#13859) 2025-07-02 21:47:02 +09:00
sleeperdeep
70a39b70f2
fix(ingest): support ownership types in AddDatasetOwnership transformer (#13081) 2025-07-01 11:34:29 -07:00
Aseem Bansal
92784ec3a4
feat(ingest/lineage): generate static json lineage file (#13906) 2025-07-01 20:51:18 +05:30
Aseem Bansal
03309b7ffa
feat(mock-data-source): add first seen urn in report (#13889) 2025-06-30 15:15:50 +05:30
Harshal Sheth
05d029d690
feat(ingest/snowflake): add extra_info for snowflake (#13539) 2025-06-27 12:23:28 -07:00
Aseem Bansal
5759711992
fix(ingest/rest): out-of-date structured report being sent (#13866)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-06-27 15:07:28 +05:30
RyanHolstien
52e49eb79b
fix(emitter): fix emitter handling of unicode characters (#13867) 2025-06-26 10:43:55 -05:00
Sergio Gómez Villamor
9a32dd7f7f
feat(dremio): add configurable time range for query lineage extraction, sql aggregator report and fix schema_pattern filtering (#13613)
Co-authored-by: Claude <noreply@anthropic.com>
2025-06-26 15:28:42 +02:00
Michael Maltese
0f0119f219
feat(ingestion): use approx_distinct when profiling Athena and Trino (#13671) 2025-06-25 16:29:26 -04:00
Aseem Bansal
f8c6db07d8
ingest(snowflake): remove email_as_user_identifier support (#13827) 2025-06-20 19:38:16 +05:30
Aseem Bansal
b3a25d6fbd
fix(ingest/bigquery): use email as user urn (#13831) 2025-06-20 18:45:41 +05:30
Aseem Bansal
614e627720
feat(cli): --streaming-batch option delete large hierarchy (#13824) 2025-06-19 17:53:56 +05:30
Gabe Lyons
efa8d7dc27
fix(snowflake summary): fixing snowflake summary source (#13785) 2025-06-17 19:17:09 -04:00
Shuixi Li
28d58e8973
fix(ingest/preset): url for view in source from edit view to explore view (#13666) 2025-06-13 15:20:58 +05:30
Hyejin Yoon
45e5ef84be
docs: update the example scripts with the new sdk (#13717) 2025-06-12 14:00:26 +09:00
Tamas Nemeth
606439cafe
fix(ingest/bigquery): Set qualified name for bigquery containers (#13747) 2025-06-12 05:53:59 +02:00
Tamas Nemeth
6b8dfb0aa8
feat(ingest/glue): Lake formation tags ingestion (#13693) 2025-06-11 12:26:27 +02:00
david-leifker
794a2ab317
feat(config): update configuration caching (#13740) 2025-06-10 15:17:05 -05:00
Harshal Sheth
78cfc49703
chore(ingest): bump sqlglot dep (#13730) 2025-06-10 11:09:37 -07:00
Jonny Dixon
0fa88189c8
feat(ingestion/mssql): detection of rds or managed sql server for jobs history (#13731) 2025-06-10 18:41:26 +01:00
david-leifker
4f5e9c7508
feat(rest-emitter): set 60s ttl on gms config cache (#13729) 2025-06-10 09:21:29 -05:00
Tamas Nemeth
0eef7a02c7
feat(ingest/unity-catalog): Tag extraction (#13642) 2025-06-06 13:24:56 +02:00
Hyejin Yoon
e82cc6672b
feat(sdk) add dashboard & chart entity (#13669) 2025-06-06 16:28:28 +09:00
Hyejin Yoon
e169b4ac05
feat(sdk): add get_lineage (#13654) 2025-06-06 12:34:52 +09:00
Sergio Gómez Villamor
67844f1a53
feat(sdk): add EntityClient.delete documentation and tests (#13688) 2025-06-05 14:46:14 +02:00
Hyejin Yoon
83881e4129
feat(sdk): add add_lineage to lineage subclient (#13622) 2025-06-05 15:26:53 +09:00
Hyejin Yoon
488f011fa0
feat(sdk): add structured properties aspect (#13689) 2025-06-05 14:24:22 +09:00
Jonny Dixon
612f68eced
feat(ingestion/sql-common): add column level lineage for external tables (#11997) 2025-06-04 13:24:44 +01:00
Atanu Chatterjee
487d52ad6e
fix(openapi_parser): add ability to parse openapi 3.0+ schemas (#13624) 2025-06-01 18:52:52 +02:00
Hyejin Yoon
a142a9e2d2
feat(sdk): add dataflow and datajob entity (#13551) 2025-05-29 22:53:56 +09:00
david-leifker
492b55322f
feat(tracing): trace error log with timestamp & update system-metadata (#13628) 2025-05-27 17:33:40 -05:00
Andrew R Smith
647fb792de
feat(ingest): add snowflake ingestion config options (#12841)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-05-27 09:22:16 -07:00
Sergio Gómez Villamor
d8d1de431f
fix(iceberg): update MinIO client commands for compatibility (#13631)
Co-authored-by: Claude <noreply@anthropic.com>
2025-05-27 11:15:41 +02:00
David Leifker
b956132e3c Revert "feat(tracing): python logging & update system metadata on no-op"
This reverts commit 64f315eb64417467cfef5cec473068dc42886293.
2025-05-26 12:15:18 -05:00
David Leifker
64f315eb64 feat(tracing): python logging & update system metadata on no-op 2025-05-26 10:22:51 -05:00
Tamas Nemeth
2ffa84be5c
fix(ingest/datahub): Create Structured property templates in advance and batch processing (#13355)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2025-05-26 14:05:17 +02:00
Sergio Gómez Villamor
7bd7a33d0b
feat(hex): consider additional context when parsing hex query metadata (#13596) 2025-05-23 08:01:04 +02:00
Harshal Sheth
7e60587dec
fix(cli): strictly validate structured property values (#13576)
Co-authored-by: Chakravarthy Racharla <chakru.racharla@acryl.io>
2025-05-21 08:36:50 -07:00
Tamas Nemeth
9fca1737ff
fix(ingest/dbt): Fix urn validation in ownership type check (#13563) 2025-05-21 13:02:26 +02:00
Tamas Nemeth
0eca4dfde2
fix(ingest/hive): Fix hive storage path formats (#13536) 2025-05-19 16:17:28 +02:00
Sergio Gómez Villamor
8cae980286
tests(ingestion): moving some tests so they are available for sdk users (#13540) 2025-05-19 08:39:53 +02:00
Jonny Dixon
132ff7081f
feat(ingestion/s3): Add externalUrls for datasets in s3 and gcs (#12763) 2025-05-17 17:03:40 +01:00