974 Commits

Author SHA1 Message Date
Harshal Sheth
da9121d9cc
feat(ingest): improve CI batches (#14239) 2025-07-28 16:34:15 -07:00
Harshal Sheth
604fd34f76
chore(ingest): reduce Okta test warnings (#14234) 2025-07-28 10:17:39 -07:00
Sergio Gómez Villamor
7a27e8e7fd
fix(ingest/sql-queries): handle string timestamps and empty table names (#14205)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-25 22:33:49 +02:00
Sergio Gómez Villamor
fa946725b2
test(kafka-connect): update Debezium connectors to fix broken integration tests (#14213) 2025-07-24 13:36:28 -07:00
Jonny Dixon
836e301e3b
feat(ingestion/grafana): Add datasets and charts to dashboards with lineage and tags. Lineage back to source (#12417)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-07-24 17:05:09 +01:00
Sergio Gómez Villamor
34d653e88f
feat(ingest/sql-queries): integrate SQL parsing aggregator with enhanced features (#14190) 2025-07-24 11:40:04 +02:00
Sergio Gómez Villamor
8fe536fd34
tests(sql-queries): add SQL queries integration tests (#14176) 2025-07-24 10:01:29 +02:00
Aarush
47e436fdfd
feat(postgres): add support for stored procedures in postgres. (#14102)
Co-authored-by: root <root@Aarush-PC.localdomain>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-07-22 13:31:42 -04:00
Sergio Gómez Villamor
da1c3d993d
fix(ingest/looker): prioritize explore project name over field-level source_file project assignment (#14119)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-22 13:30:05 +02:00
Tamas Nemeth
d63a35ab00
fix(ingest/athena): Make Athena simple column v1 conversion optional (#14112)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-07-17 09:15:48 +02:00
Harshal Sheth
29ba3673fd
feat(ingest): enable use_queries_v2 by default for snowflake/bigquery (#13601)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-07-17 09:03:45 +02:00
Petr Knez
311387410b
feat(dbt/ingestion): add support for nested meta properties mapping (#13396)
Co-authored-by: Petr Knez <pknez@gmail.com>
2025-07-17 15:12:51 +09:00
Harshal Sheth
c3dd9707bb
fix(ingest/tableau): optimize tableau test performance (#14036) 2025-07-11 08:37:27 -07:00
Benjamin Maquet
0eb2d1b2e2
feat(superset/ingest): add metrics to dataset columns (#13894)
Co-authored-by: Hyejin Yoon <0327jane@gmail.com>
2025-07-10 18:32:28 +09:00
Aseem Bansal
4509add782
chore(python): drop python 3.8 follow ups (#14004) 2025-07-09 13:24:33 +05:30
Sergio Gómez Villamor
5da40ec91e
fix(kafka-connect): handle SQL Server 3-level topic patterns and add Debezium integration tests (#13970)
Co-authored-by: Claude <noreply@anthropic.com>
2025-07-08 13:09:19 +02:00
Sergio Gómez Villamor
d85e2e5f28
fix(kafka-connect): escape dots in topic.prefix for regex patterns (#13955) 2025-07-08 09:59:03 +02:00
Tamas Nemeth
5b8d4bad7c
feat(ingest/athena): Iceberg partition columns extraction (#13607) 2025-07-07 14:46:40 +02:00
Tamas Nemeth
e331e807c6
fix(ingest/s3): Fix ingestion when path_spec had a wildcard character in the path (#13940) 2025-07-07 13:12:21 +02:00
Aseem Bansal
a7c5895d98
feat(ingest): add aspects by subtype in report, telemetry (#13921) 2025-07-03 17:07:39 +05:30
Hyejin Yoon
fedbfa3f7e
feat: update fivetran connector with new sdk (#13859) 2025-07-02 21:47:02 +09:00
Michael Maltese
0f0119f219
feat(ingestion): use approx_distinct when profiling Athena and Trino (#13671) 2025-06-25 16:29:26 -04:00
Aseem Bansal
f8c6db07d8
ingest(snowflake): remove email_as_user_identifier support (#13827) 2025-06-20 19:38:16 +05:30
Aseem Bansal
b3a25d6fbd
fix(ingest/bigquery): use email as user urn (#13831) 2025-06-20 18:45:41 +05:30
Shuixi Li
28d58e8973
fix(ingest/preset): url for view in source from edit view to explore view (#13666) 2025-06-13 15:20:58 +05:30
Tamas Nemeth
606439cafe
fix(ingest/bigquery): Set qualified name for bigquery containers (#13747) 2025-06-12 05:53:59 +02:00
Harshal Sheth
78cfc49703
chore(ingest): bump sqlglot dep (#13730) 2025-06-10 11:09:37 -07:00
Jonny Dixon
0fa88189c8
feat(ingestion/mssql): detection of rds or managed sql server for jobs history (#13731) 2025-06-10 18:41:26 +01:00
Jonny Dixon
612f68eced
feat(ingestion/sql-common): add column level lineage for external tables (#11997) 2025-06-04 13:24:44 +01:00
Atanu Chatterjee
487d52ad6e
fix(openapi_parser): add ability to parse openapi 3.0+ schemas (#13624) 2025-06-01 18:52:52 +02:00
Andrew R Smith
647fb792de
feat(ingest): add snowflake ingestion config options (#12841)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-05-27 09:22:16 -07:00
Sergio Gómez Villamor
d8d1de431f
fix(iceberg): update MinIO client commands for compatibility (#13631)
Co-authored-by: Claude <noreply@anthropic.com>
2025-05-27 11:15:41 +02:00
Tamas Nemeth
0eca4dfde2
fix(ingest/hive): Fix hive storage path formats (#13536) 2025-05-19 16:17:28 +02:00
Sergio Gómez Villamor
8cae980286
tests(ingestion): moving some tests so they are available for sdk users (#13540) 2025-05-19 08:39:53 +02:00
Jonny Dixon
132ff7081f
feat(ingestion/s3): Add externalUrls for datasets in s3 and gcs (#12763) 2025-05-17 17:03:40 +01:00
Harshal Sheth
d3944ded93
feat(ingest/snowflake): generate lineage through temp views (#13517) 2025-05-16 21:27:13 -07:00
Jonny Dixon
c756af31b1
feat(ingestion/looker): extract group_labels from looker and add as tags in datahub (#13503) 2025-05-14 13:08:13 +01:00
Michael Minichino
bc860181d8
fix(ingest/mode): Additional pagination and timing metrics (#13497)
Co-authored-by: NehaGslab <neha.marne@gslab.com>
2025-05-13 08:48:38 -05:00
Sergio Gómez Villamor
184fb09fc0
fix(mssql): improve stored proc lineage + add temporary_tables_pattern config (#13415)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-05-13 10:36:52 +02:00
Harshal Sheth
7c791db087
feat(ingest/sql): column logic + join extraction (#13426) 2025-05-12 17:19:44 -07:00
Sergio Gómez Villamor
fb7bcbaf17
tests(ingestion): fixes hex and hive docker flakiness (#13476) 2025-05-11 19:06:23 +02:00
Tamas Nemeth
60f79153ac
fix(ingest/hive): Fix hive properties with double colon (#13478) 2025-05-09 19:59:40 +02:00
Harshal Sheth
287f373a9c
fix(ingest/snowflake): fix previously broken tests (#13428) 2025-05-06 10:19:25 -07:00
skrydal
65d1c2b43c
feat(ingestion): Make jsonProps of schemaMetadata less verbose (#13416) 2025-05-06 16:18:26 +02:00
Harshal Sheth
cb3988a5f3
feat(ingest): associate queries with operations (#13404) 2025-05-05 11:27:33 -07:00
Aseem Bansal
03531520ce
fix(ingest/dynamodb): put primary keys correctly (#13373) 2025-05-02 15:25:34 +05:30
Harshal Sheth
34e74d826e
fix(ingest): update dremio golden files (#13381) 2025-04-30 17:45:54 -05:00
Hyejin Yoon
f986315582
doc: Acryl to DataHub, datahubproject.io to datahub.com (#13252)
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-04-28 10:34:33 -04:00
Michael Minichino
fee67788a3
fix(ingest/mode): Add pagination and warn on missing reports (#13322) 2025-04-25 18:21:27 -05:00
Sergio Gómez Villamor
a8637abfe2
tests(kafka): fixing flaky tests (#13171) 2025-04-22 12:58:47 +02:00