3089 Commits

Author SHA1 Message Date
Harshal Sheth
b7ef234bc7
fix(ingest): fix deps for fivetran (#13385) 2025-05-02 12:31:07 -07:00
Aseem Bansal
03531520ce
fix(ingest/dynamodb): put primary keys correctly (#13373) 2025-05-02 15:25:34 +05:30
Aseem Bansal
42aeed074f
chore: upgrade dev dependencies mypy and ruff (#13375) 2025-05-02 15:02:00 +05:30
David Leifker
e6babc3b81 Revert "feat(ingestion): refactor api-tracing EmitMode"
This reverts commit bf598aed9687e9b08ccfbd72257fc890b505d775.
2025-05-01 21:06:10 -05:00
David Leifker
bf598aed96 feat(ingestion): refactor api-tracing EmitMode
* Created EmitMode to control write guarantees
    * IMMEDIATE, QUEUE, BLOCKING_QUEUE
2025-05-01 20:30:21 -05:00
david-leifker
8919154f02
fix(ingestion): fix cloud vs core logic (#13387) 2025-05-01 16:16:41 -05:00
Harshal Sheth
bb9838d789
fix(ingest): use server config method from graph (#13391) 2025-05-01 12:32:56 -07:00
david-leifker
d25d318233
feat(ingestion-sdk): OpenAPI & Tracing With SDK (#13349)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2025-04-30 21:09:12 -05:00
Harshal Sheth
d264a7afba
feat(ingest/dbt): make catalog.json optional (#13352) 2025-04-29 10:39:53 -07:00
Hyejin Yoon
f986315582
doc: Acryl to DataHub, datahubproject.io to datahub.com (#13252)
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-04-28 10:34:33 -04:00
Sergio Gómez Villamor
312e1ff573
chore(looker): reduce verbosity if error during initialization (#13331) 2025-04-28 13:11:58 +02:00
Sergio Gómez Villamor
e143422d2d
feat(dbt): log catalog and manifest metadata (#13329) 2025-04-28 10:19:03 +02:00
Jonny Dixon
9c718c870e
feat(ingestion/neo4j): Add stateful_ingestion and platform_instance capabilities to connector (#12631)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-04-26 11:52:20 +01:00
Michael Minichino
fee67788a3
fix(ingest/mode): Add pagination and warn on missing reports (#13322) 2025-04-25 18:21:27 -05:00
Harshal Sheth
ab760f0fef
fix(ingest): handle newlines in CLI-internal log tracker (#13340) 2025-04-25 13:08:10 -07:00
david-leifker
9b0634805a
feat(ingestion-openapi): patch support (#13282)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-04-25 13:54:28 -05:00
Aseem Bansal
49ee849382
chore: update lint dependencies (#13316)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2025-04-25 20:29:11 +05:30
Harshal Sheth
f6764ee17a
chore(sdk): rename _schema_classes to mark it as internal-only (#13309) 2025-04-24 12:22:01 +05:30
Sergio Gómez Villamor
1c5b7c18fc
chore(ingestion): removes ignore for SIM117 ruff rule (#13295) 2025-04-23 15:55:46 +02:00
Sergio Gómez Villamor
1563b0e9fb
fix(ingestion): use default generate_browse_path_v2 even if no pipeline_config (#13117) 2025-04-23 13:25:58 +02:00
Aseem Bansal
1de5fb3e6f
fix(cli): redact more secrets (#13287) 2025-04-23 14:49:26 +05:30
Sergio Gómez Villamor
0b6fd75d37
feat(slack): restores retry logic for get_user_to_be_updated (#13228) 2025-04-23 09:14:12 +02:00
Harshal Sheth
f48c6b53ee
feat(ingest/snowflake): show returned query row counts (#13246) 2025-04-21 14:41:40 -07:00
Chakru
54156ea78a
fix(cli): use patch to update dataset properties (#13226) 2025-04-21 14:41:31 -07:00
Harshal Sheth
08453cfbb1
fix(ingest/hive): support multiline view definitions (#13248) 2025-04-21 11:11:35 -07:00
Harshal Sheth
9f7f3cb886
chore(ingest/snowflake): remove unused query code (#13245) 2025-04-21 10:31:36 -07:00
Harshal Sheth
64bda48b51
feat(sdk): auto-fix bad entity type casing (#13218) 2025-04-20 21:40:00 -07:00
Harshal Sheth
5ba8b7d173
fix(ingest/fivetran): use project id by default for bigquery (#13250) 2025-04-20 21:39:40 -07:00
skrydal
47490ec050
feat(ingestion/iceberg): Add capability to extract namespace properties to the iceberg ingestor (#13238) 2025-04-17 16:29:43 +02:00
Hyejin Yoon
72aab9fe63
feat(sdk): add sdk lineage client (#13244)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2025-04-17 17:44:07 +09:00
Sergio Gómez Villamor
f3dfd86680
fix(looker): missing Looker Explore relationship when Look references multiple models (#13198) 2025-04-17 07:53:13 +02:00
Hyejin Yoon
bafd93d38f
feat(sdk): add mlmodel and mlmodelgroup (#13150) 2025-04-15 16:12:38 +09:00
Sergio Gómez Villamor
60b769fbf6
feat(ariflow-plugin): ability to disable datajob lineage (#13187) 2025-04-14 16:15:19 +02:00
Andrew Sikowitz
3e37f76428
feat(ingest/tableau): Allow specifying asset types for ingest_hidden_assets (#13190) 2025-04-11 20:07:37 -07:00
Gabe Lyons
1bcdda740d
feat(data contracts): supporting structured properties on data contracts (#13176) 2025-04-11 08:30:20 -07:00
Tamas Nemeth
e048cf7ce7
fix(ingest/sigma): Fix missing key in workspace_counts (#13182) 2025-04-11 15:28:43 +02:00
ryota-cloud
ca4eab52e4
VertexAI Connector (v3 - pipeline and pipeline task) (#12960) 2025-04-10 22:54:17 -07:00
Andrew Sikowitz
ca51df880f
fix(ingest/snowflake): Use CREATE change type when creating structured properties; support MCP headers (#13158) 2025-04-10 15:13:13 -05:00
Michael Minichino
0b105395e9
feat(ingest/powerbi): Support ODBC Data Source (#13090) 2025-04-10 08:57:37 -05:00
Hyejin Yoon
443134ca96
fix(ingest/mlflow): skip experiment/run ingestion for older version of mlflow (#13122) 2025-04-10 14:09:09 +09:00
Harshal Sheth
9f0c0aa3dd
refactor(ingest/sigma): make some error cases more clear (#13110)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-04-09 12:03:17 -07:00
skrydal
7b6ab3ba15
fix(ingest): Make workunit processor ensuring schema size more aggressive (#13153) 2025-04-09 12:02:32 -07:00
Harshal Sheth
e4a8c77344
fix(ingest): quote db name in streams query (#13131) 2025-04-09 09:03:36 -07:00
Sergio Gómez Villamor
e7d8f2913c
fix(snowflake): fixes deduplication and fingerprint requirements for Hex (#13121) 2025-04-09 10:17:43 +02:00
Sergio Gómez Villamor
75894399f0
fix(hex): fixes AccessType model (#13123) 2025-04-09 09:04:05 +02:00
Harshal Sheth
1fca9855ee
fix(ingest/snowflake): fix error on stored procs in non-SQL languages (#13127) 2025-04-08 17:00:56 -07:00
Sergio Gómez Villamor
5c7b8e10ce
fix(hex): filter out queries if non scheduled runs (#13126) 2025-04-08 20:55:28 +02:00
Sergio Gómez Villamor
4e48e098dc
fix(ingestion): fixes missing platform instance aspect for DataFlow entitiy (#13080) 2025-04-06 08:19:47 +02:00
skrydal
38f1553315
feat(ingestion): Refactoring timestamping logic for WorkUnits + custom logic for Iceberg (#13030)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-04-04 22:30:27 +02:00
Tamas Nemeth
df119cea1a
fix(ingest/mlflow): Fix stateful ingestion setup (#13084) 2025-04-04 18:47:19 +02:00