139 Commits

Author SHA1 Message Date
Aseem Bansal
964a4b70c2
chore(airflow): update dev mypy to 1.14.1 (#13374) 2025-05-05 13:15:20 -07:00
Aseem Bansal
42aeed074f
chore: upgrade dev dependencies mypy and ruff (#13375) 2025-05-02 15:02:00 +05:30
david-leifker
d25d318233
feat(ingestion-sdk): OpenAPI & Tracing With SDK (#13349)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2025-04-30 21:09:12 -05:00
Hyejin Yoon
f986315582
doc: Acryl to DataHub, datahubproject.io to datahub.com (#13252)
Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-04-28 10:34:33 -04:00
Aseem Bansal
49ee849382
chore: update lint dependencies (#13316)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2025-04-25 20:29:11 +05:30
Aseem Bansal
64829a3279
chore(build): upgrade dependencies (#13286) 2025-04-23 14:49:40 +05:30
Harshal Sheth
82fafceba4
ci(airflow): separate airflow constraints from deps (#13291) 2025-04-22 13:54:15 -07:00
Harshal Sheth
a88e15c0d2
fix(ci): ensure extra airflow requirements are respected (#13289) 2025-04-22 12:08:28 -07:00
Chakru
54156ea78a
fix(cli): use patch to update dataset properties (#13226) 2025-04-21 14:41:31 -07:00
Andrew Sikowitz
d138a64a6a
ci(graphql,workflows): Format .md, .graphql, and workflow .yml files via prettier (#13220) 2025-04-16 16:55:51 -07:00
Sergio Gómez Villamor
60b769fbf6
feat(ariflow-plugin): ability to disable datajob lineage (#13187) 2025-04-14 16:15:19 +02:00
Aseem Bansal
2a75a981ca
chore(ci): upgrade ruff version (#13125) 2025-04-09 11:24:07 +05:30
Sergio Gómez Villamor
4e48e098dc
fix(ingestion): fixes missing platform instance aspect for DataFlow entitiy (#13080) 2025-04-06 08:19:47 +02:00
Rasnar
38e240e916
feat(ingest/airflow): platform_instance support in Airflow plugin (#12751)
Co-authored-by: rasnar <11248833+Rasnar@users.noreply.github.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-04-04 09:26:58 +02:00
Harshal Sheth
d3ef859802
fix(airflow): drop airflow 2.3 and 2.4 support (#13004) 2025-03-28 12:58:50 -07:00
kyungryun choi
26d9962a98
fix(airflow): fix logging typo (#13000) 2025-03-27 13:41:37 -07:00
Tamas Nemeth
0093aa8185
chore(ingestion/airflow): Example custom operator with sql parsing (#12959) 2025-03-27 09:18:29 +01:00
Aseem Bansal
d1494c2252
dev: upgrade ruff, remove 2 ruff ignore rules (#12702) 2025-02-24 14:07:57 +05:30
Aseem Bansal
effd4825f3
dev: reduce log in gradle commands (#12677) 2025-02-20 13:47:48 +05:30
Harshal Sheth
26e7743cfe
fix(dagster): support dagster 1.10.0 (#12648) 2025-02-16 22:23:57 -08:00
Harshal Sheth
79f844b208
chore(py): add unused-noqa lint rule (#12615) 2025-02-13 08:28:48 -08:00
Harshal Sheth
7472c535d3
feat(ci): add pytest hooks for updating golden files (#12581) 2025-02-12 15:32:31 -08:00
Andrew Sikowitz
b51327e0be
feat(versioning): Add V2 UI; make backend more synchronous; add to component library (#12542) 2025-02-04 09:57:41 -08:00
Tamas Nemeth
22e012a209
feat(ingestion/airflow): Add support for mutiple datahub emitter (#12398) 2025-01-29 16:44:31 +01:00
Harshal Sheth
a8d6c54965
feat(sdk): move version info to dedicated file (#12456) 2025-01-24 11:35:39 -08:00
Aseem Bansal
262dd76518
dev: remove black in favor of ruff for formatting (#12378) 2025-01-18 15:06:20 +05:30
Aseem Bansal
436b74cd3a
dev: move from flake8,isort to ruff (#12375) 2025-01-17 23:50:13 +05:30
Aseem Bansal
76e46b89db
dev(ingest): move modules from isort,flake8 to ruff (#12373) 2025-01-17 21:38:29 +05:30
Harshal Sheth
03e3f46175
fix(airflow): fix tests with Airflow 2.4 (#12279) 2025-01-07 07:46:35 -08:00
Harshal Sheth
ed8639e401
chore(python): test with python 3.11 (#11280)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-12-27 10:46:49 -08:00
Sergio Gómez Villamor
66df362c0f
fix(dagster): support dagster v1.9.6 (#12189) 2024-12-20 09:57:53 +01:00
Tamas Nemeth
b3a76cb308
feat(ingest/airflow): Add way to disable Airflow plugin without a restart (#12098) 2024-12-16 11:29:13 -08:00
Harshal Sheth
ecf6c8c588
feat(airflow): add DatahubRestHook.make_graph method (#12116) 2024-12-15 10:27:50 -05:00
Tamas Nemeth
ee82a88a75
fix(ingest/dagster): Fix Dagster build (#12121) 2024-12-13 11:51:42 -05:00
sagar-salvi-apptware
57b12bd9cb
fix(ingest): replace sqllineage/sqlparse with our SQL parser (#12020) 2024-12-10 08:36:01 -08:00
Harshal Sheth
2babbe6b0f
feat(airflow): drop Airflow < 2.3 support + make plugin v2 the default (#12056) 2024-12-09 11:08:49 -08:00
Steffen Grohsschmiedt
49b6284ebf
fix(airflow): fix AthenaOperator extraction (#11857)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-12-03 16:16:44 -08:00
Chakru
85b42e3ea5
build(coverage): enable code coverage for java and python (#11992)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-12-02 19:27:43 -06:00
Harshal Sheth
411e1a30ff
feat(airflow): show dag/task logs in CI (#11981) 2024-12-02 09:53:02 -08:00
JaeMoo Han
418259b8c4
fix(airflow): remove trino to presto mapping (#11925)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-25 12:13:26 -08:00
Tamas Nemeth
435792ca2e
fix(ingest/airflow): Add log to dag emit (#11880) 2024-11-18 17:00:22 +01:00
Mayuri Nehate
64355b490a
fix(ingest): update GX version having name arg (#11849) 2024-11-13 15:27:08 +01:00
Mayuri Nehate
5094dabdf4
feat(ingest/gx): support gx version 0.18.0 (#11823) 2024-11-11 14:38:15 +01:00
Tamas Nemeth
f6760436b7
feat(ingestion/dagster): Dagster assetless ingestion (#11262)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
2024-10-24 16:06:00 +02:00
Harshal Sheth
35f30b7d3c
feat(ingest): use mainline sqlglot (#11693) 2024-10-22 19:57:46 -07:00
Tamas Nemeth
16f9075869
fix(ingest/prefect): Fix prefect mypy errors (#11680) 2024-10-21 12:37:22 -05:00
Tamas Nemeth
ba7a43f530
fix(ingest/dagster): Fix JobSnapshot import is broken (#11672) 2024-10-18 20:16:08 +02:00
Harshal Sheth
d34717fd82
fix(ingest): remove default value from DatahubClientConfig.server (#11570) 2024-10-16 13:50:33 -07:00
Harshal Sheth
be1b8806a7
fix(airflow): fix lint related to dag_run field (#11616) 2024-10-15 14:09:41 +05:30
dushayntAW
f13dae1cd2
fix(airflow): add dag AllowDenyPattern config (#11472)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-11 10:46:01 -07:00