34 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
60b769fbf6
feat(ariflow-plugin): ability to disable datajob lineage (#13187) 2025-04-14 16:15:19 +02:00
Sergio Gómez Villamor
4e48e098dc
fix(ingestion): fixes missing platform instance aspect for DataFlow entitiy (#13080) 2025-04-06 08:19:47 +02:00
Rasnar
38e240e916
feat(ingest/airflow): platform_instance support in Airflow plugin (#12751)
Co-authored-by: rasnar <11248833+Rasnar@users.noreply.github.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-04-04 09:26:58 +02:00
Tamas Nemeth
0093aa8185
chore(ingestion/airflow): Example custom operator with sql parsing (#12959) 2025-03-27 09:18:29 +01:00
Harshal Sheth
7472c535d3
feat(ci): add pytest hooks for updating golden files (#12581) 2025-02-12 15:32:31 -08:00
Tamas Nemeth
22e012a209
feat(ingestion/airflow): Add support for mutiple datahub emitter (#12398) 2025-01-29 16:44:31 +01:00
Aseem Bansal
262dd76518
dev: remove black in favor of ruff for formatting (#12378) 2025-01-18 15:06:20 +05:30
Aseem Bansal
76e46b89db
dev(ingest): move modules from isort,flake8 to ruff (#12373) 2025-01-17 21:38:29 +05:30
Harshal Sheth
2babbe6b0f
feat(airflow): drop Airflow < 2.3 support + make plugin v2 the default (#12056) 2024-12-09 11:08:49 -08:00
Steffen Grohsschmiedt
49b6284ebf
fix(airflow): fix AthenaOperator extraction (#11857)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-12-03 16:16:44 -08:00
Harshal Sheth
411e1a30ff
feat(airflow): show dag/task logs in CI (#11981) 2024-12-02 09:53:02 -08:00
dushayntAW
f13dae1cd2
fix(airflow): add dag AllowDenyPattern config (#11472)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-11 10:46:01 -07:00
Andrew Sikowitz
3e3a05f09c
test(ingest/mcp_diff): Fallback to overwriting file on more complicated diffs (#11407)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-18 16:40:08 -07:00
dushayntAW
fca86bc5a9
fix(airflow): added support for jinja template for datahub emitter operator (#11300) 2024-09-17 12:57:45 -07:00
Harshal Sheth
1d83131601
fix(ingest/airflow): simplify env configuration (#11371) 2024-09-17 12:54:03 -07:00
Tamas Nemeth
1893b7b2dc
feat(ingest/dagster): Add automatic snowflake_pandas_io_manager asset capture (#11189)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
2024-08-28 09:22:11 +02:00
dushayntAW
0e698173fe
fix(ingestion/airflow-plugin): fixed missing inlet/outlets (#11101)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-18 20:30:27 -04:00
Harshal Sheth
840b15083a
fix(sql-parser): prevent bad urns from alter table lineage (#11092) 2024-08-08 14:05:55 -07:00
dushayntAW
2e496d599a
fix(ingestion/airflow-plugin): pipeline tasks discoverable in search (#10819) 2024-07-02 15:29:16 +05:30
Harshal Sheth
6f1f7691a0
fix(airflow): fix airflow snowflake tests (#10803) 2024-06-28 13:03:58 +05:30
dushayntAW
190f09ad7a
fix(ingestion/airflow-plugin): emit browsePathV2 (#10738) 2024-06-20 14:23:22 +05:30
dushayntAW
b2ba837987
fix(ingestion/airflow-plugin): fixed the failing pipeline (#10737) 2024-06-18 14:04:09 +02:00
Harshal Sheth
5da77df75d
feat(ingest/airflow): fix materialize_iolets bug (#10613) 2024-06-12 15:07:00 +02:00
Harshal Sheth
2e14f70864
test(ingest/sql): refactor CLL generator + add tests (#10580) 2024-05-23 18:11:22 -07:00
dushayntAW
6f02001501
fix(ingest/airflow-plugin): emit the operation aspect (#10402) 2024-05-03 19:20:00 +05:30
Harshal Sheth
07ef677ad3
feat(ingest): loosen airflow plugin dependencies requirements (#10106) 2024-03-27 22:32:53 +01:00
Harshal Sheth
0d780e5f8f
feat(ingest): sql parsing aggregator (#9786) 2024-02-09 16:27:45 -05:00
Harshal Sheth
a8476ee657
fix(airflow): support inlet datajobs correctly in v1 plugin (#9331) 2023-11-30 18:34:48 -05:00
Shubham Jagtap
bb7300251f
feat(ingestion/airflow): support datajobs as task inlets (#9211)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-11-10 15:59:18 -08:00
richenc
8b1d2094aa
feat(airflow): retry callback, support ExternalTaskSensor subclasses (#8514)
Co-authored-by: Richie Chen <richie.chen@hulu.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-10-27 10:36:47 -07:00
Harshal Sheth
8fb95e88a1
feat(sqlparser): parse create DDL statements (#9002) 2023-10-23 12:40:42 -07:00
Harshal Sheth
a300b39f15
feat(ingest/airflow): airflow plugin v2 (#8853) 2023-10-04 16:23:15 +05:30
Vincent Koc
d60f789519
chore: Refactor Python Codebase (#5113) 2022-07-06 16:09:27 +05:30
Tamas Nemeth
c131e13582
feat(airflow): Airflow lineage ingestion plugin (#4833)
feat(airflow): Airflow lineage ingestion plugin (#4833)
2022-05-25 09:12:11 +02:00