48 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
60b769fbf6
feat(ariflow-plugin): ability to disable datajob lineage (#13187) 2025-04-14 16:15:19 +02:00
Rasnar
38e240e916
feat(ingest/airflow): platform_instance support in Airflow plugin (#12751)
Co-authored-by: rasnar <11248833+Rasnar@users.noreply.github.com>
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-04-04 09:26:58 +02:00
Harshal Sheth
d3ef859802
fix(airflow): drop airflow 2.3 and 2.4 support (#13004) 2025-03-28 12:58:50 -07:00
Tamas Nemeth
0093aa8185
chore(ingestion/airflow): Example custom operator with sql parsing (#12959) 2025-03-27 09:18:29 +01:00
Tamas Nemeth
b3a76cb308
feat(ingest/airflow): Add way to disable Airflow plugin without a restart (#12098) 2024-12-16 11:29:13 -08:00
Harshal Sheth
bc4c7c633e
docs(ingest): add docs on the SQL parser (#12103) 2024-12-12 13:51:24 -08:00
Harshal Sheth
2babbe6b0f
feat(airflow): drop Airflow < 2.3 support + make plugin v2 the default (#12056) 2024-12-09 11:08:49 -08:00
Harshal Sheth
d7ccf3b2c3
docs(airflow): add known limitations for automatic lineage (#11652) 2024-10-17 16:36:18 -07:00
dushayntAW
f13dae1cd2
fix(airflow): add dag AllowDenyPattern config (#11472)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-10-11 10:46:01 -07:00
Harshal Sheth
0414443b77
feat(airflow): add a render_templates config parameter (#11537) 2024-10-09 18:24:38 -07:00
Harshal Sheth
3d4b3b900d
fix(airflow): fix tox tests + update docs (#11125) 2024-08-12 10:15:42 -07:00
Harshal Sheth
2369032077
docs(airflow): update min version for plugin v2 (#11065) 2024-08-01 10:05:49 -07:00
Ellie O'Neil
f73149a059
docs(airflow): example query to get datajobs for a dataflow (#11034) 2024-07-31 17:01:09 +05:30
Hyejin Yoon
a6eb1f47c2
docs: standardize terminology to DataHub Cloud (#11003) 2024-07-30 09:52:51 +09:00
Francesco Macagno
750aab9a51
feat: allow task ownership as group (#10742) 2024-06-19 11:14:30 +02:00
dushayntAW
177a50f9a7
fix(ingestion/airflow-plugin): airflow remove old tasks (#10485) 2024-06-10 15:24:20 +05:30
dushayntAW
12455f95d7
doc(ingestion/airflow-plugin): update for developers (#10633) 2024-06-06 14:31:39 +05:30
Harshal Sheth
c1e7574b57
feat(ingest/airflow): support BigQueryInsertJobOperator (#10452) 2024-05-15 22:07:22 +02:00
Aseem Bansal
d3fb698d8d
fix(ingest): make gms url configuration resilient in rest emitter (#10316) 2024-04-18 14:46:32 +05:30
Harshal Sheth
76b5783e70
feat(ingest/airflow): support disabling iolet materialization (#10305) 2024-04-17 17:07:22 -07:00
Peng G
ea0ae8cb54
feat(airflow) Override datajob external_url (#9681)
Co-authored-by: Peng Gao <pgao@slack-corp.com>
2024-02-07 09:28:30 +01:00
Harshal Sheth
0d9aa26410
fix(ingest/airflow): compat with pluggy 1.0 (#9365) 2023-12-04 20:00:57 -05:00
Harshal Sheth
a8476ee657
fix(airflow): support inlet datajobs correctly in v1 plugin (#9331) 2023-11-30 18:34:48 -05:00
Jia (Jason) Teoh
c348f841c6
docs(airflow): update plugin setup docs to include UI setup approach (#9253) 2023-11-16 18:05:56 -06:00
Harshal Sheth
ec13847f54
feat(airflow): make RUN_IN_THREAD configurable (#9226) 2023-11-14 14:25:26 -05:00
Harshal Sheth
a300b39f15
feat(ingest/airflow): airflow plugin v2 (#8853) 2023-10-04 16:23:15 +05:30
Mayuri Nehate
e867dbc3da
ci: separate airflow build and test (#8688)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-08-30 14:08:42 -07:00
Harshal Sheth
022d1d0784
docs(ingest/airflow): add capture_executions to docs (#8662) 2023-08-20 04:44:28 +05:30
matthew-coudert-cko
5e45db9e60
docs(airflow): add docs on custom operators (#7913)
Co-authored-by: John Joyce <john@acryl.io>
2023-06-15 12:46:25 +05:30
Harshal Sheth
f314d896b9
docs(airflow): add debug guide on url generation (#7885)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-04-25 12:25:25 -07:00
Aseem Bansal
0e04e0c80f
docs(airflow): update with information for new plugin (#7732) 2023-04-04 15:20:02 +05:30
cburroughs
cc0772f8d8
feat(ingest): unbundle airflow plugin emitter dependencies (#7493)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 09:07:42 -08:00
Paul Logan
d2719cf0af
Saas docs migration (#6603) 2023-02-16 15:44:04 -06:00
Harshal Sheth
7d7da15742
docs(ingest/airflow): clarify Airflow 1.x docs for airflow plugin (#6761) 2022-12-14 23:48:06 +01:00
Harshal Sheth
9c04c63813
docs(ingest): airflow - clarify docs around 1.x compat (#6436) 2022-11-14 21:30:42 -08:00
Harshal Sheth
879894d702
feat(ingest): airflow - support enabled flag for airflow config (#6089) 2022-10-11 00:42:13 +02:00
Harshal Sheth
a0f59825ec
docs(airflow) add back auth token info for airflow (#5960)
Seems this PR https://github.com/datahub-project/datahub/pull/5703 was accidentally reverted.
2022-09-15 17:02:28 -05:00
GYUHOON KIM
4986eac8d2
docs: datahub_conn_id => conn_id in Airflow Integration (#5920)
Co-authored-by: 김규훈[W Stat] <gyuhoon.kim@webtoonscorp.com>
2022-09-15 10:21:03 -07:00
Harshal Sheth
7c00adc0ef
docs(airflow): show that rest host points at gms (#5913) 2022-09-15 09:25:45 -07:00
Maggie Hays
59f8c9b4fe
docs(site redesign): Overhaul Docs Site (#5731)
* adding new wip docs

* update to docs-website

* update to get started link

* lint cleanup

* lint cleanup
2022-08-26 16:29:01 -05:00
Tamas Nemeth
c66c263ceb
fix(airflow): Stable tag order in DataFlow/DataJobs (#5696) 2022-08-22 23:40:53 -07:00
Harshal Sheth
1b626c7652
docs(ingest): add info about datahub auth tokens with airflow (#5703) 2022-08-22 23:39:40 -07:00
Harshal Sheth
0cb3e07aa4
docs(airflow): fix note formatting (#5679) 2022-08-18 14:12:09 -07:00
Harshal Sheth
647d82d45f
docs(airflow): fix formatting (#5617) 2022-08-14 22:44:01 -07:00
Tamas Nemeth
c131e13582
feat(airflow): Airflow lineage ingestion plugin (#4833)
feat(airflow): Airflow lineage ingestion plugin (#4833)
2022-05-25 09:12:11 +02:00
Shirshanka Das
a518e3d13e
feat(cli): improve error reporting, make sink config optional (#4718) 2022-04-24 17:12:21 -07:00
Tamas Nemeth
f99d27fd8c
feat(ingest): airflow - add support to capture airflow executions, add high level dataflow jobs api to python sdk (#4615)
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-04-12 23:19:39 -07:00
Aseem Bansal
b3ef5ee489
docs(scheduling): re-arrange docs related to scheduling, lineage, CLI (#3669) 2021-12-07 10:09:59 -08:00