139 Commits

Author SHA1 Message Date
Harshal Sheth
771cc9ffe4
feat(airflow): add airflow 2.10 to test matrix (#11579) 2024-10-10 09:30:43 -07:00
Harshal Sheth
0414443b77
feat(airflow): add a render_templates config parameter (#11537) 2024-10-09 18:24:38 -07:00
David Schmidt
a87c123611
fix(dagster-plugin): Fix in/outs format and source config (#11481) 2024-10-01 20:59:23 -07:00
Andrew Sikowitz
3e3a05f09c
test(ingest/mcp_diff): Fallback to overwriting file on more complicated diffs (#11407)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-18 16:40:08 -07:00
dushayntAW
fca86bc5a9
fix(airflow): added support for jinja template for datahub emitter operator (#11300) 2024-09-17 12:57:45 -07:00
Harshal Sheth
1d83131601
fix(ingest/airflow): simplify env configuration (#11371) 2024-09-17 12:54:03 -07:00
Harshal Sheth
dfa97758a1
chore(py): cleanup python CI (#11324) 2024-09-11 10:49:16 -07:00
Tamas Nemeth
c05b89541d
fix(ingest/prefect): Temporary pinning Prefect 2.x until we can upgrade to 3.x (#11302) 2024-09-04 15:00:24 -07:00
dushayntAW
6204cba2bb
fix(ingestion/prefect-plugin): Prefect plugin (#10643)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-08-29 15:40:10 +02:00
Aseem Bansal
15c1cfc386
ci(build): update outdated action & pin deepdiff lib (#11260) 2024-08-28 19:53:42 +05:30
Tamas Nemeth
1893b7b2dc
feat(ingest/dagster): Add automatic snowflake_pandas_io_manager asset capture (#11189)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
2024-08-28 09:22:11 +02:00
Mayuri Nehate
9568a4254d
feat: separate great-expectations action package (#11096)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-21 12:13:36 -04:00
dushayntAW
0e698173fe
fix(ingestion/airflow-plugin): fixed missing inlet/outlets (#11101)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-18 20:30:27 -04:00
Harshal Sheth
3d4b3b900d
fix(airflow): fix tox tests + update docs (#11125) 2024-08-12 10:15:42 -07:00
Harshal Sheth
840b15083a
fix(sql-parser): prevent bad urns from alter table lineage (#11092) 2024-08-08 14:05:55 -07:00
Harshal Sheth
f78b6c08fb
fix(py): remove dep on types-pkg_resources (#11076) 2024-08-02 17:27:54 +05:30
Tamas Nemeth
123e84cfed
fix(airflow/build): Pinning mypy (#10972) 2024-07-23 12:30:34 -07:00
Harshal Sheth
3733a408fd
chore(airflow): add python 3.11 w/ Airflow 2.9 to CI (#10941) 2024-07-19 10:03:12 +05:30
Ellie O'Neil
0b64de8f2b
fix(airflow): Add comma parsing of owners to DataJobs (#10903) 2024-07-17 14:58:25 -07:00
Harshal Sheth
bb24651264
fix(airflow): add error handling around render_template() (#10907) 2024-07-16 12:27:37 -07:00
dushayntAW
2e496d599a
fix(ingestion/airflow-plugin): pipeline tasks discoverable in search (#10819) 2024-07-02 15:29:16 +05:30
Harshal Sheth
6f1f7691a0
fix(airflow): fix airflow snowflake tests (#10803) 2024-06-28 13:03:58 +05:30
dushayntAW
190f09ad7a
fix(ingestion/airflow-plugin): emit browsePathV2 (#10738) 2024-06-20 14:23:22 +05:30
Francesco Macagno
750aab9a51
feat: allow task ownership as group (#10742) 2024-06-19 11:14:30 +02:00
dushayntAW
b2ba837987
fix(ingestion/airflow-plugin): fixed the failing pipeline (#10737) 2024-06-18 14:04:09 +02:00
Harshal Sheth
b861a7007d
fix(ingest): fix dagster plugin release process (#10713) 2024-06-15 18:09:11 +02:00
Harshal Sheth
5da77df75d
feat(ingest/airflow): fix materialize_iolets bug (#10613) 2024-06-12 15:07:00 +02:00
dushayntAW
177a50f9a7
fix(ingestion/airflow-plugin): airflow remove old tasks (#10485) 2024-06-10 15:24:20 +05:30
Harshal Sheth
2e14f70864
test(ingest/sql): refactor CLL generator + add tests (#10580) 2024-05-23 18:11:22 -07:00
Harshal Sheth
ebb95b8a9a
fix(ingest/airflow): fix support for bigquery insert job operator (#10567) 2024-05-22 07:38:36 +02:00
Harshal Sheth
c1e7574b57
feat(ingest/airflow): support BigQueryInsertJobOperator (#10452) 2024-05-15 22:07:22 +02:00
dushayntAW
a86ec7b088
fix(ingestion/airflow-plugin): bumping up the openlineage-airflow version (#10457) 2024-05-09 13:00:40 +05:30
dushayntAW
4d20c22a49
fix(ingestion/airflow-plugin): warning log for non-materialized iolets (#10421) 2024-05-08 14:11:16 +05:30
dushayntAW
6f02001501
fix(ingest/airflow-plugin): emit the operation aspect (#10402) 2024-05-03 19:20:00 +05:30
Hyejin Yoon
f9dbe985d3
fix: make next as default version & create redirection (#10309)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-19 15:12:32 +09:00
Aseem Bansal
d3fb698d8d
fix(ingest): make gms url configuration resilient in rest emitter (#10316) 2024-04-18 14:46:32 +05:30
Harshal Sheth
76b5783e70
feat(ingest/airflow): support disabling iolet materialization (#10305) 2024-04-17 17:07:22 -07:00
Marcin Szymański
47bf1f9858
fix(ingestion/airflow-plugin): replace deprecated calls (#10238) 2024-04-10 10:16:44 -07:00
Harshal Sheth
3d1571dd6f
fix(airflow): disable OL regardless of plugin status (#10250) 2024-04-09 10:54:12 +02:00
dushayntAW
b6c4d9c0bb
fix(ingestion/airflow-plugin): emit tag to create tag entities (#10216) 2024-04-08 17:30:50 +05:30
Harshal Sheth
f38b626d3d
fix(build): avoid nested gradle commands (#10198) 2024-04-04 12:46:00 -07:00
Harshal Sheth
32a2de4dfc
feat(ingest/airflow): allow plugin to load on listener exception (#10152) 2024-03-28 12:43:37 -07:00
Harshal Sheth
07ef677ad3
feat(ingest): loosen airflow plugin dependencies requirements (#10106) 2024-03-27 22:32:53 +01:00
Harshal Sheth
1598070899
fix(ci): simplify python release process (#10133) 2024-03-25 21:46:57 -04:00
Tamas Nemeth
7e5610f358
feat(ingest/dagster): Dagster source (#10071)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
2024-03-25 13:28:35 +01:00
Harshal Sheth
630f2a13f6
feat(ingest): check for private import path usages (#10059) 2024-03-18 14:59:18 -07:00
Harshal Sheth
0c7ca4cd3e
docs(ingest/lookml): update known discrepancy list (#9941)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-02-29 12:39:32 -08:00
Harshal Sheth
02f41b74b6
feat(ingest): use uv for python package installs (#9885) 2024-02-26 15:02:47 -08:00
Harshal Sheth
0d780e5f8f
feat(ingest): sql parsing aggregator (#9786) 2024-02-09 16:27:45 -05:00
Peng G
ea0ae8cb54
feat(airflow) Override datajob external_url (#9681)
Co-authored-by: Peng Gao <pgao@slack-corp.com>
2024-02-07 09:28:30 +01:00