62 Commits

Author SHA1 Message Date
Aseem Bansal
effd4825f3
dev: reduce log in gradle commands (#12677) 2025-02-20 13:47:48 +05:30
Aseem Bansal
262dd76518
dev: remove black in favor of ruff for formatting (#12378) 2025-01-18 15:06:20 +05:30
Harshal Sheth
f06ad1a1d5
fix(ingest): simplify + fix ruff config (#12382) 2025-01-18 05:58:12 +05:30
Aseem Bansal
765bf80282
dev(ingest): move from isort to ruff (#12364) 2025-01-16 16:48:21 +05:30
Aseem Bansal
2226820ad1
dev(ingest): use ruff instead of flake8 (#12359) 2025-01-16 08:19:07 +05:30
Harshal Sheth
cfe65ccb3c
chore(ingest): speed up lintFix command (#12346) 2025-01-15 10:59:20 -08:00
Harshal Sheth
ccf5fc708f
chore(ingest): refactor common pytest args (#12240) 2025-01-02 08:53:31 -08:00
Harshal Sheth
08605a95a7
feat(python): split out temp wheel builds (#12157) 2024-12-19 08:02:37 -08:00
Chakru
85b42e3ea5
build(coverage): enable code coverage for java and python (#11992)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-12-02 19:27:43 -06:00
Tamas Nemeth
a31c88e622
fix[ingest/build]: Disable preflight script as it is not needed anymore (#11989) 2024-12-02 10:03:06 +01:00
Harshal Sheth
3733a408fd
chore(airflow): add python 3.11 w/ Airflow 2.9 to CI (#10941) 2024-07-19 10:03:12 +05:30
Harshal Sheth
f38b626d3d
fix(build): avoid nested gradle commands (#10198) 2024-04-04 12:46:00 -07:00
Harshal Sheth
0c7ca4cd3e
docs(ingest/lookml): update known discrepancy list (#9941)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-02-29 12:39:32 -08:00
Harshal Sheth
f399a872ad
feat(ingest/patch): add helper for auto-quoting (#9938) 2024-02-28 15:03:09 +01:00
Harshal Sheth
02f41b74b6
feat(ingest): use uv for python package installs (#9885) 2024-02-26 15:02:47 -08:00
Harshal Sheth
24da7a573b
test(ingest): use pytest-random-order for unit tests (#9753) 2024-02-09 18:20:05 -05:00
Harshal Sheth
1498c36875
chore(cli): drop support for python 3.7 (#9731) 2024-01-29 10:50:47 -08:00
Harshal Sheth
031e6fbb6d
fix(ci): ignore binaries from test results upload (#9560) 2024-01-05 12:18:52 +05:30
Harshal Sheth
08a9b9b6de
fix(ingest): bug fixes and docs updates (#9422) 2023-12-08 16:18:34 -05:00
Harshal Sheth
9deb7be3fc
fix(ingest): refactor test markers + fix disk space issues in CI (#8938) 2023-10-03 20:17:49 -07:00
Harshal Sheth
791e2e7bf5
feat(python): support custom models without forking (#8774) 2023-09-22 16:43:58 -07:00
Harshal Sheth
449cc9ba91
ci: make wheel builds more robust (#8815) 2023-09-12 13:15:05 -07:00
Mayuri Nehate
303a2d0863
build(ingest): upgrade to sqlalchemy 1.4, drop 1.3 support (#8810)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-09-12 11:30:24 -07:00
Harshal Sheth
090f8af8e4
ci(ingest): make ingestion caching rules correct (#8685) 2023-08-23 22:06:28 -07:00
Harshal Sheth
2495b50f8c
ci: fix commandLine usage in build.gradle (#8510) 2023-07-26 11:12:22 +05:30
Shirshanka Das
b3c790aab6
feat: Add support for Data Products (#8039)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2023-05-17 07:17:25 +00:00
Aseem Bansal
1324231e70
chore(ci): add coverage code for python (#7681) 2023-03-24 07:10:45 +00:00
Harshal Sheth
2722ad54d4
feat(cli): build and upload Python wheels in CI (#7537) 2023-03-22 21:31:32 +05:30
Aseem Bansal
c43d7dd9d2
chore(ci): fix flakiness, misc improvements (#7605) 2023-03-17 09:34:00 +05:30
Aseem Bansal
b2ecd7f968
chore(ci): update dependencies, fix smoke image build (#7580) 2023-03-15 21:07:30 +05:30
Aseem Bansal
776806d348
fix(gms): convert obj to string, fix wrong setup (#7582) 2023-03-15 03:02:30 +05:30
Andrew Sikowitz
1402e88e3a
build(idea): mark metadata-ingestion sources and tests (#7394) 2023-02-21 09:50:03 -05:00
Shirshanka Das
07e4d0696f
feat(ingest): json-schema - add json schema support for files and kaf… (#7361) 2023-02-19 08:43:13 -08:00
Harshal Sheth
e3af6168d3
fix(ingest): upgrade feast to avoid build issues (#7218) 2023-02-02 15:24:28 +01:00
Harshal Sheth
b7c03731c4
fix(ingest): fix athena and GE lint errors (#6482) 2022-11-17 19:07:05 -05:00
Harshal Sheth
3e907ab0d1
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ (#6204)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-11 15:04:36 -05:00
Harshal Sheth
b4687ffceb
feat(ingest): drop plugin support for airflow 1.x (#6331) 2022-11-01 21:12:34 -07:00
mohdsiddique
2f65e2f226
feat(transformers): Add semantics & transform_aspect support in transformers (#5514)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-06 14:44:14 -07:00
Harshal Sheth
355c129c7c
chore(ingest): drop python 3.6 support (#5521) 2022-08-10 15:00:31 -07:00
Harshal Sheth
6840240355
build(ingest): make gradle build less chatty (#5491) 2022-07-26 16:37:46 -07:00
Harshal Sheth
fdea718e38
build: improve comprehensiveness of gradle clean (#5003) 2022-07-15 15:38:32 +05:30
Aseem Bansal
c942350e3c
feat(ci): split integration tests to reduce run time (#5205) 2022-06-20 21:08:37 +05:30
Aseem Bansal
4541379024
feat(build): changes to decrease build time, cancel runs in case of multiple commits (#5187) 2022-06-17 18:05:10 +05:30
Shirshanka Das
a9ad138172
feat(ingest): docs - overhaul source connector docs to make it code driven (#4798)
Co-authored-by: MugdhaHardikar-GSLab <mugdha.hardikar@gslab.com>
2022-05-02 00:18:15 -07:00
mayurinehate
e2723bc981
feat(assertion): update python example, assertion entity doc (#4507) 2022-03-29 13:25:51 -07:00
Swaroop Jagadish
eaf7b02b2a
docs(model): auto-generated docs and hand-written docs for the metadata model (#4189) 2022-02-18 09:45:45 -08:00
mayurinehate
1d7ec8dba8
feat(ingest): add nifi source (#3681) 2021-12-08 14:56:31 -08:00
Swaroop Jagadish
a16c432a1b
feat(metadata-model): adding metadata model doc generation and upload… (#3667) 2021-12-05 12:22:17 -08:00
Tamas Nemeth
f043b79f4c
feat(build): Preflight script for metadata ingestion on m1 (#3652) 2021-12-01 11:16:32 -08:00
Swaroop Jagadish
46861600fa
feat(build): add incremental builds for python (#3647) 2021-11-30 18:01:56 -08:00