452 Commits

Author SHA1 Message Date
noggi
f8db90926e
Upload metadata model to s3 (#9325) 2023-11-29 09:26:14 -08:00
david-leifker
38adff8693
fix(gha): fix gha for single tag (#9283) 2023-11-21 15:14:52 -06:00
noggi
b7df9e0996
Sync datahub-head on merge (#9267) 2023-11-17 16:53:19 -08:00
RyanHolstien
582eebe739
feat(policies): reduce default access for all users (#9067)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-11-13 10:57:49 -06:00
Harshal Sheth
0e3efabd2c
fix(build): set @cliMajorVersion@ correctly (#9228) 2023-11-10 16:00:20 -08:00
Harshal Sheth
4a4c29030c
chore: stop ingestion-smoke CI errors on forks (#9160) 2023-11-06 12:47:24 -08:00
Aseem Bansal
f5c8192cca
ci: tweak to only run relevant workflows (#9052) 2023-10-30 18:32:51 +05:30
david-leifker
3f4ab44a91
feat(kafka): increase kafka message size and enable compression (#9038)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-10-29 16:26:05 -05:00
Shirshanka Das
c9309ff157
feat(ci): enable ci to run on PR-s targeting all branches (#8933) 2023-10-05 09:07:12 -07:00
Aseem Bansal
2bc685d3b9
ci: tweak ci to decrease wait time of devs (#8945) 2023-10-05 09:31:32 +05:30
Harshal Sheth
a300b39f15
feat(ingest/airflow): airflow plugin v2 (#8853) 2023-10-04 16:23:15 +05:30
Harshal Sheth
9deb7be3fc
fix(ingest): refactor test markers + fix disk space issues in CI (#8938) 2023-10-03 20:17:49 -07:00
david-leifker
4d9a7ce7c9
ci: Docker slim tag fix (#8925) 2023-09-29 23:07:45 -05:00
david-leifker
b45b7f7d2a
fix(docker): slim tags (#8922) 2023-09-29 17:49:36 -05:00
david-leifker
d33a85314d
fix(docker): move base image to -base tag, full image to head (#8919)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-09-28 21:21:04 -05:00
Harshal Sheth
ed1d35c79b
fix(build): run codegen when building datahub-ingestion image (#8869) 2023-09-25 23:10:49 -05:00
Aseem Bansal
ec714fc1e5
feat(ci): disable ingestion smoke build (#8845) 2023-09-15 20:04:11 +05:30
Harshal Sheth
5a0ce38bc6
docs: add 0.11 docs to docs site (#8813)
Co-authored-by: Indy Prentice <iprentic@users.noreply.github.com>
2023-09-11 13:29:31 -07:00
david-leifker
486bd861d2
test(managed_ingestion): fix managed ingestion test by fixing actions… (#8820) 2023-09-11 14:20:21 -05:00
Harshal Sheth
6c55eb8e6d
ci: upgrade python to 3.10 for builds (#8808)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-09-08 16:30:08 -07:00
david-leifker
68ae3bfc26
fix(spark-test): upgrade gradle and fix spark smoke test (#8777) 2023-09-07 20:17:50 -05:00
david-leifker
7dd6e09ac5
refactor(build): upgrade to gradle 7 & guava update (#8745) 2023-09-01 19:36:01 +05:30
Mayuri Nehate
e867dbc3da
ci: separate airflow build and test (#8688)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-08-30 14:08:42 -07:00
Tamas Nemeth
d86b336e70
chore(ingest/s3) Bump Deequ and Pyspark version (#8638)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-08-29 18:11:37 +02:00
Harshal Sheth
d6e36f16de
ci: add needs_artifact_download output for ingestion image (#8695) 2023-08-22 21:27:02 -07:00
david-leifker
655914841b
fix(docker): misc docker fixes (#8677) 2023-08-22 10:38:08 +05:30
david-leifker
40caf35812
feat(datahub-ingestion): refactor datahub ingestion slim images (#8515) 2023-08-17 10:40:17 +05:30
Harshal Sheth
e157992b77
feat(ci): use docker registry cache (#8544) 2023-08-02 17:17:13 -05:00
Harshal Sheth
a9bc5643f3
ci: reduce git fetch depth (#8473) 2023-08-01 19:34:11 +05:30
david-leifker
3640291493
config(ebean): add ebean retry configuration (#8500) 2023-07-28 14:09:44 -05:00
RyanHolstien
93fde6bf93
feat(io): refactor metadata-io module (#8306) 2023-07-20 06:39:14 +05:30
Pedro Silva
dd9613027b
feat(images): Add build processes for arm64v8 image variants (#7990)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-05-22 17:23:47 +01:00
Harshal Sheth
4ecb459f1e
ci: prevent qodana from using all of our cache (#8054) 2023-05-19 14:16:26 +05:30
david-leifker
429c54f3ad
fix(ci): fix spark lineage test (#7829) 2023-04-17 15:29:45 -07:00
RyanHolstien
97ac8d93f8
fix(ci): prevent republish of images on release edits (#7828) 2023-04-15 13:31:58 -07:00
RyanHolstien
cbf0809c4c
chore(snakeyaml): upgrade to snakeyaml 2 (#7786)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-04-13 12:01:51 -05:00
Harshal Sheth
dac540e691
ci: limit qodana concurrency to group (#7764)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-04-06 10:48:58 -07:00
Aseem Bansal
1324231e70
chore(ci): add coverage code for python (#7681) 2023-03-24 07:10:45 +00:00
Harshal Sheth
3b519924e8
fix(cli): protect against timeseries get_aspects (#7665) 2023-03-22 22:04:11 -07:00
Harshal Sheth
d9231addd2
feat(docs-website): add vercel preview environment (#7644) 2023-03-22 12:28:52 -07:00
Harshal Sheth
2722ad54d4
feat(cli): build and upload Python wheels in CI (#7537) 2023-03-22 21:31:32 +05:30
david-leifker
697e8e2647
fix(misc): misc fixes (#7633) 2023-03-21 19:42:50 +05:30
Shirshanka Das
3e39b955f6
fix(docker): fix image name for datahub-ingestion-slim (#7653) 2023-03-20 22:13:26 -07:00
david-leifker
0a9dc73402
build(docker): metadata-ingestion images build and add slim version (#7412) 2023-03-20 18:06:35 -05:00
Aditya Radhakrishnan
0ff7bea043
fix(smoke-test): fix native user and access token tests (#7628) 2023-03-20 16:04:42 -05:00
Aseem Bansal
c43d7dd9d2
chore(ci): fix flakiness, misc improvements (#7605) 2023-03-17 09:34:00 +05:30
Aseem Bansal
2ef26e8b74
fix(policies): add missing policies, add check to prevent problems (#7586)
Co-authored-by: John Joyce <john@acryl.io>
2023-03-16 09:49:55 +05:30
Aseem Bansal
b2ecd7f968
chore(ci): update dependencies, fix smoke image build (#7580) 2023-03-15 21:07:30 +05:30
Aseem Bansal
39f7f74329
chore(ci): add daylight savings timezone for tests, fix daylight saving bug in analytics charts (#7484) 2023-03-14 16:12:29 +05:30
Aseem Bansal
64b4d0bace
chore(ci): try Qodana Scan for quality (#7560) 2023-03-14 14:26:06 +05:30