david-leifker
ecc01b9a46
refactor(restli-mce-consumer) ( #6744 )
...
* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
Aseem Bansal
6fdbf6b42d
Revert "fix(ci): remove warnings due to deprecated action ( #6735 )" ( #6857 )
...
This reverts commit 1da27edcbc97d64d5e88321c8727fabf92a576f8.
2022-12-26 17:56:46 +05:30
Shirshanka Das
b236d0958c
feat(ci): adding github security reporting to trivy scans ( #6773 )
2022-12-15 21:28:07 -08:00
Aseem Bansal
1da27edcbc
fix(ci): remove warnings due to deprecated action ( #6735 )
2022-12-13 10:42:21 +05:30
david-leifker
68e30d3bc2
test(quickstart): add debugging to quickstart test ( #6718 )
2022-12-09 09:26:22 -06:00
Harshal Sheth
1165884c6b
ci: stop commenting unit test results on PR ( #6700 )
...
The results will still be surfaced under the "Test Results" action
workflow, but the results won't be commented on the PR itself.
2022-12-08 23:19:27 +01:00
Aseem Bansal
5b52534fd4
fix(ci): reduce warnings due to deprecated action ( #6686 )
2022-12-08 10:00:05 +05:30
mohdsiddique
b46d52dc0f
feat(gms): Enable auth-api publishing to maven ( #6671 )
2022-12-06 21:56:36 -08:00
Harshal Sheth
fceef480a2
chore(ingest): remove feast-legacy ( #6661 )
2022-12-06 14:19:38 -08:00
Aseem Bansal
0862a22632
fix(build): remove warnings in github actions ( #6512 )
2022-11-23 16:43:23 +05:30
Harshal Sheth
3e907ab0d1
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ ( #6204 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-11 15:04:36 -05:00
Harshal Sheth
b4687ffceb
feat(ingest): drop plugin support for airflow 1.x ( #6331 )
2022-11-01 21:12:34 -07:00
Kevin Neville
3eae087a30
chore(ci): update actions versions addressing warnings in builds. ( #6308 )
2022-10-28 17:10:30 -07:00
Aseem Bansal
98eba90edc
fix(build): allow image tag via env, fix requirements ( #6237 )
2022-10-19 12:26:22 +05:30
Shirshanka Das
48b46971b8
feat(ingest): remove hardcoded env variable default for cli version ( #6075 )
2022-10-02 16:46:34 -07:00
leifker
203a6ff57b
refactor(java11) - convert most modules to java 11 ( #5836 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-25 08:39:22 -07:00
Tamas Nemeth
46494bc0ad
feat(ingest): presto-on-hive - Description and hive view support ( #5993 )
...
* Presto on Hive Improvements like:
- Capturing table/column descriptions
- Adding Hive view support
- Various bug fixes
2022-09-21 00:40:10 +02:00
Shirshanka Das
e24e5b61b9
fix(datahub-ranger-plugin): add support to publish jars ( #5983 )
2022-09-19 17:46:01 -07:00
Harshal Sheth
2f7fe593b7
ci: reduce pip backtracking in airflow plugin ( #5982 )
2022-09-19 12:40:51 -07:00
Aseem Bansal
eca73c0524
feat(ci): exempt from stale label ( #5964 )
2022-09-16 11:34:41 -07:00
Aseem Bansal
ff04e0e0bd
fix(ci): process older issues first ( #5926 )
2022-09-14 14:46:41 +05:30
Shirshanka Das
15a33fadc8
feat(ingest): lookml - support for git checkout ( #5924 )
2022-09-13 16:40:59 -07:00
Shirshanka Das
5a0154f3fe
fix(ci): fix jar release action dependencies ( #5884 )
2022-09-09 10:16:09 -07:00
Shirshanka Das
083ab9bc0e
feat(jars): better jar versioning for datahub-client, spark-lineage and protobuf ( #5883 )
2022-09-08 23:36:15 -07:00
Shirshanka Das
fe30f54689
feat(datahub-protobuf): add support for shadow jar, publish ( #5882 )
2022-09-08 22:10:36 -07:00
Pedro Silva
102754f138
feat(smoke-tests) Make smoke tests use standalone consumers ( #5856 )
2022-09-08 16:34:11 +05:30
Harshal Sheth
6f09f1025e
ci(ingest): test with python 3.10 ( #5863 )
2022-09-07 17:08:49 -07:00
Aseem Bansal
055dd27015
feat(ci): auto label PRs ( #5839 )
2022-09-06 14:46:34 -07:00
Pedro Silva
20138a32e5
fix(standalone-consumers): Exclude Solr from spring boot application config & make them run on M1 ( #5827 )
...
* fix(standalone-consumers) Removes Solr from spring boot application config
* Adds standalone consumer option to datahub quickstart cli
* Rename files
* Make dockerize platform agnostic & change docker compose utility to work with M1
* Fix MAE/MCE dockerfiles for arm & make smoke tests use standalone consumers
* Fix identation
* Use master code
* Adds ARM image publishing for consumers
* Fix linter
* Fix lint
2022-09-06 13:55:31 -05:00
Harshal Sheth
355c129c7c
chore(ingest): drop python 3.6 support ( #5521 )
2022-08-10 15:00:31 -07:00
Aseem Bansal
581765dd91
fix(ci): cancel docker-unified workflow only on PRs on new commits ( #5608 )
2022-08-10 16:22:56 +05:30
Aseem Bansal
fedf451668
fix(ci): use different image for smoke base image ( #5607 )
2022-08-10 14:53:30 +05:30
Aseem Bansal
006141ab2e
fix(ci): paths for github workflows ( #5595 )
2022-08-09 10:10:08 -07:00
Aseem Bansal
d20071ad91
fix(ci): smoke test less flaky, add src, dev dep in smoke image ( #5594 )
2022-08-09 19:07:12 +05:30
Aseem Bansal
8ea5295294
chore(build): tweak stale issue timing ( #5571 )
2022-08-04 10:40:05 -07:00
Aseem Bansal
120875f3fd
feat(ci): test quickstart works ( #5518 )
...
* feat(ci): test quickstart works
* do not fail fast
* remove macos
* add some debug information
* tweak triggers
* fix workflow file
* remove running on every PR
* Update .github/workflows/check-quickstart.yml
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
* Update .github/workflows/check-quickstart.yml
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-08-02 17:38:54 -07:00
Aditya Radhakrishnan
d0f0fae431
chore(issues): reduce time for issues to be marked stale and then closed ( #5537 )
2022-08-01 15:53:18 -07:00
mohdsiddique
c31e380489
feat(apache-ranger): Apache Ranger Authorizer support in datahub-gms ( #4999 )
2022-08-01 09:47:08 -07:00
Harshal Sheth
2d68097ff4
refactor(ci): refactor Docker build-and-push workflows ( #5494 )
2022-07-27 22:27:31 +05:30
Harshal Sheth
44122d4e08
refactor(ci): use custom action for checking codegen status ( #5493 )
2022-07-26 18:04:57 -07:00
Aseem Bansal
639feafcfb
feat(build): automatically mark issues as stale to close inactive issues ( #5482 )
2022-07-26 09:30:40 -07:00
Aseem Bansal
252992f15e
fix(ci): intermittent failure in github actions ( #5452 )
2022-07-25 23:02:02 -07:00
Shirshanka Das
86012fd40c
feat(ci): datahub-upgrade - support multiplatform image ( #5477 )
2022-07-24 18:25:39 -07:00
Aseem Bansal
935b423452
fix(docs,build): remove base image not needed, cleanup docs ( #5469 )
2022-07-22 19:43:48 +05:30
Aseem Bansal
e21efbe9d2
fix(build): add base image with gradle wrapper cached ( #5467 )
2022-07-22 17:44:37 +05:30
Aseem Bansal
9fca5277dc
fix(build): tweak ingestion build ( #5374 )
2022-07-12 12:37:47 +05:30
Aseem Bansal
b73477f31e
feat(build): reduce build time for ingestion image ( #5225 )
2022-06-29 15:26:12 +05:30
Aseem Bansal
e2d849de4b
fix(build): use correct creds ( #5261 )
2022-06-27 20:31:51 +05:30
Aseem Bansal
60ca9580d1
feat(build): add base image for ingest ( #5243 )
2022-06-27 14:08:16 +05:30
Aseem Bansal
c942350e3c
feat(ci): split integration tests to reduce run time ( #5205 )
2022-06-20 21:08:37 +05:30
