17 Commits

Author SHA1 Message Date
Aseem Bansal
867007f209
chore: remove obsolete attribute from docker (#11573) 2024-10-09 20:52:50 +05:30
david-leifker
edb9a87b84
fix(elasticsearch): refactor idHashAlgo setting (#11193) 2024-08-16 14:41:44 -05:00
Pinaki Bhattacharjee
a5a33f074b
feat(config): support alternate hashing algorithm for doc id (#10423)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2024-08-06 13:50:38 -05:00
david-leifker
77c4629ccf
refactor(docker): move to acryldata repo for all images (#9459) 2024-04-02 09:36:44 -05:00
Harshal Sheth
5793467e29
feat(quickstart): remove container_name from compose files (#9809) 2024-02-09 20:45:30 -05:00
RyanHolstien
131c8f878c
feat(kafka): add health indicator for kafka (#9662) 2024-01-19 13:44:48 -06:00
RyanHolstien
55cb56821c
fix(kafka): fix infinite deserialization logging (#9494) 2023-12-21 10:33:25 -06:00
Guillaume Cernier
ef1ada118d
fix(docker-compose): make startup more robust with deterministic services' dependencies (#7880) 2023-05-08 16:42:15 -05:00
david-leifker
cd05f5b174
feat(schema-registry): replace confluent schema registry (#7930)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2023-05-01 13:18:41 -05:00
Shirshanka Das
516aee0aaa
fix(docker): add env variable to control mysql setup image, sort dock… (#7266) 2023-02-07 00:56:08 -08:00
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements (#6894) 2023-01-31 18:44:37 -06:00
Pedro Silva
bef59b0928
feat(quickstart): Remove kafka-setup as a hard deployment requirement (#7073) 2023-01-24 16:12:57 +00:00
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) (#6744)
* fix(security): commons-text in frontend

* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint

* testing docker build

* Add retry configuration options for entity client

* Kafka debugging

* fix(kafka-setup): parallelize topic creation

* Adjust docker build

* Docker build updates

* WIP

* fix(lint): metadata-ingestion lint

* fix(gradle-docker): fix docker frontend dep

* fix(elastic): fix race condition between gms and mae for index creation

* Revert "fix(elastic): fix race condition between gms and mae for index creation"

This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.

* fix(test): fix datahub frontend test for clean/test cycle

* fix(test): datahub-frontend missing assets in test

* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer

* gitingore update

* fix(docker): remove platform on docker base image, set by buildx

* refactor(kafka-producer): update kafka producer tracking/logging

* updates per PR feedback

* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool

Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
david-leifker
637b540fa6
fix(kafka-setup): fix return code check (#6782)
* fix(kafka-setup): parallelize topic creation
* Remove -setup from docker compose (not services)
2022-12-16 13:39:18 -06:00
david-leifker
a2dc229c62
fix(tests): Misc updates for tests, auth log level, and quickstart (#6491) 2022-11-29 08:44:55 -08:00
Aseem Bansal
98eba90edc
fix(build): allow image tag via env, fix requirements (#6237) 2022-10-19 12:26:22 +05:30
Pedro Silva
20138a32e5
fix(standalone-consumers): Exclude Solr from spring boot application config & make them run on M1 (#5827)
* fix(standalone-consumers) Removes Solr from spring boot application config

* Adds standalone consumer option to datahub quickstart cli

* Rename files

* Make dockerize platform agnostic & change docker compose utility to work with M1

* Fix MAE/MCE dockerfiles for arm & make smoke tests use standalone consumers

* Fix identation

* Use master code

* Adds ARM image publishing for consumers

* Fix linter

* Fix lint
2022-09-06 13:55:31 -05:00