* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
* Add Datasource as variable in dashboard
(cherry picked from commit e75b3f7333dd9436f7434eefa76ab9fbfb1babab)
* Update datahub_dashboard.json
(cherry picked from commit 701592697702a99ddf33f75d57826282df999c82)
* Bump docker compose version to 3.8
(cherry picked from commit ff6a97b1692ac21368eacc83a0daed30b0364d5d)
* Update grafana image tag from latest to 9.1.4
(cherry picked from commit 2c88e2a3041b9ddfa70bb92d13e80ca15be0ead2)
* Update old metric name in datahub_dashboard.json
(cherry picked from commit 21b502e25392f37c5053dcf00161fc5c7d35c94d)
* Add panel for new metrics
(cherry picked from commit 194452778bc825820dda87ab2e159b7064a360ef)
Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
* Add AWS Postgres Iam Auth jar to GMS
* Added IAM support for Postgres
* Setting IAM support for Postgres as false by default
* format fix
Co-authored-by: syed.javed <syed.javed@precisely.com>
* refactor(elasticsearch-setup-job): create-indices.sh readability
The script contains many copy-pasting and is not easy to follow.
Add comments, extract commonly used operations into functions, unify approaches.
* fix(elasticsearch-setup-job): AWS indices creation
Fix the issue where Amazon OpenSearch (AWS ES) indices are incorrectly initialised
and the Analytics screen shows errors only.
* feat(elasticsearch-setup-job): configuration hint
mention USE_AWS_ELASTICSEARCH env value if it seems it's set the wrong way
* fix(elasticsearch-setup-job): silent curl
* fix(elasticsearch-setup-job): better USE_AWS_ELASTICSEARCH hint
* docs(elasticsearch-setup-job): index dropping explained
- more comments
- more defensive approach
- index file renamed
* fix(elasticsearch-setup-job): script fixes
* merge(elasticsearch-setup-job): merging in PR #5937
* merge(elasticsearch-setup-job): merging in PR #5963
* merge(elasticsearch-setup-job): merging in PR #5975
Co-authored-by: Pedro Silva <pedro@acryl.io>
Pull request #5827 introduced a regression by removing coreutils
from the mae-consumer Dockerfile (coreutils was added in #3723).
This broke the base64 call in the startup script s.th. the Elasticsearch
auth header will not be correctly set when username and password are
provided.
To make sure that the startup script fails on these errors in the
future, set "-euo pipefail" which lets the bash script fail on errors
and unset variables.
Also refactor the startup script to make it more stable and readable.
Co-authored-by: Pedro Silva <pedro@acryl.io>
