jaegwon.seo
fc7b6853b9
change generate kafka connect properties from env ( #10545 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-07-31 15:05:55 -05:00
david-leifker
3f4ab44a91
feat(kafka): increase kafka message size and enable compression ( #9038 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-10-29 16:26:05 -05:00
Mert Tunç
ce6f833be4
fix(kafka-setup): Don't set truststore pass for PEM files ( #8656 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-10-26 17:06:37 -05:00
david-leifker
655914841b
fix(docker): misc docker fixes ( #8677 )
2023-08-22 10:38:08 +05:30
david-leifker
cd05f5b174
feat(schema-registry): replace confluent schema registry ( #7930 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2023-05-01 13:18:41 -05:00
Jinlin Yang
a3a8107b92
bug(7882): run kafka-configs.sh on DataHubUpgradeHistory_v1 to make sure the retention.ms is set to infinite ( #7883 )
2023-04-21 18:31:51 -05:00
Kirill Popov
10d389f895
fix(kafka-setup): configure sasl.mechanism in case SASL_PLAINTEXT ( #7447 )
2023-02-28 16:07:27 -08:00
david-leifker
b4b3a39829
fix(system-update): fixes system-update with more than 1 partition ( #7302 )
2023-02-09 18:54:20 -06:00
david-leifker
d557fd65ff
fix(elasticsearch): fix orphan index clean up pattern, consistent top… ( #7242 )
2023-02-03 07:09:19 -06:00
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements ( #6894 )
2023-01-31 18:44:37 -06:00
Peter Szalai
8a2230aea4
fix(quickstart): adding back kafka-setup ( #7181 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-01-31 00:34:36 +01:00
Pedro Silva
5db3a1f1f8
fix(kafka-setup): Make topic name consistent with other images ( #7103 )
2023-01-24 10:48:23 -08:00
Pedro Silva
ae2d781359
fix(kafka-setup): Remove reference to non-existing topic ( #7019 )
2023-01-12 07:25:17 -08:00
Pedro Silva
2fe9e8d962
fix(kafka-setup): Make kafka-setup run with multiple threads ( #6970 )
2023-01-09 12:27:00 +00:00
david-leifker
a5b5abf270
fix(kafka): datahub-upgrade job ( #6864 )
2022-12-27 12:35:48 -06:00
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) ( #6744 )
...
* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
david-leifker
637b540fa6
fix(kafka-setup): fix return code check ( #6782 )
...
* fix(kafka-setup): parallelize topic creation
* Remove -setup from docker compose (not services)
2022-12-16 13:39:18 -06:00
david-leifker
cca3639848
Optimize kafka setup ( #6778 )
...
* fix(kafka-setup): parallelize topic creation
2022-12-15 20:09:29 -06:00
Parham Ghazanfari
269342bec0
feat(kafka-setup): Added support for MSK IAM authentication. ( #6435 )
2022-11-22 15:37:24 -06:00
Pedro Silva
164bc1aa53
Revert "chore(setup): change defaults for partitions ( #6074 )" ( #6086 )
2022-09-29 15:44:32 +05:30
Aseem Bansal
4cf635e16e
chore(setup): change defaults for partitions ( #6074 )
2022-09-28 15:04:35 +05:30
Pedro Silva
12eb2c6240
feat(kafka) Adds SASL SSL support in kafka setup docker image ( #5697 )
2022-08-22 16:01:10 -07:00
Shirshanka Das
9d79fd3767
fix(docker): fix kafka-setup command to support same capabilities as previous ( #5428 )
2022-07-18 13:25:17 -07:00
Shirshanka Das
024797df0d
feat(quickstart): moving to official confluent images for m1 ( #5416 )
2022-07-17 18:59:09 -07:00
abiwill
7bbac5ef4d
feat(kafka-setup): add options for sasl_plaintext ( #5385 )
...
allow sasl_plaintext options using environment variables
2022-07-13 13:47:23 -07:00
Dexter Lee
93145d1591
fix(Kafka-setup): Make sure it doesn't fail when the new envs are not set ( #5168 )
2022-06-14 10:29:24 -07:00
chen4119
3c32ac7916
fix(kafka-setup): Check if keystore/truststore location env variables are set ( #4924 )
...
* OIDC discovery URL will not have none as auth method
* Fix code style
* fix compilation error
* add keystore/truststore type env variables
* check keystore/truststore location is set
Co-authored-by: Dexter Lee <dexter@acryl.io>
2022-05-16 13:51:19 -07:00
chen4119
a0e7262b00
fix(kafka-setup): Add ssl.keystore.type and ssl.truststore.type ( #4923 )
2022-05-16 10:22:55 -07:00
John Joyce
c69310522b
feat(metadata service): Introducing Platform Events ( #4477 )
2022-03-29 18:32:04 -07:00
Tamas Nemeth
111d6d3520
fix(docker): alpine based multi-platform docker build for kafka-setup ( #3991 )
2022-01-28 16:25:57 -08:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect ( #2983 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
John Joyce
29832e5385
feat(Product Analytics): Introducing In-App Analytics Beta ( #2499 )
...
Co-authored-by: Harshal Sheth <harshal@acryl.io>
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2021-05-11 15:41:42 -07:00
Dexter Lee
c1c587d164
fix(kafka-setup): Fix start script for kafka setup ( #2421 )
2021-04-19 15:39:38 -07:00
Dexter Lee
1b3a1459e3
fix(kafka-setup): Add the correct context to the git workflow for pushing kafka-setup image ( #2414 )
2021-04-16 17:51:43 -07:00
G-nther
177161a850
feat(kafka-setup): add option for SSL and topic partition config via environment ( #2398 )
2021-04-16 09:40:39 -07:00
