mirror of https://github.com/datahub-project/datahub.git synced 2025-10-31 10:49:00 +00:00

History

Hendrik Richert 966cb175f7

feat(dev): Make repositories configurable for enterprise developers (#9230 )

Co-authored-by: Hendrik Richert <hendrik.richert@swisscom.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>

2023-11-28 14:52:11 -06:00

env

feat(schema-registry): replace confluent schema registry (#7930 )

2023-05-01 13:18:41 -05:00

build.gradle

feat(dev): Make repositories configurable for enterprise developers (#9230 )

2023-11-28 14:52:11 -06:00

Dockerfile

feat(dev): Make repositories configurable for enterprise developers (#9230 )

2023-11-28 14:52:11 -06:00

kafka-config.sh

feat(kafka): increase kafka message size and enable compression (#9038 )

2023-10-29 16:26:05 -05:00

kafka-ready.sh

fix(docker): misc docker fixes (#8677 )

2023-08-22 10:38:08 +05:30

kafka-setup.sh

feat(kafka): increase kafka message size and enable compression (#9038 )

2023-10-29 16:26:05 -05:00

kafka-topic-workers.sh

feat(kafka): increase kafka message size and enable compression (#9038 )

2023-10-29 16:26:05 -05:00

README.md

…

README.md

Kafka, Zookeeper and Schema Registry

DataHub uses Kafka as the pub-sub message queue in the backend. Official Confluent Kafka Docker images found in Docker Hub is used without any modification.

Debugging Kafka

You can install kafkacat to consume and produce messaged to Kafka topics. For example, to consume messages on MetadataAuditEvent topic, you can run below command.

kafkacat -b localhost:9092 -t MetadataAuditEvent

However, kafkacat currently doesn't support Avro deserialization at this point, but they have an ongoing work for that.