9f119d3898
* feat: Enable Failed Metadata Change Event for MCE Processor No Failed MCE was sent when a MCE was rejected by the system - Added FailedMetadataChangeEvent kafka topic - Created FailedMCE event from the MCE record - Added Kafka Producer to send FailedMCE to the FailedMetadataChangeEvent Topic * fix: Review Comment - Added Nonnull Annotations - Made Local variables as final * bug: added pegasus2avro test json - Added pegasus2avro test fmce json * feat: Add Tests for Event Utils Avro2pegasusfmce - Added Tests for avro2Pegasusfmce
Docker Images
The easiest way to bring up and test DataHub is using DataHub Docker images which are continuously deployed to Docker Hub with every commit to repository.
Above Docker images are created for DataHub specific use. You can check subdirectories to check how those images are generated via Dockerbuild files or how to start each container using Docker Compose. Other than these, DataHub depends on below Docker images to be able to run:
Local-built ingestion image allows you to create on an ad-hoc basis metadatachangeevent with Python script.
The pipeline depends on all the above images composing up.
Prerequisites
You need to install docker and docker-compose.
Quickstart
If you want to quickly try and evaluate DataHub by running all necessary Docker containers, you can check Quickstart Guide.