The environment was not set correctly, so it could not fire kafka events. It (mce-cli) always worked when running outside of docker.
I also added a dev ingestion docker image / script which may be much faster if you've already built locally.
Tested:
1. Cleaned docker volumes and started datahub. Verified it is empty.
2. Built with gradle.
3. Ran ./docker/ingestion/ingestion-dev.sh. Verified data shows in DataHub.
4. Ran step 1 again.
5. Ran ./docker/ingestion/ingestion.sh. Verified data shows in DataHub.
New env vars:
- METADATA_CHANGE_EVENT_NAME: The name of the metadata change event topic.
- METADATA_AUDIT_EVENT_NAME: The name of the metadata audit event topic.
- FAILED_METADATA_CHANGE_EVENT_NAME: The name of the failed metadata change event topic.
This will need to be consistent throughout your ecosystem.
CLOSES: #1840
Start adding java ETL examples, starting with kafka etl.
We've had a few requests to start providing Java examples rather than Python due to type safety.
I've also started to add these to metadata-ingestion-examples to make it clearer these are *examples*. They can be used directly or as a basis for other things.
As we port to Java we'll move examples to contrib.
