yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-12-01 21:27:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-jobs
History
Abe 6ec6f0150d
refactor(metrics): Make MetricUtils.registry non-nullable 
- Make MetricUtils.registry non-nullable with default no-op implementation. This allows us to remove boilerplate for handling the null case
- Rename request context metric names to follow convention
2025-09-09 11:08:16 -07:00
..
common
refactor(metrics): Make MetricUtils.registry non-nullable
2025-09-09 11:08:16 -07:00
mae-consumer
refactor(metrics): Make MetricUtils.registry non-nullable
2025-09-09 11:08:16 -07:00
mae-consumer-job
feat(monitoring): modernize datahub monitoring (#13898)
2025-07-08 12:40:24 -05:00
mce-consumer
refactor(metrics): Make MetricUtils.registry non-nullable
2025-09-09 11:08:16 -07:00
mce-consumer-job
feat(kafka): refactor kafka topic mgmt and move setup to system-update (#14564)
2025-08-27 17:03:56 +05:30
pe-consumer
refactor(metrics): Make MetricUtils.registry non-nullable
2025-09-09 11:08:16 -07:00
README.md
ci(graphql,workflows): Format .md, .graphql, and workflow .yml files via prettier (#13220)
2025-04-16 16:55:51 -07:00

README.md

MXE Processing Jobs

DataHub uses Kafka as the pub-sub message queue in the backend. There are 2 Kafka topics used by DataHub which are MetadataChangeEvent and MetadataAuditEvent.

  • MetadataChangeEvent: This message is emitted by any data platform or crawler in which there is a change in the metadata.
  • MetadataAuditEvent: This message is emitted by DataHub GMS to notify that metadata change is registered.

To be able to consume from these two topics, there are two Spring jobs DataHub uses: