| entityUrn | String | The unique identifier for the Entity being changed. For example, a Dataset's urn. | False |
| entityType | String | The type of the entity the new aspect is associated with. This corresponds to the entity name in the DataHub Entity Registry, for example 'dataset'. | False |
| entityKeyAspect | Object | The key struct of the entity that was changed. Only present if the Metadata Change Proposal contained the raw key struct. | True |
| changeType | String | The change type. CREATE, UPSERT and DELETE are currently supported. | False |
| aspectName | String | The entity aspect which was changed. | False |
| aspect | Object | The new aspect value. Null if the aspect was deleted. | True |
| aspect.contentType | String | The serialization type of the aspect itself. The only supported value is `application/json`. | False |
| aspect.value | String | The serialized aspect. This is a JSON-serialized representing the aspect document originally defined in PDL. See https://github.com/datahub-project/datahub/tree/master/metadata-models/src/main/pegasus/com/linkedin for more. | False |
| systemMetadata | Object | The new system metadata. This includes the the ingestion run-id, model registry and more. For the full structure, see https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/SystemMetadata.pdl | True |
The PDL schema can be found [here](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/MetadataChangeProposal.pdl).
Note how the aspect payload is serialized as JSON inside the "value" field. The exact structure
of the aspect is determined by its PDL schema. (For example, the [ownership](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/common/Ownership.pdl) schema)
Two distinct topics are maintained for Metadata Change Log. The default Kafka topic name for **versioned** aspects is `MetadataChangeLog_Versioned_v1` and for
**timeseries** aspects is `MetadataChangeLog_Timeseries_v1`.
### Consumption
DataHub ships with a Kafka Consumer Job (mae-consumer-job) which listens for MCLs and uses them to update DataHub's search and graph indices,
In addition, the [Actions Framework](../actions/README.md) consumes Metadata Change Logs to power its [Metadata Change Log](../actions/events/metadata-change-log-event.md) event API.
| entityUrn | String | The unique identifier for the Entity being changed. For example, a Dataset's urn. | False |
| entityType | String | The type of the entity the new aspect is associated with. This corresponds to the entity name in the DataHub Entity Registry, for example 'dataset'. | False |
| entityKeyAspect | Object | The key struct of the entity that was changed. Only present if the Metadata Change Proposal contained the raw key struct. | True |
| changeType | String | The change type. CREATE, UPSERT and DELETE are currently supported. | False |
| aspectName | String | The entity aspect which was changed. | False |
| aspect | Object | The new aspect value. Null if the aspect was deleted. | True |
| aspect.contentType | String | The serialization type of the aspect itself. The only supported value is `application/json`. | False |
| aspect.value | String | The serialized aspect. This is a JSON-serialized representing the aspect document originally defined in PDL. See https://github.com/datahub-project/datahub/tree/master/metadata-models/src/main/pegasus/com/linkedin for more. | False |
| previousAspectValue | Object | The previous aspect value. Null if the aspect did not exist previously. | True |
| previousAspectValue.value | String | The serialized aspect. This is a JSON-serialized representing the aspect document originally defined in PDL. See https://github.com/datahub-project/datahub/tree/master/metadata-models/src/main/pegasus/com/linkedin for more. | False |
| systemMetadata | Object | The new system metadata. This includes the the ingestion run-id, model registry and more. For the full structure, see https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/SystemMetadata.pdl | True |
| previousSystemMetadata | Object | The previous system metadata. This includes the the ingestion run-id, model registry and more. For the full structure, see https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/SystemMetadata.pdl | True |
| created | Object | Audit stamp about who triggered the Metadata Change and when. | False |
| created.time | Number | The timestamp in milliseconds when the aspect change occurred. | False |
The PDL schema for can be found [here](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/MetadataChangeLog.pdl).
Note how the aspect payload is serialized as JSON inside the "value" field. The exact structure
of the aspect is determined by its PDL schema. (For example, the [ownership](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/common/Ownership.pdl) schema)
## Platform Event (PE)
A Platform Event represents an arbitrary business-logic event emitted by DataHub. Each
Platform Event has a `name` which determines its contents.
### Types
- **Entity Change Event** (entityChangeEvent): The most important Platform Event is named **Entity Change Event**, and represents a log of semantic changes
The [Actions Framework](../actions/README.md) consumes Platform Events to power its [Entity Change Event](../actions/events/entity-change-event.md) API.
| header.timestampMillis | Long | The time at which the event was generated. | False |
| name | String | The name / type of the event. | False |
| payload | Object | The event itself. | False |
| payload.contentType | String | The serialization type of the event payload. The only supported value is `application/json`. | False |
| payload.value | String | The serialized payload. This is a JSON-serialized representing the payload document originally defined in PDL. See https://github.com/datahub-project/datahub/tree/master/metadata-models/src/main/pegasus/com/linkedin for more. | False |
The full PDL schema can be found [here](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/PlatformEvent.pdl).
### Examples
An example of an 'Entity Change Event' Platform Event that is emitted when a new owner is added to a Dataset:
Note how the actual payload for the event is serialized as JSON inside the 'payload' field. The exact
structure of the Platform Event is determined by its PDL schema. (For example, the [Entity Change Event](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/platform/event/v1/EntityChangeEvent.pdl) schema)
## Failed Metadata Change Proposal (FMCP)
When a Metadata Change Proposal cannot be processed successfully, the event is written to a [dead letter queue](https://en.wikipedia.org/wiki/Dead_letter_queue)
in an event called Failed Metadata Change Proposal (FMCP).
The default Kafka topic name for FMCPs is `FailedMetadataChangeProposal_v1`.
### Consumption
No active consumers.
### Schema
The PDL schema can be found [here](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/FailedMetadataChangeProposal.pdl).
# Deprecated Events
DataHub ships with a set of deprecated events, which were historically used for proposing and logging
Consequently, only successfully accepted and processed MCEs will lead to the emission of a corresponding MAE / MCLs.
### Emission
MCEs may be emitted by clients of DataHub's low-level ingestion APIs (e.g. ingestion sources)
during the process of metadata ingestion.
The default Kafka topic name for MCEs is `MetadataChangeEvent_v4`.
### Consumption
DataHub's storage layer actively listens for new Metadata Change Events, attempts
to apply the requested changes to the Metadata Graph.
### Schema
The PDL schema can be found [here](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/MetadataChangeEvent.pdl).
### Examples
An example of an MCE emitted to change the 'ownership' aspect for an Entity:
A Metadata Audit Event captures changes made to one or multiple metadata [aspects](aspect.md) associated with a particular [entity](entity.md), in the form of a metadata [snapshot](snapshot.md) (deprecated) before the change, and a metadata snapshot after the change.
Every source-of-truth for a particular metadata aspect is expected to emit a MAE whenever a change is committed to that aspect. By ensuring that, any listener of MAE will be able to construct a complete view of the latest state for all aspects.
Furthermore, because each MAE contains the "after image", any mistake made in emitting the MAE can be easily mitigated by emitting a follow-up MAE with the correction. By the same token, the initial bootstrap problem for any newly added entity can also be solved by emitting a MAE containing all the latest metadata aspects associated with that entity.
The PDL schema can be found [here](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/MetadataAuditEvent.pdl).
When a Metadata Change Event cannot be processed successfully, the event is written to a [dead letter queue](https://en.wikipedia.org/wiki/Dead_letter_queue) in an event called Failed Metadata Change Event (FMCE).
The event simply wraps the original Metadata Change Event and an error message, which contains the reason for rejection.
This event can be used for debugging any potential ingestion issues, as well as for re-playing any previous rejected proposal if necessary.
### Emission
FMCEs are emitted when MCEs cannot be successfully committed to DataHub's storage layer.
### Consumption
No active consumers.
### Schema
The PDL schema can be found [here](https://github.com/datahub-project/datahub/blob/master/metadata-models/src/main/pegasus/com/linkedin/mxe/FailedMetadataChangeEvent.pdl).
