yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-12-25 00:48:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(ingestion): add architecture diagrams (#11628)

Browse Source
This commit is contained in:
david-leifker 2024-10-15 11:44:55 -05:00 committed by GitHub
parent 8653b56bf6
commit fdae71d25f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 12 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
docs/advanced/mcp-mcl.md
View File

@ -14,6 +14,18 @@ To mitigate these downsides, we are committed to providing cross-language client
Ultimately, we intend to realize a state in which the Entities and Aspect schemas can be altered without requiring generated code and without maintaining a single mega-model schema (looking at you, Snapshot.pdl). The intention is that changes to the metadata model become even easier than they are today.
### Synchronous Ingestion Architecture
<p align="center">
<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/advanced/mcp-mcl/sync-ingestion.svg"/>
</p>
### Asynchronous Ingestion Architecture
<p align="center">
<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/advanced/mcp-mcl/async-ingestion.svg"/>
</p>
## Modeling
A Metadata Change Proposal is defined (in PDL) as follows