yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-12-01 21:27:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-ingestion/examples/recipes/file_to_console.dhub.yaml

11 lines
311 B
YAML
Raw Permalink Normal View History

Codegen avro + datahub kafka sink (#3) * Add codegen * New architecture + setup file -> console pipeline * Cleanup source loader * Basic Kafka metadata source * Kafka source and extractor * Add kwargs construct interface * Fix kafka source unit test * start working on pipeline test * kafka datahub sink * Make myself a profile * Ingest to datahub from kafka * Update codegen * Add restli transport * Fix bug in restli conversion
2021-02-05 21:03:04 -08:00
---
doc: Acryl to DataHub, datahubproject.io to datahub.com (#13252) Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-04-28 23:34:33 +09:00
# see https://docs.datahub.com/docs/generated/ingestion/sources/file for complete documentation
Codegen avro + datahub kafka sink (#3) * Add codegen * New architecture + setup file -> console pipeline * Cleanup source loader * Basic Kafka metadata source * Kafka source and extractor * Add kwargs construct interface * Fix kafka source unit test * start working on pipeline test * kafka datahub sink * Make myself a profile * Ingest to datahub from kafka * Update codegen * Add restli transport * Fix bug in restli conversion
2021-02-05 21:03:04 -08:00
source:
type: "file"
Use type + config everywhere
2021-02-12 12:05:41 -08:00
config:
Create examples directory
2021-02-10 16:35:31 -08:00
filename: "./examples/mce_files/single_mce.json"
Codegen avro + datahub kafka sink (#3) * Add codegen * New architecture + setup file -> console pipeline * Cleanup source loader * Basic Kafka metadata source * Kafka source and extractor * Add kwargs construct interface * Fix kafka source unit test * start working on pipeline test * kafka datahub sink * Make myself a profile * Ingest to datahub from kafka * Update codegen * Add restli transport * Fix bug in restli conversion
2021-02-05 21:03:04 -08:00
doc: Acryl to DataHub, datahubproject.io to datahub.com (#13252) Co-authored-by: Jay <159848059+jayacryl@users.noreply.github.com>
2025-04-28 23:34:33 +09:00
# see https://docs.datahub.com/docs/metadata-ingestion/sink_docs/console for complete documentation
Codegen avro + datahub kafka sink (#3) * Add codegen * New architecture + setup file -> console pipeline * Cleanup source loader * Basic Kafka metadata source * Kafka source and extractor * Add kwargs construct interface * Fix kafka source unit test * start working on pipeline test * kafka datahub sink * Make myself a profile * Ingest to datahub from kafka * Update codegen * Add restli transport * Fix bug in restli conversion
2021-02-05 21:03:04 -08:00
sink:
type: "console"
Reference in New Issue Copy Permalink