yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-28 11:59:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/metadata-ingestion/examples/library/lineage_job_dataflow_new_api.py

13 lines
387 B
Python
Raw Permalink Normal View History

feat(ingest): airflow - add support to capture airflow executions, add high level dataflow jobs api to python sdk (#4615) Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-04-13 08:19:39 +02:00
from datahub.api.entities.datajob import DataFlow, DataJob
from datahub.emitter.rest_emitter import DatahubRestEmitter
# Create an emitter to the GMS REST API.
emitter = DatahubRestEmitter("http://localhost:8080")
fix(sdk/dataflow): deprecate cluster and use env and platform_instance instead (#8201) Co-authored-by: mohdsiddique <mohdsiddiquebagwan@gmail.com> Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-16 02:36:28 +05:30
jobFlow = DataFlow(env="prod", orchestrator="airflow", id="flow_new_api")
feat(ingest): airflow - add support to capture airflow executions, add high level dataflow jobs api to python sdk (#4615) Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-04-13 08:19:39 +02:00
jobFlow.emit(emitter)
dataJob = DataJob(flow_urn=jobFlow.urn, id="job1")
dataJob.emit(emitter)
Reference in New Issue Copy Permalink