2.5 KiB
| title | slug |
|---|---|
| Pipeline Mixin | /sdk/python/api-reference/pipeline-mixin |
module pipeline_mixin
Mixin class containing Pipeline specific methods
To be used by OpenMetadata class
class OMetaPipelineMixin
OpenMetadata API methods related to the Pipeline Entity
To be inherited by OpenMetadata
method add_pipeline_status
add_pipeline_status(fqn: str, status: PipelineStatus) → Pipeline
Given a pipeline and a PipelineStatus, send it to the Pipeline Entity
method add_task_to_pipeline
add_task_to_pipeline(pipeline: Pipeline, *tasks: Task) → Pipeline
The background logic for this method is that during Airflow backend lineage, we compute one task at a time.
Let's generalise a bit the approach by preparing a method capable of updating a tuple of tasks from the client.
Latest changes leave all the task management to the client. Therefore, a Pipeline will only contain the tasks sent in each PUT from the client.
method clean_pipeline_tasks
clean_pipeline_tasks(pipeline: Pipeline, task_ids: List[str]) → Pipeline
Given a list of tasks, remove from the Pipeline Entity those that are not received as an input.
e.g., if a Pipeline has tasks A, B, C, but we only receive A & C, we will remove the task B from the entity