mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-07-19 15:31:59 +00:00
aec586a5fe
* updated metadata-ingestion and apis documents * minor changes in * addressing maven ci findings Co-authored-by: Parth Panchal <parthpanchal@Parths-MacBook-Air.local>
2.8 KiB
2.8 KiB
Pipeline
This schema defines the Pipeline entity. A pipeline enables the flow of data from source to destination through a series of processing steps. ETL is a type of pipeline where the series of steps Extract, Transform and Load the data.
$id: https://open-metadata.org/schema/entity/data/pipeline.json
Type: object
Properties
- id
required- Unique identifier that identifies a pipeline instance.
- $ref: ../../type/basic.json#/definitions/uuid
- name
required- Name that identifies this pipeline instance uniquely.
- Type:
string - Length: between 1 and 64
- displayName
- Display Name that identifies this Pipeline. It could be title or label from the source services.
- Type:
string
- fullyQualifiedName
- A unique name that identifies a pipeline in the format 'ServiceName.PipelineName'.
- Type:
string - Length: between 1 and 64
- description
- Description of this Pipeline.
- Type:
string
- version
- Metadata version of the entity.
- $ref: ../../type/basic.json#/definitions/entityVersion
- updatedAt
- Last update time corresponding to the new version of the entity.
- $ref: ../../type/basic.json#/definitions/dateTime
- updatedBy
- User who made the update.
- Type:
string
- pipelineUrl
- Pipeline URL to visit/manage. This URL points to respective pipeline service UI.
- Type:
string - String format must be a "uri"
- concurrency
- Concurrency of the Pipeline.
- Type:
integer
- pipelineLocation
- Pipeline Code Location.
- Type:
string
- startDate
- Start date of the workflow.
- $ref: ../../type/basic.json#/definitions/dateTime
- tasks
- All the tasks that are part of pipeline.
- Type:
array- Items
- $ref: ../../type/entityReference.json
- followers
- Followers of this Pipeline.
- $ref: ../../type/entityReference.json#/definitions/entityReferenceList
- tags
- Tags for this Pipeline.
- Type:
array- Items
- $ref: ../../type/tagLabel.json
- href
- Link to the resource corresponding to this entity.
- $ref: ../../type/basic.json#/definitions/href
- owner
- Owner of this pipeline.
- $ref: ../../type/entityReference.json
- service
required- Link to service where this pipeline is hosted in.
- $ref: ../../type/entityReference.json
This document was updated on: Monday, October 18, 2021