yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-11-03 20:19:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenMetadata/openmetadata-docs/content/partials/v1.4/connectors/pipeline/configure-ingestion.md
Imri Paran 67c57ec4f0
docs: 1.4.x-SNAPSHOT (#15506) 
* docs: 1.4.x-SNAPSHOT
added partials
added images
updated paths
2024-03-11 09:42:26 +00:00

1.7 KiB
Raw Blame History

{% step srNumber=7 %}

{% stepDescription title="7. Configure Metadata Ingestion" %}

In this step we will configure the metadata ingestion pipeline, Please follow the instructions below

{% /stepDescription %}

{% stepVisualInfo %}

{% image src="/images/v1.2/connectors/configure-metadata-ingestion-pipeline.png" alt="Configure Metadata Ingestion" caption="Configure Metadata Ingestion Page" /%}

{% /stepVisualInfo %}

{% /step %}

{% extraContent parentTagName="stepsContainer" %}

Metadata Ingestion Options

  • Name: This field refers to the name of ingestion pipeline, you can customize the name or use the generated name.
  • Pipeline Filter Pattern (Optional): Use to pipeline filter patterns to control whether or not to include pipeline as part of metadata ingestion.
    • Include: Explicitly include pipeline by adding a list of comma-separated regular expressions to the Include field. OpenMetadata will include all pipeline with names matching one or more of the supplied regular expressions. All other schemas will be excluded.
    • Exclude: Explicitly exclude pipeline by adding a list of comma-separated regular expressions to the Exclude field. OpenMetadata will exclude all pipeline with names matching one or more of the supplied regular expressions. All other schemas will be included.
  • Include lineage (toggle): Set the Include lineage toggle to control whether to include lineage between pipelines and data sources as part of metadata ingestion.
  • Enable Debug Log (toggle): Set the Enable Debug Log toggle to set the default log level to debug.
  • Mark Deleted Pipelines (toggle): Set the Mark Deleted Pipelines toggle to flag pipelines as soft-deleted if they are not present anymore in the source system.

{% /extraContent %}