OpenMetadata/openmetadata-docs/content/partials/v1.7/connectors/pipeline/configure-ingestion.md

{% step srNumber=7 %}

{% stepDescription title="7. Configure Metadata Ingestion" %}

In this step we will configure the metadata ingestion pipeline, Please follow the instructions below

{% /stepDescription %}

{% stepVisualInfo %}

{% image src="/images/v1.7/connectors/configure-metadata-ingestion-pipeline.png" alt="Configure Metadata Ingestion" caption="Configure Metadata Ingestion Page" /%}

{% /stepVisualInfo %}

{% /step %}

{% extraContent parentTagName="stepsContainer" %}

Metadata Ingestion Options

• Name: This field refers to the name of ingestion pipeline, you can customize the name or use the generated name.
• Pipeline Filter Pattern (Optional): Use to pipeline filter patterns to control whether or not to include pipeline as part of metadata ingestion.
  • Include: Explicitly include pipeline by adding a list of comma-separated regular expressions to the Include field. OpenMetadata will include all pipeline with names matching one or more of the supplied regular expressions. All other schemas will be excluded.
  • Exclude: Explicitly exclude pipeline by adding a list of comma-separated regular expressions to the Exclude field. OpenMetadata will exclude all pipeline with names matching one or more of the supplied regular expressions. All other schemas will be included.
• Include lineage (toggle): Set the Include lineage toggle to control whether to include lineage between pipelines and data sources as part of metadata ingestion.
• Enable Debug Log (toggle): Set the Enable Debug Log toggle to set the default log level to debug.
• Mark Deleted Pipelines (toggle): Set the Mark Deleted Pipelines toggle to flag pipelines as soft-deleted if they are not present anymore in the source system.

{% /extraContent %}