yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-31 12:39:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenMetadata/openmetadata-docs/content/openmetadata/connectors/pipeline/airflow/index.md
Pere Miquel Brull 15e1bb531a
Docs - Python requirements & metadata docker (#6790) 
Docs - Python requirements & metadata docker (#6790)
2022-08-18 11:43:45 +02:00

1.9 KiB
Raw Blame History

title slug
Airflow /openmetadata/connectors/pipeline/airflow

Airflow

In this section, we provide guides and references to use the Airflow connector.

Configure and schedule Airflow metadata workflow from the OpenMetadata UI:

If you don't want to use the OpenMetadata Ingestion container to configure the workflows via the UI, then you can check the following docs to extract metadata directly from your Airflow instance or via the CLI:

Connection Options

  • Host and Port: URL to the Airflow instance.
  • Number of Status: Number of status we want to look back to in every ingestion (e.g., Past executions from a DAG).
  • Connection: Airflow metadata database connection. See these docs for supported backends.

In terms of connection we support the following selections:

  • backend: Should not be used from the UI. This is only applicable when ingesting Airflow metadata locally by running the ingestion from a DAG. It will use the current Airflow SQLAlchemy connection to extract the data.
  • MySQL, Postgres, MSSQL and SQLite: Pass the required credentials to reach out each of these services. We will create a connection to the pointed database and read Airflow data from there.