mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-07-19 15:31:59 +00:00

* FIX #16284 - Toggle if we want to raise workflow errors * schema * schema * move prop * fix * move prop * improve error handling * Update openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/IngestionPipelineRepository.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/IngestionPipelineRepository.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add the `Raise on Error` option to the ingestion schedule step * Revert "Update openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/IngestionPipelineRepository.java" This reverts commit 985b73513a59695c6bb39ad41c2d273bbf4e5d22. * Update the tests * Fix sonar issue --------- Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This guide will help you setup the Ingestion framework and connectors
OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata. Prerequisites
- Python >= 3.8.x
Docs
Please refer to the documentation here https://docs.open-metadata.org/connectors

TopologyRunner
All the Ingestion Workflows run through the TopologyRunner.
The flow is depicted in the images below.
TopologyRunner Standard Flow
TopologyRunner Multithread Flow