mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-10-22 14:24:35 +00:00
47e953f9d3
* Ensure we take columns ordered from the sampler This is to avoid analyzing columns with data from other columns * Remove expectation of address to have Sensitive tag This is for a couple of reasons: - First: per our internal definition it should actually be Non Sensitive. - Second: presidio actually picks SOME of them up as PERSON (Sensitive) entities, but since we've raised the tolerance, now we're not classifying them as Sensitive.
This guide will help you setup the Ingestion framework and connectors
OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata. Prerequisites
- Python >= 3.9.x
Docs
Please refer to the documentation here https://docs.open-metadata.org/connectors
TopologyRunner
All the Ingestion Workflows run through the TopologyRunner.
The flow is depicted in the images below.
TopologyRunner Standard Flow
TopologyRunner Multithread Flow
