OpenMetadata

mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-09-18 21:42:27 +00:00

History

Enhance SQL column processing for BigQuery ingestion (#20408 )

- Refactored the handling of nested columns in `sql_column_handler.py` to prioritize source-provided children, ensuring they override any derived children.
- Removed the overridden `_process_col_type` method in `bigquery/metadata.py` to streamline column type handling, enforcing the use of the standard path for BigQuery.

This update improves the accuracy of column metadata processing and simplifies the codebase.

2025-03-25 14:40:46 +05:30

examples

Minor: Query Cost Table Aggregation Endpoint (#20270 )

2025-03-17 11:33:50 +05:30

operators

Feature: Reverse Metadata Ingestion (#19723 )

2025-02-25 18:41:08 +05:30

pipelines

…

plugins

…

src

Enhance SQL column processing for BigQuery ingestion (#20408 )

2025-03-25 14:40:46 +05:30

tests

Revert "Fixes #17660 : Oracle handle quotes for lowercase columns in workflow agents (#20309 )" (#20364 )

2025-03-20 21:02:58 +05:30

__init__.py

…

Dockerfile

VERSION - Bump ingestion debian to bookworm (#19917 )

2025-02-24 10:45:25 +01:00

Dockerfile.ci

VERSION - Bump ingestion debian to bookworm (#19917 )

2025-02-24 10:45:25 +01:00

ingestion_dependency.sh

MINOR - Upgrade Airflow to 2.10.5 (#19840 )

2025-02-20 17:11:38 +01:00

LICENSE

Docs - Ingestion License (#17893 )

2024-09-17 08:58:53 -07:00

Makefile

…

pyproject.toml

MINOR: add reportExplicitAny = false for basedpyright (#18777 )

2024-11-25 14:38:04 +00:00

README.md

…

setup.py

Opensearch connector (#19698 )

2025-03-18 18:45:25 +05:30

sonar-project.properties

…

README.md

This guide will help you setup the Ingestion framework and connectors

OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata. Prerequisites

Python >= 3.8.x

Docs

Please refer to the documentation here https://docs.open-metadata.org/connectors

TopologyRunner

All the Ingestion Workflows run through the TopologyRunner.

The flow is depicted in the images below.

TopologyRunner Standard Flow

TopologyRunner Multithread Flow