yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-24 09:50:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenMetadata/ingestion
History
Keith Sirmons 42000053aa
Fixed Issue #10943: Impala query engine metadata ingestion and median function profiler (#10944) 
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.

* added the ordinalPosition data point into the Column constructor.

* renamed variable to better describe its usage.

* updated profile errors.
Hive connections now comment columns by default.

* removed print statements

* Cleaned up code by pulling check into its own function

* Updated median function to return null when it is being used for first and third quartiles.

* removed print statements and ran make py_format

* updated to fix some pylint errors.
imported Dialects to remove string compare to "impala" engine

* moved huge comment into function docstring.
This comment shows us the sql to get quartiles in Impala
2023-04-06 18:07:42 +02:00
..
examples
Fix#10584: Add Data Model as an entity (#10636)
2023-03-28 15:07:38 +00:00
operators
fixed #9656 - Add support for date type to column values to be between (#10890)
2023-04-04 17:16:44 +02:00
pipelines
Revert Change (#9534)
2022-12-28 14:05:18 +01:00
plugins
Add Pylint print checker and py-checkstyle as required (#8849)
2022-11-17 16:44:26 +00:00
src
Fixed Issue #10943: Impala query engine metadata ingestion and median function profiler (#10944)
2023-04-06 18:07:42 +02:00
tests
Fix#10584: Add lineage for Tableau data models (#10923)
2023-04-06 07:28:41 +02:00
__init__.py
[issue-1079] - MlFlow Connector (#1677)
2021-12-11 18:40:19 -08:00
.coveragerc
Add E2E CLI tests coverage to Sonar (#9908)
2023-01-25 11:51:25 +01:00
Dockerfile
1.0.0-alpha version upgrade (#10826)
2023-03-29 23:24:39 +05:30
Dockerfile.ci
Fix #9707 - Install instantclient and enable thick mode (#9761)
2023-01-19 08:08:56 +01:00
ingestion_dependency.sh
Prepare Ingestion Base Docker image (#8065)
2022-10-11 07:50:49 +02:00
LICENSE
…
README.md
[WIP] - Add Scarf gateway (#9652)
2023-01-11 09:33:09 +01:00
setup.cfg
Fix#7272: Improve logging when initializing workflow from CLI (#7522)
2022-09-19 08:00:00 -07:00
setup.py
Fix #10896: Fix snappy coded issue (#10919)
2023-04-05 12:12:47 +05:30
sonar-project.properties
Add E2E CLI tests coverage to Sonar (#9908)
2023-01-25 11:51:25 +01:00

README.md

This guide will help you setup the Ingestion framework and connectors
This guide will help you setup the Ingestion framework and connectors

Python version 3.7+

OpenMetadata Ingestion is a simple framework to build connectors and ingest metadata of various systems through OpenMetadata APIs. It could be used in an orchestration framework(e.g. Apache Airflow) to ingest metadata. Prerequisites

  • Python >= 3.7.x

Docs

Please refer to the documentation here https://docs.open-metadata.org/connectors