Logo
Explore Help
Register Sign In
yujunjun/OpenMetadata
1
0
Fork 0
You've already forked OpenMetadata
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-23 01:12:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenMetadata/ingestion/connectors/main.py

22 lines
456 B
Python
Raw Normal View History

[issue-1698] - Airflow DockerOperator setup (#1730) * Fix typo * Clean setup * Update ingestion local image to be barebone on connector dependencies * Prepare ingestion connectors base image * Add system dependencies * Prepare docker CLI * Add docker provider * Prepare entrypoint for the image * Remove DBT pipeline as per Issue 1658 * Add TODO for ingestion build * Bind docker socket * Update comment * Update README * Use DockerOperator in sample data * Build images with latest tag * Prepare symlink to pass the volume to the DockerOperator * Update README * Prepare Base image for CI * COPY multiple files into dir * COPY multiple files into dir * Remove DBT source as is now part of table ingestion * Build docker base in run_local_docker
2021-12-18 16:41:38 +01:00
import json
import os
from metadata.ingestion.api.workflow import Workflow
def main():
# DockerOperator expects an env var called config
config = os.environ["config"]
# Load the config string representation
workflow_config = json.loads(config)
workflow = Workflow.create(workflow_config)
workflow.execute()
workflow.raise_from_status()
workflow.print_status()
workflow.stop()
if __name__ == "__main__":
main()
Reference in New Issue Copy Permalink
Powered by Gitea Version: 1.23.5 Page: 1023ms Template: 72ms
English
Bahasa Indonesia Deutsch English Español Français Gaeilge Italiano Latviešu Magyar nyelv Nederlands Polski Português de Portugal Português do Brasil Suomi Svenska Türkçe Čeština Ελληνικά Български Русский Українська فارسی മലയാളം 日本語 简体中文 繁體中文(台灣) 繁體中文(香港) 한국어
Licenses API