19 Commits

Author SHA1 Message Date
Onkar Ravgan
f1969c4fce
added dependencies for pymssql (#4876)
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-05-11 10:41:13 +02:00
Pere Miquel Brull
23b95a267b
Fix #4828 - Remove lineage backend from airflow image (#4833)
* Remove callbacks from DAG

* Remove lineage backend from image
2022-05-11 07:36:33 +02:00
Vj-L
2c7a327c6b
Issue #3094 (#4500)
* Added the named volumes for improvement

* Mapped the named volumes

* Updated the Named Volumes

* Updated the Volumes on top level

* Added the named volumes
2022-05-03 14:19:31 +05:30
Akash Jain
792c67a8f1
Arguments to provide ingestion depenedency. defaults to 'all' (#4613) 2022-05-02 16:09:44 +05:30
codingwithabhi
32d392e0cb
deltalake-test-connection-fixed (#4387) 2022-04-23 00:40:44 -07:00
Mayur Singal
e4914cfae4
Fix #4354: Added Hive SASL dependency (#4355) 2022-04-22 19:52:59 +05:30
Pere Miquel Brull
d6b49f070c
Fix #3951 - OpenMetadata Airflow REST (#4034)
Fix #3951 - OpenMetadata Airflow REST (#4034)
2022-04-11 18:38:26 +02:00
Ayush Shah
116f8d333d
Airflow Auth fix + Version Downgrade + Increased Docker Min Memory Limit (#3233) 2022-03-08 13:51:07 -08:00
Pere Miquel Brull
2ec32017bb
[issue-1698] - Airflow DockerOperator setup (#1730)
* Fix typo

* Clean setup

* Update ingestion local image to be barebone on connector dependencies

* Prepare ingestion connectors base image

* Add system dependencies

* Prepare docker CLI

* Add docker provider

* Prepare entrypoint for the image

* Remove DBT pipeline as per Issue 1658

* Add TODO for ingestion build

* Bind docker socket

* Update comment

* Update README

* Use DockerOperator in sample data

* Build images with latest tag

* Prepare symlink to pass the volume to the DockerOperator

* Update README

* Prepare Base image for CI

* COPY multiple files into dir

* COPY multiple files into dir

* Remove DBT source as is now part of table ingestion

* Build docker base in run_local_docker
2021-12-18 16:41:38 +01:00
Ayush Shah
c95f1542eb
Docker fix (#1101)
* Docker Fix, User Ingestion Fix

* Docker file optimized

* ES docker version rolledback

* Healthcheck removed - incompatible with 3.9

* Docker & Airflow Config files updated with - resolved comments

* Openmetadata managed Airflow apis support added

* Dockerfile updated

* Docker file Updated with release package

* Elasticsearch allocated memory increased
2021-11-10 21:22:32 -08:00
Akash Jain
06b2f70e3e
kubernetes docker image compatibility for #1026 (#1042) 2021-11-03 08:18:52 -07:00
Ayush Shah
93921814af
Docker fix - latest release changes (#983)
* Docker fix - Architecture, MySQL

* Docker Airflow API Dagrun support

* Docker latest changes modified
2021-10-30 09:05:30 -07:00
Ayush Shah
759574a8de
Ingestion Optimization - Sample Users, Dockerfiles, Removal of Pandas (#935)
* Sample Profile Data for Sample Tables (#815)

* Sample Profile Data for Sample Tables

* Disabling Profile as Default

* Added Sample Profile Data to 3 additional sample tables

* Sample Tables fixed (#850)

* Pydantic fix, Docker update (#860)

* Setup.py Modified with openmetadata-airflow package, docker update

* Setup.py Modified

* Update setup.py

* Removed Pandas from Sample Data

* Sample Users added under sample data

* Sample User Standalone pipelines and modules removed

* Docker release package updated

* Dockerfile updated, removed redundant files

* Setup.py removed from ingestion src directory

* User Resource failing check resolved

* Modifying Usage Columns Datatype
2021-10-26 09:14:24 -07:00
Ayush Shah
d4d66ea21d
Docker fix (#200)
* Modifications in Docker

* Docker config and script Modifications

* Config changes ElasticSearch
2021-08-16 18:37:04 +05:30
Ayush Shah
03ad583744 Registry dependency removed 2021-08-14 00:09:51 +05:30
Ayush Shah
d9cd3e4856 Connector Dependency Cleanup and Docker Modification 2021-08-13 02:29:36 +05:30
Ayush Shah
87764d723d Pipeline Credential Changes 2021-08-09 09:31:43 +05:30
Ayush Shah
982ca639b5 Docker Scripts and Commands modified 2021-08-03 01:26:20 +05:30
sureshms
8ce7c879e8 OpenMetadata snapshot release 0.3 2021-08-01 14:27:44 -07:00