7 Commits

Author SHA1 Message Date
Pere Miquel Brull
6693b4ae40
[issue-2262] - Simplify sample data DAG (#2337)
* Simplify sample data DAG

* Remove mkdir

* Generate sources before running compose

* Generate sources to install models to ingestion image

* Add python-on-whales for docker --start

* Remove python-on-whales from base

* Install venv

* Setup python
2022-01-22 16:37:56 +05:30
Pere Miquel Brull
2ec32017bb
[issue-1698] - Airflow DockerOperator setup (#1730)
* Fix typo

* Clean setup

* Update ingestion local image to be barebone on connector dependencies

* Prepare ingestion connectors base image

* Add system dependencies

* Prepare docker CLI

* Add docker provider

* Prepare entrypoint for the image

* Remove DBT pipeline as per Issue 1658

* Add TODO for ingestion build

* Bind docker socket

* Update comment

* Update README

* Use DockerOperator in sample data

* Build images with latest tag

* Prepare symlink to pass the volume to the DockerOperator

* Update README

* Prepare Base image for CI

* COPY multiple files into dir

* COPY multiple files into dir

* Remove DBT source as is now part of table ingestion

* Build docker base in run_local_docker
2021-12-18 16:41:38 +01:00
parthp2107
a9e70e6a89
License header update (#1498)
* updated liscense header

* updated license header

* updated license header

* updated license header

* addressing stylecheck findings
2021-12-01 12:46:28 +05:30
Ayush Shah
5dc3bb9297
Docker Support from Python added (#1158)
* Script modified - supports running from different locations

* Docker support from Python CLI

* Docker plugin setup.py

* Paths for latest and local dockers updated

* Resolved Comments - Docker CLI optimized, timestamp added

* help attribute added to options

* Docker clean code refactored
2021-11-12 10:30:28 -08:00
Ayush Shah
93921814af
Docker fix - latest release changes (#983)
* Docker fix - Architecture, MySQL

* Docker Airflow API Dagrun support

* Docker latest changes modified
2021-10-30 09:05:30 -07:00
Ayush Shah
72e4b8831f
Removing Cron from all configs (#773)
* Removing Cron from all configs

* Removing Cron from Documentation
2021-10-14 15:21:47 -07:00
Ayush Shah
85b6b72848
Airflow docker (#762)
* Airflow Docker implementation - Ingestion

* Dockerfiles modified
2021-10-14 07:46:24 -07:00