250 Commits

Author SHA1 Message Date
Vivek Ratnavel Subramanian
f0927ecebb
Fix #3236: Add support for secure docker compose with an env file (#3241)
* Fix #3236: Add support for secure docker compose with an env file
2022-03-08 07:05:31 -08:00
Snyk bot
01e17d0d66
fix: docker/images/minimal-ubuntu/Dockerfile to reduce vulnerabilities (#3126)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-UBUNTU1604-LIBGCRYPT20-1585790
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
- https://snyk.io/vuln/SNYK-UBUNTU1604-SYSTEMD-1320131
2022-03-04 12:03:11 -08:00
Akash Jain
123ef9cab7
Bump up release version (#2968) 2022-02-24 12:48:10 +05:30
Akash Jain
3744cd250b
fix: cherry pick for 0.8.3 (#2893)
* update dockerfile with new release download url

* update tags to 0.8.3
2022-02-21 13:55:29 +05:30
Pere Miquel Brull
76f4ccd590
Fix #2735 - Simplify Airflow properties extraction (#2749)
* Simplify lineage properties extraction

* Add network name

* Format
2022-02-14 16:53:42 +01:00
Pere Miquel Brull
edea57b4ad
Add env vars for host and port + verbosity (#2490) 2022-01-29 16:02:36 -08:00
Akash Jain
48c1ad63c4
fix: ElasticSearch and Ingestion connectivity from openmetadata post soft link (#2470)
* soft linked openmetadata.yaml

* quick fix for ingestion and elasticsearch connect from server
2022-01-27 13:55:54 +05:30
Akash Jain
59aa59e284
soft linked openmetadata.yaml (#2469) 2022-01-27 13:07:10 +05:30
Akash Jain
20309cde64
fix:cherry pick from 0.8.0 and miscellaneous fixes (#2445)
* updating ingestion-core version to 0.8.0

* updating ingestion version to 0.8.0

* fixing file copy path

* update docker images to point to 0.8.0

* set up new release version

* reset ingestion version to 0.9.0.dev0

* reset ingestion-core version to 0.9.0
2022-01-26 14:00:27 +05:30
Pere Miquel Brull
1eb3f1ad41
[issue-2322] - Backup CLI (#2349)
* Prepare cli module

* Check call

* Prepare backup

* Run pytest on schema changes

* Remove traces

* Add backup plugin

* run generate during CI

* Install after generate

* Address logger hotspot
2022-01-22 12:08:14 -08:00
Pere Miquel Brull
6693b4ae40
[issue-2262] - Simplify sample data DAG (#2337)
* Simplify sample data DAG

* Remove mkdir

* Generate sources before running compose

* Generate sources to install models to ingestion image

* Add python-on-whales for docker --start

* Remove python-on-whales from base

* Install venv

* Setup python
2022-01-22 16:37:56 +05:30
Pere Miquel Brull
94ecb1e030
Map ports to localhost (#2078) 2022-01-06 14:32:31 +01:00
Akash Jain
0bbb283e70
fix: update base image to alpine linux and refactor dockerflie instructions (#2050)
* +alpine img and refactor dockerflie instructions

* local-metadata server image with alpine linux

* correcting rm openmetadata-*.tar.gz

* fix COPY destination folder ref
2022-01-05 10:11:24 -08:00
Pere Miquel Brull
2ec32017bb
[issue-1698] - Airflow DockerOperator setup (#1730)
* Fix typo

* Clean setup

* Update ingestion local image to be barebone on connector dependencies

* Prepare ingestion connectors base image

* Add system dependencies

* Prepare docker CLI

* Add docker provider

* Prepare entrypoint for the image

* Remove DBT pipeline as per Issue 1658

* Add TODO for ingestion build

* Bind docker socket

* Update comment

* Update README

* Use DockerOperator in sample data

* Build images with latest tag

* Prepare symlink to pass the volume to the DockerOperator

* Update README

* Prepare Base image for CI

* COPY multiple files into dir

* COPY multiple files into dir

* Remove DBT source as is now part of table ingestion

* Build docker base in run_local_docker
2021-12-18 16:41:38 +01:00
Sriharsha Chintalapani
39e4163d6f
Fix #1814: Add migrate-all option to update ElasticSearch index mappings (#1815)
* Fix #1814: Add migrate-all option to update ElasticSearch index mappings

* Fix #1814: Add migrate-all option to update ElasticSearch index mappings
2021-12-16 20:56:58 -08:00
Akash Jain
b79784b921
updated url to match with 0.7.0 release (#1798) 2021-12-16 16:04:53 +05:30
Ayush Shah
f288cef6d1
Update docker-compose.yml (#1792)
* Update docker-compose.yml

* Update docker-compose.yml
2021-12-16 14:39:45 +05:30
Ayush Shah
07924f80c3
Added Metabase Connector (#1726)
* Added Metabase Connector

* Metabase Added support for lineage

* Modified Metabase as per comments

* Metabase filter patterns added

* Update databaseService.json
2021-12-15 09:53:55 -08:00
Akash Jain
a9969dec99
update docker image tags to 0.6.0 (#1766) 2021-12-15 14:55:51 +05:30
Akash Jain
d0d3fce332
fix: add container_name to docker-compose (#1639)
* adding container_name

* fixed typo --run to --start

* updated container_name prefixing openmetadata_

* updated container_name to prefix openmetadata_
2021-12-10 08:01:08 -08:00
Ayush Shah
777afd1d3f
Openmetadata.yaml for docker updated (#1606)
* Dockerfiles changed - updated elasticsearch, openmetadata.yaml

* Added blue color to docker info
2021-12-07 11:27:16 -08:00
parthp2107
a9e70e6a89
License header update (#1498)
* updated liscense header

* updated license header

* updated license header

* updated license header

* addressing stylecheck findings
2021-12-01 12:46:28 +05:30
Ayush Shah
47a97e7be0 FQN changed from bigquery to bigquery_gcp 2021-11-29 16:57:39 +05:30
Ayush Shah
267a3d7aa2 Removed Platform from docker-compose.yml files, emulation fix 2021-11-26 18:10:27 +01:00
Akash Jain
74a10382a2
Fix: Docker Image to use new Dist Location while running locally (#1381)
* fix path for dist folder

* updated docs wrt new dist folder
2021-11-25 12:33:20 +05:30
Ayush Shah
11e2229972
Added OMeta API, removed wget (#1376)
* Added OMeta API, removed wget

* Added curl instead of Wget in run_local_docker.sh
2021-11-24 11:57:33 -08:00
Akash Jain
e7072a3177
prepare for 0.6.0 Docker Release (#1213) 2021-11-17 14:57:12 +05:30
Ayush Shah
2111f6c938
Docker Commands Modified (#1199)
* Docker ingestion modified

* Command line commands updated
2021-11-16 00:25:40 -08:00
Akash Jain
58cf1b012f
feat: Docker Image Push Workflow (#1191)
* copy command use path from root folder

* Add workflows for docker release to github
2021-11-15 09:36:57 -08:00
Ayush Shah
5dc3bb9297
Docker Support from Python added (#1158)
* Script modified - supports running from different locations

* Docker support from Python CLI

* Docker plugin setup.py

* Paths for latest and local dockers updated

* Resolved Comments - Docker CLI optimized, timestamp added

* help attribute added to options

* Docker clean code refactored
2021-11-12 10:30:28 -08:00
Akash Jain
b2d1d0ca34
fix:Airflow configurations to match docker-compose (#1157)
Airflow configurations to match docker-compose yaml values
2021-11-11 19:23:24 +05:30
Ayush Shah
dd7bb5a79f
airflow config added to openmetadata conf file (#1155) 2021-11-11 18:31:53 +05:30
Ayush Shah
c95f1542eb
Docker fix (#1101)
* Docker Fix, User Ingestion Fix

* Docker file optimized

* ES docker version rolledback

* Healthcheck removed - incompatible with 3.9

* Docker & Airflow Config files updated with - resolved comments

* Openmetadata managed Airflow apis support added

* Dockerfile updated

* Docker file Updated with release package

* Elasticsearch allocated memory increased
2021-11-10 21:22:32 -08:00
Akash Jain
06b2f70e3e
kubernetes docker image compatibility for #1026 (#1042) 2021-11-03 08:18:52 -07:00
Ayush Shah
93921814af
Docker fix - latest release changes (#983)
* Docker fix - Architecture, MySQL

* Docker Airflow API Dagrun support

* Docker latest changes modified
2021-10-30 09:05:30 -07:00
Ayush Shah
759574a8de
Ingestion Optimization - Sample Users, Dockerfiles, Removal of Pandas (#935)
* Sample Profile Data for Sample Tables (#815)

* Sample Profile Data for Sample Tables

* Disabling Profile as Default

* Added Sample Profile Data to 3 additional sample tables

* Sample Tables fixed (#850)

* Pydantic fix, Docker update (#860)

* Setup.py Modified with openmetadata-airflow package, docker update

* Setup.py Modified

* Update setup.py

* Removed Pandas from Sample Data

* Sample Users added under sample data

* Sample User Standalone pipelines and modules removed

* Docker release package updated

* Dockerfile updated, removed redundant files

* Setup.py removed from ingestion src directory

* User Resource failing check resolved

* Modifying Usage Columns Datatype
2021-10-26 09:14:24 -07:00
Pere Miquel Brull
1291bf6ec0
[Issue-892] - GET operations return service (#916)
* Force image build in local docker

* Use service as default field query param for entities with required service

* Return service in List

* Naming convention

* Update Repository setFields to include service

* Revert resource changes

* Revert resource changes

* Update tests to check we always return the service, even if not specified

* Return service for database
2021-10-24 10:26:28 -07:00
vijaypm
b8863648d6
Issue 898 (#905)
* ISSUE-898: additional information in the prerequisities for building and running code

* ISSUE-898: removed unreachable old doc

* ISSUE-898: added new docker compose to expose MySQL and ES ports to host machines

* ISSUE-898: changed jdbc connect url to allow Public Key Retrieval

* ISSUE-898: fixed log name to openmetadata.log

Co-authored-by: Vijay Mariadassou <vijay@mariadassou.com>
2021-10-22 19:35:12 -07:00
Ayush Shah
85b6b72848
Airflow docker (#762)
* Airflow Docker implementation - Ingestion

* Dockerfiles modified
2021-10-14 07:46:24 -07:00
Ayush Shah
319f7a795c
Docker local (#723)
* added docker file for local build and ingestion

* Docker Modified for local setup

* Ingestion script optimized

* Added script to directly run Local Docker

Co-authored-by: Parth Panchal <parthpanchal@Parths-MacBook-Air.local>
2021-10-09 10:58:33 -07:00
Ayush Shah
0ae4ef21cd Folder not found fixed 2021-08-16 23:55:28 +05:30
Ayush Shah
ca5ca88f37 change release path 2021-08-16 23:55:28 +05:30
Ayush Shah
71444a815a Docker Platform and Commands Modified 2021-08-16 23:55:28 +05:30
Ayush Shah
d4d66ea21d
Docker fix (#200)
* Modifications in Docker

* Docker config and script Modifications

* Config changes ElasticSearch
2021-08-16 18:37:04 +05:30
Ayush Shah
03ad583744 Registry dependency removed 2021-08-14 00:09:51 +05:30
Suresh Srinivas
1619b6f2e2 Fix Docker: update the binary names 2021-08-12 11:10:30 -07:00
parthp2107
8a92d253db
replaced catalog to openmetadata #2 (#47)
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
2021-08-05 10:15:52 +05:30
Ayush Shah
982ca639b5 Docker Scripts and Commands modified 2021-08-03 01:26:20 +05:30
parthp2107
b5baa9f2db applied license to the backend files 2021-08-02 15:08:30 +05:30
sureshms
8ce7c879e8 OpenMetadata snapshot release 0.3 2021-08-01 14:27:44 -07:00