68 Commits

Author SHA1 Message Date
Sriharsha Chintalapani
9004434c18
Fix #9752: docker openmetadata_ingestion should only depend on db (#9753)
* Fix #9752: docker openmetadata_ingestion should only depend on db

* Fix the typo

* Adding service_started for local docker

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
2023-01-18 08:45:15 +01:00
Nahuel
2c43ebba6f
Fix#9448: Add ES volumes (#9506)
* Add ES volumes

* Fix run_local_docker script

* Fix error run_local_docker script

* Update Es volumes in docker-compose files
2022-12-23 17:33:30 +01:00
Pere Miquel Brull
8806244348
Removing ES volumes (#9503)
* Fix ES volumes permissions

* Remove ES volumes
2022-12-22 18:02:09 +01:00
Nahuel
a2b34dd0f4
Fix: Update Ingestion docker images and fix python libraries dependencies (#9342)
* Update Ingestion docker images and fix python libraries dependencies

* Install also apache-airlfow-providers-http
2022-12-16 14:46:25 +00:00
Nahuel
e998ecc407
Deprecate configurations for auth provider configuration on Airflow (#8896)
* Deprecate configurations for auth provider configuration on Airflow

* Fix code after merging

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2022-12-15 12:28:46 -08:00
Vj-L
0a570c8585
Fixed the mounting of volumes (#8963)
* Fixed the mounting of volumes

* Update docker/local-metadata/docker-compose-postgres.yml

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update docker/metadata/docker-compose-postgres.yml

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Added permission step

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
2022-11-23 14:37:53 +05:30
Vj-L
2c055641be
Added host volumes for Database service (#8722)
* Added host volumes for Database service

* Formated the python file

* Done changes according to the comments on PR

* Changes done as per the comments on Draft PR

* Fix PyLint

* Changes done as per the comments on Draft PR

* Updated the path of volume for Database

* Added cleanup state in workflows

* Added cleanup state in workflows

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-11-18 14:05:33 +01:00
Rahul Surwade
a286a0fcbc
Removed unnecessary docker-compose files (#8085)
* Delete docker-compose-dev-postgres.yml

* Delete docker-compose-dev.yml

* Update openmetadata-server.md

* Update openmetadata-server.md

* Update openmetadata-server.md
2022-10-31 12:50:12 +05:30
Pere Miquel Brull
339f123f92
Fix #7477 - Paginate Airflow logs (#8145)
* Paginate last dag logs

* clean logic

* Prep pagination

* Prep pagination

* Format

* Update docker/local-metadata/docker-compose.yml

Co-authored-by: Nahuel <nahuel@getcollate.io>

* UI : Add pagination support for airflow logs

* Addressing review comments

Co-authored-by: Nahuel <nahuel@getcollate.io>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-10-21 17:32:56 +05:30
Vj-L
e1b4229074
Fix/env variable (#8199)
* Updated env variable for heap

* Updated env variable for heap

* Updated the env variables

* Updated the env variables

* Updated the env variables

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-10-19 11:00:11 +05:30
Pere Miquel Brull
19b2158d56
Use postgres 14 (#8207) 2022-10-18 18:29:25 +05:30
Sriharsha Chintalapani
65ee573a7d
Fix HEAP OPTS param (#8094) 2022-10-11 16:30:09 -07:00
Vj-L
0926c754aa
Updated env variable for heap (#8067)
* Updated env variable for heap

* Updated env variable for heap

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-10-11 19:29:22 +05:30
Pere Miquel Brull
339abc5bf3
Prepare Ingestion Base Docker image (#8065)
Prepare Ingestion Base Docker image (#8065)
2022-10-11 07:50:49 +02:00
Sriharsha Chintalapani
801012d2a6
Add remote debug port (#8045)
* Add remote debug port

* Add remote debug port

* Add remote debug port

* Add remote debug port
2022-10-11 11:13:15 +05:30
kmalki
69ddc7081c
Add environment variables in docker compose for elasticsearch (#8026) 2022-10-08 23:51:59 +05:30
Akash Jain
1e31245bbe
fix: Remove Lineage Backend Configs (#7836) 2022-10-01 11:23:24 +05:30
Akash Jain
c4b0bb0538
fix path for airflow volumes (#7827) 2022-09-30 18:20:38 +05:30
Vj-L
489d506e3a
Removed the volume for Database (#7798)
* Removed the volume for Database

* Removed the volume for Database

* Added named volume

* Added named volume

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-09-29 21:44:12 +05:30
Vj-L
6cfd16925a
Added default volumes (#7741)
* Added default volumes

* Ran make py_format for formatting

* Added sample data in mount directories

* Updated as per the comments

* Done changes for --ingest_sample_data flag

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-09-29 14:53:55 +05:30
Akash Jain
b3d340e403
Fix: env variables with basic auth changes for docker compose (#7693) 2022-09-23 23:52:28 +05:30
Ashok Mokshagundam
f62f29c806
Update mysql-script.sql (#7652)
updating database airflow_db with proper character set and collate
2022-09-22 14:41:58 +05:30
Akash Jain
08166ac239
fix typo error (#7542) 2022-09-19 15:21:49 +05:30
Akash Jain
fc8312c5fb
feat: Refactor openmetadata/ingestion docker image (#7456)
* centralize openmetadata/ingestion docker image

* update volume mappings; fix ingestion tag!

* fix PR comments

* fix cypress mysql tests!
2022-09-19 09:20:54 +05:30
Pere Miquel Brull
6700f169ab
Fix #7165 - Handle SSL certificates on ometa client (#7502)
Fix #7165 - Handle SSL certificates on ometa client (#7502)
2022-09-16 09:39:27 +02:00
Sriharsha Chintalapani
656b50dd3a
Fix #7469: Refactor OpenMetadata code modules (#7474) 2022-09-14 23:14:02 -07:00
Nahuel
ed58c27f76
Remove unused property extra_hosts from docker-compose files (#7096)
* Remove unused properties from docker-compose files

* Minor fix in run_local_docker script
2022-08-31 21:30:24 +02:00
Nahuel
db41be4c4a
Fix#6929: docker-compose services should wait until OpenMetadata DB is created (#7025)
* Added the docker compose for postgres

* Added changes as per the comments on PR

* Airflow Connectivity to Postgres

* Changes are done as per the comments

* Replocate changes to local dockerfile

* Fixed Airflow Dockerfile

* docker-compose services should wait until OpenMetadata DB is created

* remove unnecesary wait for db service

* Add dependencies to airflow image

* Add missing dependencies to airflow Dockerfile

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2022-08-30 23:55:46 +02:00
Vj-L
8bf34fcdc1
Fix/airflow dag issue (#6759)
* Added the correct path for dags volume

* Updated the dags volume path

* Removed the library from the Dockerfile of ingestion

* Removed the libmxl2 library from Docker file

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-08-25 18:49:46 +05:30
Ayush Shah
a6db2e8a84
Fix for profiler: modified filter patterns and added error handling (#6608) 2022-08-08 10:43:17 +05:30
Akash Jain
fa5105191b
Fix/issue-5883 (#5958) 2022-07-12 19:55:38 +05:30
Vivek Ratnavel Subramanian
da93e7c925
Fix Authentication Public Keys env variable in docker compose (#5948) 2022-07-07 12:19:16 -07:00
Pere Miquel Brull
97761b97a6
Prepare postgres migrations (#5814) 2022-07-01 10:18:24 +02:00
mohitdeuex
be18feb928
[Backend] Websocket implementation added (#5440)
* [Backend] Websocket implementation added

* [Backend] Websocket implementation added [google formatter fix]

* [backend] Updated Reiew Comments

* [backend] Updated CheckStyle
2022-06-15 10:14:47 +05:30
Vivek Ratnavel Subramanian
059acfc529
Backend: Enforce principal domain in the JWT filter (#5155) 2022-06-01 10:07:12 -07:00
Sriharsha Chintalapani
cf3d4314dd
Fix #1607 Operational metrics for OpenMetadata system observability (#4386)
Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
2022-05-27 15:39:13 +05:30
Vj-L
2c7a327c6b
Issue #3094 (#4500)
* Added the named volumes for improvement

* Mapped the named volumes

* Updated the Named Volumes

* Updated the Volumes on top level

* Added the named volumes
2022-05-03 14:19:31 +05:30
Akash Jain
0f63a4063e
Enable custom oidc provider name env override. (#4642) 2022-05-02 22:38:11 -07:00
Akash Jain
792c67a8f1
Arguments to provide ingestion depenedency. defaults to 'all' (#4613) 2022-05-02 16:09:44 +05:30
Vivek Ratnavel Subramanian
12a4936671
Fix ingestion bot principals in docker-compose (#4507) 2022-04-26 15:33:16 -07:00
Akash Jain
0beeef3f97
pluralize list input environment varialbles (#4473) 2022-04-25 17:11:02 +05:30
Akash Jain
54162b7c6d
fix: List Configuration from Environment Variable for Authorizer and Authentication Configuration (#4305)
* ability to provide list from env in yaml config

* remove security config file as we do not maintain
2022-04-21 11:29:26 +05:30
Akash Jain
018cd0aec6
fix: Google SSO Audience default value (#4285)
* Update openmetadata.yaml

* updated audience default value
2022-04-20 15:48:13 +02:00
Akash Jain
7958e81918
server host api url env for airflow configuration (#4261) 2022-04-19 23:24:48 -07:00
Akash Jain
ff26b6d93e
fix: OpenMetadata Airflow config with SSO (#4197) 2022-04-19 10:39:50 -07:00
Ayush Shah
351db56687
Added fix for Sample_Usage not being ingested in Local docker (#3650) 2022-03-27 22:38:13 -07:00
Vivek Ratnavel Subramanian
f0927ecebb
Fix #3236: Add support for secure docker compose with an env file (#3241)
* Fix #3236: Add support for secure docker compose with an env file
2022-03-08 07:05:31 -08:00
Pere Miquel Brull
76f4ccd590
Fix #2735 - Simplify Airflow properties extraction (#2749)
* Simplify lineage properties extraction

* Add network name

* Format
2022-02-14 16:53:42 +01:00
Akash Jain
48c1ad63c4
fix: ElasticSearch and Ingestion connectivity from openmetadata post soft link (#2470)
* soft linked openmetadata.yaml

* quick fix for ingestion and elasticsearch connect from server
2022-01-27 13:55:54 +05:30
Pere Miquel Brull
94ecb1e030
Map ports to localhost (#2078) 2022-01-06 14:32:31 +01:00