250 Commits

Author SHA1 Message Date
Mayur Singal
a5e6acbed4
Fix Dockerfile Path (#10840) 2023-03-30 10:39:14 +00:00
Mayur Singal
54b635dd60
1.0.0-alpha version upgrade (#10826) 2023-03-29 23:24:39 +05:30
Nahuel
551df24d35
Fix: pass correct env var to the ingestion docker image (#10598) 2023-03-16 12:28:23 +01:00
Sriharsha Chintalapani
de10cd254b
New docker structure (#10438)
* Adding the different docker-compose file openmetadata and ingestion

* Added two different env files for mysql and postgres

* Updated the docker file path

* Updated the path of docker folder structure

* Fix docker

* Updating the PR with necessary changes required

---------

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
2023-03-06 08:58:53 -08:00
Nahuel
247016307d
Fix#8648: Mask sensitive info from API responses (#10307)
* Mask sensitive info from API responses

* Rename converter classes

* Add missing Java classes from JSON schemas and class converters

* Update test service connection schema

* Update datalakeConnection JSON schema and fix some tests

* Fix AlertsRuleEvaluatorResourceTest and minor error in run_local_docker.sh

* Fix Pipeline and Database service tests

* Minor refactor

* Fix CsvUtilTest

* Fix EventMonitorFactoryTest

* Fix CloudWatchEventMonitorTest

* Update datalake metadata

* Update bigquery metadata

* Fix test connection functionality

* Fix OMeta service api test

* Update gcsValues title and revert changes in GH actions

* Mask sensitive enabled by default for local docker

* Add missing tests

* Address PR comments

* Address PR comments

* fix ui breaks on gcsValues.json

* Address PR comments

* Minor refactor

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-03-03 18:10:01 +00:00
Sachin Chaurasiya
4ce038557e
feat(ui)#10274: UI - allow to configure OM logo (#10316)
* feat(ui)#10274: UI - allow to configure OM logo

* chore: use logo config to render the brand and login page logo

* fix : cy tests

* doc: update the doc

* chore: remove check for filePath logo location type

* test: add unit test

* - removed FilePath
- Added configs to docker

* address comment

* remove config from test yaml

* update logo aspect ratio

* remove type

* chore: update ui logic as per backend config

* update doc

* address comments

* Revert "address comments"

This reverts commit c4f9da88918343cab08cd5e603cba29bd05b2ec9.

* address comments

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-03-01 14:09:44 +05:30
Ayush Shah
785142d86a
Add policy tags from Bigquery (#10189) 2023-02-20 19:13:45 +00:00
Pere Miquel Brull
1835de8bf2
Group airflowConfiguration into pipelineService config (#10120)
Group airflowConfiguration into pipelineService config (#10120)
2023-02-07 16:30:42 +01:00
Suresh Srinivas
1990ac56d9
10041 Part1 - Refactor and clean up System APIs (#10042)
* 10041 Part1 - Refactor and clean up System APIs

* Update the config resource endpoint url
2023-02-03 16:25:40 -08:00
Nahuel
b9a3c06104
Bump main branch to version 1.0.0 (#10040)
* Bump to version 0.13.2

* Bump mvn projects to 1.0.0-SNAPSHOT

* Bump python projects to 1.0.0.dev0

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-02-02 12:56:14 +01:00
Sriharsha Chintalapani
9004434c18
Fix #9752: docker openmetadata_ingestion should only depend on db (#9753)
* Fix #9752: docker openmetadata_ingestion should only depend on db

* Fix the typo

* Adding service_started for local docker

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
2023-01-18 08:45:15 +01:00
Pere Miquel Brull
e3ae3a8a33
[WIP] - Add Scarf gateway (#9652)
* Update ingestion Readme with scarf

* Test scarf docker

* Update ingestion Readme with scarf

* Test scarf docker

* Add docker domain
2023-01-11 09:33:09 +01:00
Nahuel
2c43ebba6f
Fix#9448: Add ES volumes (#9506)
* Add ES volumes

* Fix run_local_docker script

* Fix error run_local_docker script

* Update Es volumes in docker-compose files
2022-12-23 17:33:30 +01:00
Pere Miquel Brull
8806244348
Removing ES volumes (#9503)
* Fix ES volumes permissions

* Remove ES volumes
2022-12-22 18:02:09 +01:00
Vj-L
f3855a1909
Added the default volumes for ElasticSearch (#9452)
* Added the default volumes for ElasticSearch

* Added docs for ES Volumes

* Updated as per the comments

* Updated as per the comments

* Added the docs for bare-metal upgrade

* Apply suggestions from code review

* Update docker/local-metadata/docker-compose.yml

* Pylint check addressed

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: Nahuel <nahuel@getcollate.io>
2022-12-21 18:29:49 +00:00
Pere Miquel Brull
ddf2adf52c
Add port mapping for MySQL (#9436) 2022-12-20 11:13:17 -08:00
Nahuel
a2b34dd0f4
Fix: Update Ingestion docker images and fix python libraries dependencies (#9342)
* Update Ingestion docker images and fix python libraries dependencies

* Install also apache-airlfow-providers-http
2022-12-16 14:46:25 +00:00
Nahuel
e998ecc407
Deprecate configurations for auth provider configuration on Airflow (#8896)
* Deprecate configurations for auth provider configuration on Airflow

* Fix code after merging

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2022-12-15 12:28:46 -08:00
Ayush Shah
231b28fc87
Fix 7278: add test connection (#9196)
* Add Test Connection to Profiler

* remove Code Lint

* Fix AttributeError

* Fix Pytest

* Fix Bigquery Partition tests

* Fix Lint
2022-12-10 17:16:28 +01:00
Vj-L
0a570c8585
Fixed the mounting of volumes (#8963)
* Fixed the mounting of volumes

* Update docker/local-metadata/docker-compose-postgres.yml

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Update docker/metadata/docker-compose-postgres.yml

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>

* Added permission step

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
2022-11-23 14:37:53 +05:30
Ayush Shah
b25336a55c
Add Generation of Pydantic Models via script (#8923) 2022-11-22 11:19:22 +05:30
Vj-L
2c055641be
Added host volumes for Database service (#8722)
* Added host volumes for Database service

* Formated the python file

* Done changes according to the comments on PR

* Changes done as per the comments on Draft PR

* Fix PyLint

* Changes done as per the comments on Draft PR

* Updated the path of volume for Database

* Added cleanup state in workflows

* Added cleanup state in workflows

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-11-18 14:05:33 +01:00
Ayush Shah
2d7d89754c
Remove Click and Add ArgParse (#8182) 2022-10-31 18:12:26 +05:30
Rahul Surwade
a286a0fcbc
Removed unnecessary docker-compose files (#8085)
* Delete docker-compose-dev-postgres.yml

* Delete docker-compose-dev.yml

* Update openmetadata-server.md

* Update openmetadata-server.md

* Update openmetadata-server.md
2022-10-31 12:50:12 +05:30
Pere Miquel Brull
339f123f92
Fix #7477 - Paginate Airflow logs (#8145)
* Paginate last dag logs

* clean logic

* Prep pagination

* Prep pagination

* Format

* Update docker/local-metadata/docker-compose.yml

Co-authored-by: Nahuel <nahuel@getcollate.io>

* UI : Add pagination support for airflow logs

* Addressing review comments

Co-authored-by: Nahuel <nahuel@getcollate.io>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-10-21 17:32:56 +05:30
Vj-L
e1b4229074
Fix/env variable (#8199)
* Updated env variable for heap

* Updated env variable for heap

* Updated the env variables

* Updated the env variables

* Updated the env variables

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-10-19 11:00:11 +05:30
Pere Miquel Brull
19b2158d56
Use postgres 14 (#8207) 2022-10-18 18:29:25 +05:30
Sam Firke
58b86a24bc
Remove inconsistent space in docker-compose files (#8138)
* remove extra space for consistency
2022-10-13 15:33:22 -07:00
Sriharsha Chintalapani
65ee573a7d
Fix HEAP OPTS param (#8094) 2022-10-11 16:30:09 -07:00
Vj-L
0926c754aa
Updated env variable for heap (#8067)
* Updated env variable for heap

* Updated env variable for heap

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-10-11 19:29:22 +05:30
Pere Miquel Brull
339abc5bf3
Prepare Ingestion Base Docker image (#8065)
Prepare Ingestion Base Docker image (#8065)
2022-10-11 07:50:49 +02:00
Sriharsha Chintalapani
801012d2a6
Add remote debug port (#8045)
* Add remote debug port

* Add remote debug port

* Add remote debug port

* Add remote debug port
2022-10-11 11:13:15 +05:30
kmalki
69ddc7081c
Add environment variables in docker compose for elasticsearch (#8026) 2022-10-08 23:51:59 +05:30
Akash Jain
1e31245bbe
fix: Remove Lineage Backend Configs (#7836) 2022-10-01 11:23:24 +05:30
Akash Jain
c4b0bb0538
fix path for airflow volumes (#7827) 2022-09-30 18:20:38 +05:30
Vj-L
489d506e3a
Removed the volume for Database (#7798)
* Removed the volume for Database

* Removed the volume for Database

* Added named volume

* Added named volume

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-09-29 21:44:12 +05:30
Vj-L
6cfd16925a
Added default volumes (#7741)
* Added default volumes

* Ran make py_format for formatting

* Added sample data in mount directories

* Updated as per the comments

* Done changes for --ingest_sample_data flag

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-09-29 14:53:55 +05:30
Akash Jain
b3d340e403
Fix: env variables with basic auth changes for docker compose (#7693) 2022-09-23 23:52:28 +05:30
Ashok Mokshagundam
f62f29c806
Update mysql-script.sql (#7652)
updating database airflow_db with proper character set and collate
2022-09-22 14:41:58 +05:30
Akash Jain
08166ac239
fix typo error (#7542) 2022-09-19 15:21:49 +05:30
Akash Jain
fc8312c5fb
feat: Refactor openmetadata/ingestion docker image (#7456)
* centralize openmetadata/ingestion docker image

* update volume mappings; fix ingestion tag!

* fix PR comments

* fix cypress mysql tests!
2022-09-19 09:20:54 +05:30
Pere Miquel Brull
6700f169ab
Fix #7165 - Handle SSL certificates on ometa client (#7502)
Fix #7165 - Handle SSL certificates on ometa client (#7502)
2022-09-16 09:39:27 +02:00
Sriharsha Chintalapani
656b50dd3a
Fix #7469: Refactor OpenMetadata code modules (#7474) 2022-09-14 23:14:02 -07:00
Akash Jain
2162e51e96
fix: airflow docker image python dependency issues (#7412) 2022-09-13 11:33:15 +02:00
Vivek Ratnavel Subramanian
027f317725
Add port mapping for postgres container (#7354)
* Add port mapping for postgres container

* Fix DB_SCHEME
2022-09-08 17:24:47 -07:00
Akash Jain
f2fc45ea41
feat: prepare-for-next-release-0-13-0 (#7348) 2022-09-09 01:25:41 +05:30
Akash Jain
72374f449e
dockerfile to support airflow 2.3.3 (#7302) 2022-09-08 12:48:07 +05:30
Nahuel
7601181052
Fix docker-compose files (#7218) 2022-09-05 11:46:00 +02:00
Nahuel
ed58c27f76
Remove unused property extra_hosts from docker-compose files (#7096)
* Remove unused properties from docker-compose files

* Minor fix in run_local_docker script
2022-08-31 21:30:24 +02:00
Teddy
811f640a18
Fixes #6891 by implementing support for VARIANT DType (#7084) 2022-08-31 19:01:00 +02:00