27 Commits

Author SHA1 Message Date
Pere Miquel Brull
a4521fd664
Fix #6562 - Sources have their own package (#9521)
Fix #6562 - Sources have their own package (#9521)
2022-12-27 15:00:22 +01:00
Teddy
9e01fe0636
feat(CLI): added migration logic cmd (#9437) 2022-12-21 07:55:18 +00:00
Nahuel
b14a3bc6ca
Review sensitive data not marked as password in the JSON schemas (#8818) 2022-11-18 11:35:08 +01:00
Pere Miquel Brull
b203a92565
Add config file env var loading tests (#8799)
* Add env var load test

* Add env var load test
2022-11-17 16:52:42 +00:00
Nahuel
91d0460b27
Fix#8577: Refactor part of the secrets manager implementation (#8617)
* Removed part of the secrets manager implementation in the server side

* Removed part of the secrets manager implementation in the openmetadata python library side

* Remove deprecated test

* Address pylint checks

* Address new pylint checks

* Address PR comments

* Fix import on airflows apis

* Clear singleton instances for running Ometa secrets manager test
2022-11-11 09:59:15 +01:00
Suresh Srinivas
229b56e7b3
Fixes #8236 - Initialize bots from JSON data files (#8304) 2022-10-21 08:38:52 -07:00
Ashish Gupta
8ddab79dc1
UI :- Remove rc-tree and react-table dependencies (#8049)
* Remove rc-tree and react-table dependencies

* fix unit test issue

* File Formatted

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-10-11 15:42:58 +05:30
Nahuel
5c499d2a7a
Add CLI E2E tests for MySQL (#8041)
* Add CLI E2E tests for MySQL

* Fix setup.py and pylint

* Add missing doc and update code after pylint refactor
2022-10-10 11:36:20 +02:00
Teddy
63b5233fd3
Added util command to overwrite dag_generated_config path in DAG file (#7831)
* Added util command to overwrite dag_generated_config path in DAG file

* Added logic to only write file on change
2022-10-03 09:24:47 +02:00
Nahuel
593ca3a4a0
Fix bot creation and edition logic (#7796)
* Fix bot creation and edition logic

* Fix minor error creating user

* Fix failing test

* Minor fixes

* Add missing tests for new flows

* Fix put_failIfBotHasARelationshipToAnotherUser test

* Changes after manual testing

* Move where auth_provider is retrieved in the secret manager
2022-10-01 11:25:56 +02:00
Nahuel
df909703eb
Fix#6615: Fix failing test still in CI (#7725)
* Fix failing test still some tests are failing

* Fix bug in service entity repository

* Fix all tests except Ingestion Pipeline integration tests

* Fix Ingestion Pipeline integration tests

* Fix failing Python tests
2022-09-26 20:39:42 +02:00
Onkar Ravgan
35efd49256
Added control for DBT descriptions (#7653)
* Added control for DBT descriptions

* Fixed tests

* Added UI changes

* fixed maven ci tests

* Java formatting changes

* ui review fixes

* Fixed pytests

* Fixed python integration tests

* fixed airflow tests

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-09-26 16:19:47 +05:30
Teddy
f2bf5194bb
Fixes #7623 -- Added logic to encode and decode entityLink (#7670)
* Encode entityLink string when processing request

* Added logic to decode column type from entityLink

* mvn code formating

* Extracted unquote step into its own function
2022-09-23 09:42:33 +02:00
Teddy
c34281251f
Fixes #7509 - implement import path migration script (#7515)
* Added script to migarte import

* Added tests

* Updated doc to include info about import migration

* renamed test resource files
2022-09-22 14:43:25 +02:00
Nahuel
b829a2cbf3
Fix: Improvements on secret manager implementation (#7282)
* Change local secret manager by noop

* Update openmetadata-secure-test.yaml
2022-09-07 09:18:59 +02:00
Onkar Ravgan
9be3df4a1e
Integrated DBT tests (#7122)
* Integrated DBT tests

* Fixed py tests

* Review comments addressed

* Changed test suite name default value

* Added fixes

* update ui changes for new field

* send data along with api

* update name

* fixes for UI

* fixes for UI

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2022-09-05 22:25:31 +05:30
Nahuel
4e176fbc66
Fix#5917: Implementation of temp secret for testing connection (#6832)
* Implementation of temp secret for testing connection

* Fix tests
2022-08-22 08:43:23 +02:00
Nahuel
8691022d0f
AWS SSM secrets manager implementation on ingestion (#6805)
* Implementation of AWS SSM as secrets manager

* Remove dead code

* Minor fixes

* Allow using default credentials in AWS client

* Fixed py style

* Fixed tests imports

* Minor changes

* Add patch to failing test

* Fix how we were storing dbtConfigSource

* Address PR comments
2022-08-19 16:15:40 +02:00
Nahuel
00ce67c01e
Fix#6643: Secure in secrets manager the dbtConfigSource object (#6743)
* Secure in secrets manager the dbtConfigSource object

* Add missing tests

* Fix minor bug

* Minor code refactor
2022-08-17 10:32:43 +02:00
Nahuel
cf2cb6d531
Fix#6517: Add clusterName property to the application config yaml (#6610)
* Add cluster name in the app configuration and start using it to create secrets id

* Update secret manager client in openmetadata for using default auth provider

* Add missing property in test config file
2022-08-09 09:00:43 +02:00
Mayur Singal
12f81c43f8
ES pipeline minor changes (#6565) 2022-08-04 15:22:09 +02:00
Nahuel
a878aa911c
Fix#6212: Retrieve connection params from secret manager in CLI commands (#6441)
* Retrieve connection params from secret manager for database connectors

* Retrieve connection params from secret manager for all services except database connectors

* Stop retrieving connection from SM in Airflow rest plugin

* Retrieve connection params from secret manager for dashboard services

* Retrieve connection params when initializing Workflow/ProfilerWorkflow objects

* Align services topologies + comment changes in topology runner

* Address SonarCloud bug detected

* Update database service topology

* Address PR comments

* Address PR comments

* Address PR comments
2022-08-02 09:13:46 +02:00
Nahuel
fdfdaa14aa
Fix#5921: Implentation for retrieving auth provider config from Secret Manager (#6330)
* Implentation for retrieving auth provider config from Secret Manager

* Address PR comments

* Address code smells from SonarCloud

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-07-26 15:42:40 +02:00
Pere Miquel Brull
7560635114
Fix #6280 - Bump DeltaLake version, tests and docs (#6307)
Fix #6280 - Bump DeltaLake version, tests and docs (#6307)
2022-07-24 18:49:15 +02:00
Francisco J. Jurado Moreno
756dae5605
[TASK-6241] Cache ES ingestion when ingesting usage (#6276)
* Cache ingestion

* Uncomment secrets manager

* Fix experiment stuff

* Fix style

* Add LRU cache

* Add tests

* Fix code smell
2022-07-22 23:33:22 +02:00
Nahuel
4d4a2fc2cf
FIX#5916: Only admin user retrieves connection params when external Secrets Manager is configured (#6228)
* Implementation for stop sending connection credentials when user is BOT

* Change way we add the connection to the service in the Secret Manager

* Services connection is not required as we want to stop returning it when SM is configured
2022-07-22 14:02:29 -07:00
Nahuel
0602e8c894
Fix#5924: Implement secret manager for Airflow REST Plugin (#6128)
Fix#5924: Implement secret manager for Airflow REST Plugin (#6128)
2022-07-19 14:51:44 +02:00