2200 Commits

Author SHA1 Message Date
Mohit Yadav
ef2634970f
Fix Classification Migration for missing fqn in json causing failures (#12913) 2023-08-18 14:51:50 +05:30
Onkar Ravgan
5b47fd4acf
Added source url to entities (#12901)
* Added source url to entites

* added support to create and update sourceUrl

* fixed pytests

---------

Co-authored-by: 07Himank <himank07mehta@gmail.com>
2023-08-18 10:17:38 +02:00
Sriharsha Chintalapani
524e611edc
Deprecate Flyway and use Native migrations (#12865)
* Add ascii_bin for fqnhash, Add Constraints on timeseries

* Attemp of moving migrations to files

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Move upstream migrations to new path
2023-08-17 14:39:05 -07:00
Mohit Yadav
801d07289c
Added LogBack File (#12897) 2023-08-17 12:20:09 +05:30
Teddy
751c96e437
Fixes Issue #12297 - Added upsert logic for DI (#12466)
* fix: enforce DB upsert on DI ingestion

* fix: ran python linting

* feat: fix conflict and added BindFQN while removing hash

* feat: update aggregation from day to minute

* feat: ran python linting
2023-08-17 08:07:44 +02:00
07Himank
ad3871b3bc
Added source url to search indexes and missing entites (#12773)
* added sourceUrl to missing entites and added sourceUrl to search  indexes

* edited desc in json files

* edited desc in json files

* working on adding api for getting entity from sourceUrl

* added API for getting Entity from sourceUrl

* added api to fetch entity based on sourceUrl

* remove desc

* working on adding alias
2023-08-16 15:56:27 +05:30
Ayush Shah
d7606dd4b8
Fix Ldap (#12872) 2023-08-14 20:15:13 +05:30
Teddy
8e4388c35e
Issue 11345 (#12859)
* feat: added serviceName dimension to entity report

* feat: fix python test
2023-08-14 08:05:14 +02:00
Suresh Srinivas
e3c0539301
12142 Task 4 - Add support for adding entities as part of Data Product (#12867) 2023-08-12 15:45:56 -07:00
Mohit Yadav
c13892d5c2
test suite listing fixes (#12864) 2023-08-12 08:22:46 +05:30
Sriharsha Chintalapani
b6eb51d503
Fix Cypress Test failures for search (#12850) 2023-08-10 23:42:19 -07:00
Sriharsha Chintalapani
192a2ec5fe
Fix #12346: Allow updating Query used by other tables while editing … (#12847)
* Fix #12346: Allow updating Query used by other tables  while editing query

* Fix #12346: Allow updating Query used by other tables  while editing query

---------

Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-08-10 23:20:13 -07:00
Sriharsha Chintalapani
7aaf654f01
Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch (#12782)
* Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch

* Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch

* Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch

* Rebase fixes with main

* Add Sample Data

* lint fix

* remove unused import

* Fix service count test

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-08-10 16:47:37 -07:00
Sriharsha Chintalapani
da0c79cf78
Fix bulk soft-delete or restore from search indexes (#12807)
* Fix bulk soft-delete or restore from search indexes

* Fix bulk soft-delete or restore from search indexes

* Add entityRestored to list events api
2023-08-09 23:24:10 -07:00
Pere Miquel Brull
d8e288626c
Storage connection naming, ADLS service and namespace packaging (#12810)
* Update storage connection naming

* Update storage services and remove typing of generic service

* Remove inits for namespace package

* Add test connection

* Revert "Remove inits for namespace package"

This reverts commit 89f1ae9ea8a633593776f51caf2586d2110d4636.

* Add enum

* Remove init to allow namespace packaging in some modules

* Remove init to allow namespace packaging in some modules

* Move examples and fix test

* Format

* Linting
2023-08-10 07:32:58 +02:00
Sriharsha Chintalapani
a60af7c019
Fix #10164: Check any name voilations during the import dryrun (#12800)
* Fix #10164: Check any name voilations during the import dryrun

* Fix test

* fix test

* Fix checkstyle
2023-08-09 18:50:50 -07:00
Suresh Srinivas
8acc1f61ec
12803.part2 - Display names are not updated in entity references (#12829) 2023-08-09 17:17:55 -07:00
Ayush Shah
d09d0689ec
Fix Test Case (#12818) 2023-08-09 18:41:25 +05:30
Suresh Srinivas
340a029807
Fixes #12802 Add support for generic entity cache (#12803)
* Add entity cache in EntityRepository for all the entities

* Set entity fields only when it is not already set

* Clone entities from the cache

* Remove SubjectCache

* Remove RoleCache

* Remove PolicyCache

* Remove TagLabelCache

* Deleting lingering files

* Test failure fixes

* Remove getEntityReference methods from EntityDAO

* Move quoteName logic to EntityRepository

* Fix test failures
2023-08-08 16:42:54 -07:00
Suresh Srinivas
0e08d6d412
Turn JSONProcessingException into Runtime Exception (#12765) 2023-08-06 18:16:04 -07:00
Mohit Yadav
505a70982a
test suite listing fixes (#12758) 2023-08-04 16:53:51 +05:30
Suresh Srinivas
dc819d0369
Fixes #12741 Getting entity by name does not return inherited properties (#12742) 2023-08-04 03:46:06 -07:00
07Himank
cda004e35e
fixed issue for when classification has . in name the termCount was n… (#12729)
* fixed issue for when classification has . in name the termCount was not getting

* fix url encoding issues

* changes as per comments and sonar fix

* changes as per comments

* fixed issue for when classification has . in name and testcase and pytest issue

* checkstyle

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2023-08-04 11:37:17 +05:30
Suresh Srinivas
c5e7a63fbb
Fixes #11305 Delete tasks corresponding to an entity when the entity is deleted (#12740) 2023-08-03 18:27:01 -07:00
07Himank
00885380b6
fixed displayName issue for owner (#12724) 2023-08-03 19:09:35 +05:30
Mohit Yadav
daf6012374
Fix Failing TestSuite Migration (#12730) 2023-08-03 17:06:30 +05:30
Sriharsha Chintalapani
ef6e281f05
separate mysql/postgres tests (#12703)
* separate mysql/postgres tests

* separate mysql/postgres tests

* separate mysql/postgres tests

* separate mysql/postgres tests

* Update pom.xml
2023-08-02 05:59:09 -07:00
Mohit Yadav
191cee2da0
Fix Test Suite Migrations (#12697)
* Fix Test Suite Migrations

* Fix Test Suite Migrations

* Fix Test
2023-08-02 16:58:22 +05:30
Esmaeil Chitgar
744a3d403f
add automate exception message for invalid enum value as api request parameter (#12400)
* add exception Message for invalid enum value

when enum value is incorrect in api call, this method generates the exactly message as we expect. for example the Include.java in databaseServiceResource.java list() api.

* add an example for invalid enum value as unit test for databaseServiceResource class.

A unit test example that shows how to use invalidEnumValue() method instead of traditional way i.e. writing plain string.

* apply google-formatter

* Merge branch 'EsmaeilChitgar-AddExceptionMessageForInvalidEnumValue' of https://github.com/EsmaeilChitgar/OpenMetadata into EsmaeilChitgar-AddExceptionMessageForInvalidEnumValue

* fix reported issues in pull request

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-08-01 22:52:48 -07:00
Sriharsha Chintalapani
43e06357a2
Fix #8208: OpenMetadata server supports Postgres 15 (#12370)
* Fix #8208: OpenMetadata server supports Postgres 15

* fix: PSQL 15 Initial scripts to make database owners for permissions

* Update to generic base image tag

* fix: type for airflow username

---------

Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
2023-08-02 11:13:28 +05:30
Ashish Gupta
de06d50e7f
fix(ui): ui breaking as url was not encoded. (#12643)
* fix ui breaking because of url having percent in it

* added support for % in about

* fix test cases

* checkstyle

* url fix

* fix storage url

* encoded topic, mlmodel and pipeline entity

---------

Co-authored-by: 07Himank <himank07mehta@gmail.com>
2023-08-01 14:12:47 +05:30
Sriharsha Chintalapani
878fda47a1
Fix tests logging; do not initialize the search client if the search … (#12681)
* Fix tests logging; do not initialize the search client if the search config is not present

* Fix logging and parallelization defaults 1C

* Fix logging and parallelization defaults 1C

* Fix logging and parallelization defaults 1C

* Fix logging and parallelization defaults 1C

* Fix logging and parallelization defaults 1C

* Fix logging and parallelization defaults 1C
2023-07-31 23:35:08 -07:00
Chirag Madlani
098625c549
feat(server): description support for team hierarchy (#12638) 2023-08-01 11:25:42 +05:30
Suresh Srinivas
19e223eabd
Fix exceptions thrown as reflections scan non java classes (#12680) 2023-07-31 17:33:47 -07:00
Suresh Srinivas
d968304853
Add BindFQN jdbi annotation to change fqn to fqnHash (#12658) 2023-07-31 11:51:40 -07:00
Mohit Yadav
361629fffd
no batch in migration updates (#12660) 2023-07-31 18:55:09 +05:30
07Himank
a95d19eaeb
Refactor Es OS code (#12572)
* refactor Es OS code

* refactoring Es OS code

* hardcore RUN_ELASTIC_SEARCH_TESTCASES value to true

* updated the size of maven-surefire from 1024 to 2048

* fixed blank service type issue for aggregation

* debugging pr

* debugging pr

* added configuration to openmetadata-secure-test.yaml

* debugging ci failure

* debugging ci failure

* debugging ci

* debugging ci issue

* added container to be created single time

* added container to start just once

* removed elasticSearchContainer = null

* removed elasticSearchContainer = null

* removed elasticsearchcontainer.stop

* reverted ElasticSearchContainer Changes

* revert extra memory added for jvm added

* reverted JUnit.run.xml

* resolved conflicts

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2023-07-31 12:07:10 +05:30
Suresh Srinivas
786dde170e
Fixes #12651 Organization policy changes are overwritten during server restart (#12652) 2023-07-29 11:42:49 -07:00
Suresh Srinivas
b7e3242911
Make LoadingCache members final variables (#12648)
* Make LoadingCache members final variables

* Simplify singleton classes

* Reuse Entity fields instead of declaring them again
2023-07-29 11:42:28 -07:00
Suresh Srinivas
cb9e5d8b6f
Simplify getting from and to entity relationships (#12627) 2023-07-28 04:59:16 -07:00
Sriharsha Chintalapani
50fb66c8dc
Fix #12610: adminUsers lookup should be quoted, do not allow bot users s to be promoted to admin user via adminUsers config (#12611)
* Fix #12610: adminUsers lookup should be quoted, do not allow bot users to be promoted to admin user via adminUsers config

* Fix #12610: adminUsers lookup should be quoted, do not allow bot users to be promoted to admin user via adminUsers config

* remove changes to yaml

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2023-07-27 23:34:48 -07:00
Suresh Srinivas
8830d1707d
Initialize EntityResoruceTest support flags automatically (#12513) 2023-07-27 20:42:10 -07:00
mohitdeuex
3e17c984eb added list to ignore migration if already done for 2023-07-27 12:09:01 +05:30
Pere Miquel Brull
6773541d15
[1.1.1] - Bump size for FQN (#12092)
* Bump size for FQN

* Bump table entityName size

* Bump table entityName size

* Fix table resource tests

* Remove pattern from fqn

* Remove pattern from fqn

* Remove pattern from fqn

* Generalize get_by_name in ometa client

* Generalize get_by_name in ometa client

* Format

* Fix test suite

* Remove limit from FQN max size

* Remove limit from FQN max size

* Add sample data

* Update lint names

* Add more sample data

* Bump column name size

* 1024 max FQN length

* 1024 max FQN length

* 1024 max FQN length

* Bump FQN
2023-07-26 12:36:42 -07:00
Sriharsha Chintalapani
67b68a70c9
Fix #7712: Align AuthProvider values in the JSON schema definition (#11508)
* Fix #7712: Align AuthProvider values in the JSON schema definition

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Handle new providers

* Lint

* Fix #7712: Align AuthProvider values in the JSON schema definition

* Handle new providers

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Lint

* fix rebase

* Fix test

* Fix test

* Fix Usage of Auth Provider in User Resource

* remove unused function

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2023-07-26 11:34:58 -07:00
mohitdeuex
5af1bc4a55 Update SQL queries for reading and comming migration 2023-07-26 17:48:46 +05:30
mohitdeuex
410cb984a2 Merge remote-tracking branch 'origin/main' 2023-07-26 16:41:02 +05:30
mohitdeuex
8ab4072504 Change ignoreFileChksum to force 2023-07-26 16:40:52 +05:30
Onkar Ravgan
0f8ebf4afe
Fixed Tableau-Bigquery lineage issue (#12570) 2023-07-26 16:07:48 +05:30
Shailesh Parmar
f36ed5f204
fixed: ui: add support of azure config in dbt form #12589 (#12594)
* fixed: ui: add support of azure config in dbt form #12589

* translation-sync

* fixed azure option selection in edit mode
2023-07-26 14:32:22 +05:30