1565 Commits

Author SHA1 Message Date
Suresh Srinivas
a313dc19f3
11587 Task 1 - Simplify field inheritance from parent to child entities (#11588) 2023-07-04 10:00:16 -07:00
mohitdeuex
71fed53e33 Update query to use for list to fqn 2023-07-04 20:26:22 +05:30
Pere Miquel Brull
16280ffd1e
Fix db filter name (#12271) 2023-07-04 09:50:34 +02:00
Suresh Srinivas
97140e13fd
12142.task2 - Add domain and data product properties to entities (#12255) 2023-07-03 15:42:35 -07:00
mohitdeuex
2bb5c970f9 Remove quoted string postgres 2023-07-04 00:44:03 +05:30
mohitdeuex
89b0de4eac Migration error in listing by Name because of duplicate name 2023-07-03 22:57:36 +05:30
mohitdeuex
989914f84e Add server migration validation 2023-07-03 20:15:13 +05:30
mohitdeuex
e5d761445f Make read size as 1000 2023-07-03 16:38:17 +05:30
mohitdeuex
4a2e0d6c19 Use Batch for commit and remove transactions 2023-07-03 16:34:50 +05:30
07Himank
245303c8c8
solved table api throw 404 error (#12261) 2023-07-03 12:38:06 +02:00
Sriharsha Chintalapani
bdb438d12d
Fix migrations to handle entities with mal-formed FQN due to . in their name (#12217) 2023-07-03 15:48:40 +05:30
Suresh Srinivas
4d4993effc
12142.task1 - Add support for Domains (#12143) 2023-07-02 11:32:52 -07:00
Sriharsha Chintalapani
63de9a160b
Fix migrations to handle entities with mal-formed FQN due to . in their name (#12249)
* Fix migrations to handle entities with mal-formed FQN due to . in their name

* Fix migrations to handle entities with mal-formed FQN due to . in their name
2023-07-01 14:04:27 -07:00
mohitdeuex
60859791ff Add Logging in TableInitializer 2023-07-01 02:13:52 +05:30
Pere Miquel Brull
222a8f8984
[Docs] - SSO updates & Connectors workflow config (#12241)
* Rename docs and clean SSO

* Add connector partials

* Add connector partials

* Rename path
2023-06-30 12:25:11 +02:00
mohitdeuex
f51781c32d Missing try catch for es-migrate option 2023-06-30 11:58:21 +05:30
07Himank
7244c3f530
changed enum from Elastic/Open Search to elastic/open search (#12237) 2023-06-29 20:13:49 -07:00
Suresh Srinivas
b88f5519ad
Move storing common relationships to base EntityRepository (#12164) 2023-06-29 19:54:05 -07:00
Mohit Yadav
6410fcbea4
Fix Alert Intermittent Issues for some entities (#12189)
* Fix Alert Intermittent Issues for some entities

* Fix Failing test for disabled publisher

* Fix Failing test due to deleted entity

* Fix Tests

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-06-29 19:00:54 -07:00
MihirR123
41f6945cfc
Fixes #11821 Incorrect swagger documentation for Pipelines / Add status data (#12216) 2023-06-29 12:15:54 -07:00
Mohit Yadav
a8c6f8bce0
Revert "Test suite index task(addded test_suite_search_index) (#12184)" (#12233)
This reverts commit a7ef0a82b1df4ec103fcbf8f2573ed907483060f.
2023-06-29 20:18:49 +05:30
07Himank
70b76a34c1
updated the aggregate from owner.fullyQualifiedName.keyword to owner.… (#12219)
* updated the aggregate from owner.fullyQualifiedName.keyword to owner.displayName.keyword

* fix owner search for filters

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-06-29 17:06:32 +05:30
07Himank
405896e009
updated es to 7.16.3 (#12218) 2023-06-29 16:03:51 +05:30
Teddy
2963fd1137
Move table list empty test suite to dao (#12212)
* fix: moved testSuite filter in table list to DAO

* fix: fixed tests as we won't filter out test suite with no test cases

* fix: handled null queryParam

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-06-29 15:53:07 +05:30
07Himank
a7ef0a82b1
Test suite index task(addded test_suite_search_index) (#12184)
* added search test case index

* checkstyle

* working on adding test suite index

* addes test_suite_search_index

* removed comments

* improvemtn

* fixes

* added test_suite_index_mapping for jp and zh languages

* check style
2023-06-28 23:08:10 -07:00
Sriharsha Chintalapani
d346101249
Update fqn size (#12188)
* Fix #10198: Filter tags and classifications if the classification is disabled

* Fix #10198: Fix tests

* Fix #10198: Fix tests

* Fix #10198: Fix Postgres tests

* Fix #10198: Fix Postgres tests

* Fix fqnHash column size to be 768

* Fix Failing test due to primary_key size limit

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-06-29 00:31:10 +05:30
Teddy
37a7cf99b6
fix: test defintion migration for parameter definition changes (#12207) 2023-06-28 16:02:49 +00:00
07Himank
88abdc0a98
Added owner aggregations (#12205)
* added owner aggregation and ignore setting query_filter if its empty

* added owner aggregation and ignore setting query_filter if its empty

* style
2023-06-28 07:07:21 -07:00
07Himank
569539f280
updated elastic search version from 7.16.0 to 7.16.1 (#12203)
* updated elastic search version from 7.16.0 to 7.16.1

* dummy commit

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-06-28 07:05:00 -07:00
Sriharsha Chintalapani
47440849bd
Fix #10198: Filter tags and classifications if the classification is disabled (#12144)
* Fix #10198: Filter tags and classifications if the classification is disabled

* Fix #10198: Fix tests

* Fix #10198: Fix tests

* Fix #10198: Fix Postgres tests

* Fix #10198: Fix Postgres tests

* Fix #12083 - Backoff retry when polling for pipeline service client status (#12105)

* Fix #12083

* Fix #12083

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>

* Databricks Support Table Constraints (#12138)

* Databricks Support Table Constraints

* pylint fix

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Fix #11659: Add support for filter patterns in dbt workflow (#12063)

* Move Top Level Imports (#12145)

* interfaces for db migrations (#12057)

* interfaces for db migrations

* Add Data Migration for entities to 1.1

* Migration for FQN hash

* Added Tag Usage And Fixed some transaction issue in testing

* Added SERVER_CHANGE_LOG validations

* Update Path for files

* Added Postgres Migration

* Test Suite Migration [WIP]

* remove ingestion pipeline

* moved Migration to Table Initializer

* init Data source config in TableInitializer

* make checksum ignore configurable

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>

* Fix Migrations (#12146)

* Mask queries, Table sample data and Topic sample data (#12139)

* Filter out PII queries

* Mask topic PII

* Mask topic PII

* Update sample data test

* Format

* Moved logic to repository or PIIMasker

* Fix test

* chore(ui): update sample data api endpoint for topic

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>

* Dockerfile Update (#11066)

* Updating Dockerfile with multistage

* Updating reviewed changes

* Docker Development changes

* Docker workflow changes

* Arguments update

* Script path update

* Arguments update

* Resolving the reviewed suggestions

* Docker script update (#12115)

* Create manual.yml

* Fixing Typo

* Removing manual test github action file

* Testing IP runner

* Updating docker script for sandbox db task

* removing the ip test action

* Updating docker script for sandbox db task postgres

* Volume cleaning argument addition

* Volume cleaning argument addition

* Fix Failing Maven CI Test (#12151)

* Fix SQLLineage Test (#12152)

* Docker server workflow (#12157)

* Updating Dockerfile with multistage

* Updating reviewed changes

* Docker Development changes

* Docker workflow changes

* Arguments update

* Script path update

* Arguments update

* Resolving the reviewed suggestions

* updating workflow syntax

* remvoed ibm db arch (#12158)

* fix: resolve general feedbacks (#12153)

* fix: feedbacks

* fix: add tier as static item

* fix: feedbacks

* fix: add localization

* fix: feedbacks

* fix: add localization

* fix: breadcrumb wrapping issue

* fix: alignment issue for glossary

* fix: border issue in drawer

* ui: worked on DQ feedback part 3 (#12149)

* ui: worked on DQ feedback part 3

* updated name of test suite ingestion name
- Added description field in test case expand view
- Added functionality to update displayname of test case

* added testCase in re-indexing

* integrated permission in DQ

* fix: check on undefined bool value of clause (#12162)

* chore(ui): task feedback (#12154)

* chore(ui): task feedback

* fix copy notification info

* fix code font

* fix glue pipeline

* fix unit tests

* fix cy failures

* Update CODEOWNERS for UI code

* Address backend DQ feedback (#12150)

* fix: add jp and zh index

* fix: updated testCase tooltips

* fix: changed list testSuite permission to VEW_TEST + set owner of table for executable testSuite

* Run System Migrations on CREATE

* Update logging messages for JWT tokens (#12169)

* Update Test Case Activity Feed Messaging

* Auto tagging Windows troubleshooting (#12173)

* feat(ui): supported pagination and search in tags and term select for both table and entity level (#12155)

* supported pagination and search in tags and term select for both table and entity level

* supported pagination of tags in glossary overiew section for tags

* fix cypress and address comments

* fix cypress issue

* fixed(ui): updated tour flow and feedback (#12171)

* fixed(ui): updated tour flow and feedback

* translation-sync

* addressing comments

* feat: added view definition tab in table details page (#12172)

* Update profiler dq doc (#12182)

* doc: update documentation for profiler and dq

* doc: renamed image path from 1.0.0 to 1.1.0

* fix connection kwargs error in profiler (#12181)

* Fix Profiler Workflow infinite loop (#12185)

* Fix Profiler Workflow infinite loop

* fix: scrolling behavior feedback on mydata page (#12176)

* fix(ui): task layout update (#12175)

* fix(ui): task layout update

* fix title for edit task

* remove local for edit-task

* feat(ui): supported pagination of glossary terms in glossary overview section (#12177)

* supported pagination of glossary terms in glossay overview section

* remove unwanted try catch

* change as per comments

* Update S3 storage permissions (#12064)

* Add disabled flag to search mappings

* Revert unwanted changes

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Anuj359 <anuj.j@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2023-06-28 13:30:09 +05:30
mohitdeuex
c5b1619eb5 Update Test Case Activity Feed Messaging 2023-06-27 18:18:22 +05:30
mohitdeuex
6735db8ae7 Run System Migrations on CREATE 2023-06-27 17:18:31 +05:30
Teddy
36dd1b6b2d
Address backend DQ feedback (#12150)
* fix: add jp and zh index

* fix: updated testCase tooltips

* fix: changed list testSuite permission to VEW_TEST + set owner of table for executable testSuite
2023-06-27 11:57:05 +05:30
Mohit Yadav
80b273a559
Fix Failing Maven CI Test (#12151) 2023-06-26 16:01:22 +05:30
Pere Miquel Brull
a9135e8db5
Mask queries, Table sample data and Topic sample data (#12139)
* Filter out PII queries

* Mask topic PII

* Mask topic PII

* Update sample data test

* Format

* Moved logic to repository or PIIMasker

* Fix test

* chore(ui): update sample data api endpoint for topic

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-26 11:47:39 +02:00
Sriharsha Chintalapani
33c752d2f5
interfaces for db migrations (#12057)
* interfaces for db migrations

* Add Data Migration for entities to 1.1

* Migration for FQN hash

* Added Tag Usage And Fixed some transaction issue in testing

* Added SERVER_CHANGE_LOG validations

* Update Path for files

* Added Postgres Migration

* Test Suite Migration [WIP]

* remove ingestion pipeline

* moved Migration to Table Initializer

* init Data source config in TableInitializer

* make checksum ignore configurable

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-06-26 13:03:44 +05:30
Pere Miquel Brull
e13a5de5ae
Fix #12083 - Backoff retry when polling for pipeline service client status (#12105)
* Fix #12083

* Fix #12083

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-06-26 06:52:21 +02:00
Teddy
efec8a1019
fix: hash name of test suite + append (#12125)
* fix: hash name of test suite + append

* fix: ran java linting

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-06-25 19:14:34 -07:00
Esmaeil Chitgar
c8b3efa205
Remove duplicated line in OpenMetadata-service pom.xml (#12134)
Remove duplicated line in OpenMetadata-service pom.xml.<sonar.tests>${project.basedir}/src/test/java</sonar.tests>
2023-06-25 10:36:15 -07:00
MihirR123
0ceb856aeb
Fix sonar issues for static variable names (#12111)
Co-authored-by: Mihir Rao <mihir@MacBook-Pro-76.local>
2023-06-22 20:32:49 -07:00
Teddy
712dcf8587
fix: added field to testCase entity to list all Test Suites (#12093)
* fix: added  field to testCase entity to list all the linked test suites (executable and logical)

* fix: resolved conflicts from main

* fix: removed unecessary comment
2023-06-22 18:00:19 +02:00
07Himank
3e3bd79f89
added search for test case index (#12100)
* added search test case index

* checkstyle
2023-06-22 19:14:32 +05:30
07Himank
9355c0677d
Open search elastic search client task (#11879)
* Added diffrent clients for elasticSearch and openSearch and did the interface implementation

* removed some comments and some changes

* Add Elastic Search Test Container

* upgarded es version

* added test cases for elastic search

* added test case for, when tag is deleted cehck its relationships also get deleted

* changed package of open,elastic to openSearch and elasticSearch

* added support for skip search test case if index is not present of that entity

* Refactor bunch of files

* unnecessary semicolon

* refactor test case indexing

* fix checkstyle

* checkstyle

* added hardcoded configs and changed package name

* fixed no response from hits

* stopped elasticsearch container and some fixes

* closed elasticsearch container

* revert

* closed client after used in test

* style

* edit es config in junit

* checking if using already es version release memory for maven ci

* revert docker file changes

* remove es container for testing the ci

* commented out all test cases for elasticsearch

* fix

* creating elastic search with reusable enabled

* trying to fix ci error

* start container only if it is not running

* revert docker changes and done changes for elastic search container

* changed var name of sort_order for es

* lower the version of es client

* fixes

* all latest version of es including docker

* Use Single sort order

* Fix Keyword issue

* Immediate refresh policy for entity created

* increase sleep time

* disable test

* added reindexing logic in run_local_docker.sh

* enabled test case

---------

Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-06-22 16:55:05 +05:30
mohitdeuex
a9a830a93a Fix failine reindexing issue 2023-06-22 14:26:46 +05:30
Pere Miquel Brull
520d5e2657
Add Pipeline Service Status Job to publish to Prometheus (#12046)
* Add Pipeline Service Status Job to publish to Prometheus

* Add Pipeline Service Status Job to publish to Prometheus

* Add config for health check interval
2023-06-22 06:53:56 +02:00
Teddy
6e92dc6938
Fixes #12069 - Test Case Summary for testSuite endpoint return 0 (#12076) 2023-06-21 20:53:18 +02:00
Ayush Shah
9fa7b78d87
Issue 11686: Add IncludeOwner + scheduleInterval in Airflow (#12005) 2023-06-21 14:18:45 +05:30
Pere Miquel Brull
bfa37df298
Add platform in Pipeline Service status msg (#12041)
* Add platform in Pipeline Service status msg

* chore(ui): show message based on platform

* update the styling

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-21 07:21:19 +02:00
Pere Miquel Brull
3c430ca722
Fix #11966 - Add delete secrets options for managed SM (#12051)
* Add delete secrets options for managed SM

* Align tests with mysql auth changes
2023-06-21 06:59:02 +02:00
Chirag Madlani
ef578ecb40
feat(ui): support disable option for tags and classification (#10925)
* feat(ui): support disable option for tags and classification

* backend changes for PATCH API to work for updating disabled flag in classifications and tags

* made UI changes to provide enable/disable option for classifications and tags

* localization changes for other languages

* Improved tags disabling functionality

* changes to disable the system tags
fixed permission checks while editing tags

* localization change for other languages

* fixed unit tests and minor improvements

* fixed failing cypress tests

* disabled the edit actions for the disabled classifications

* localization changes for other languages

* changes to use disabled badge

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2023-06-21 00:13:05 +05:30