2200 Commits

Author SHA1 Message Date
Teddy
5646d0409d
Minor fix test case sorting in search client (#16035)
* fix: search sorting on null timestamp

* skipping cypress tests for filters

* fix: use getChildAlias for ES children update/delete

* fix: add retry logic for test case resource ES

* style: ran java linting

* fix: cascade search deletion to children

* updated profilerConfig

* fix: move ES deletion to end of flow

* style: ran java linting

* chore: test mvn execution time without test case resource test

* style: fix java linting

* chore: re enable test case resource tests

* fix: reduce wait time for get test summary

* fix: add rertry on deleteTag for search tests

* fix: denormalize test entities relationship for search

* style: ran java linting

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-05-03 08:02:02 +02:00
sonika-shah
b7c54c9ab5
fix issue in directChildrenOf api for postgresql (#16093) 2024-05-01 15:15:00 +05:30
Teddy
91f930d4c4
FIX #15956 - Send web socket notification on incident assignment/ack (#16086)
* fix: send websocket notification on incident assignment

* style: ran java linting
2024-04-30 14:17:06 +02:00
Mohit Yadav
5954ecf060
Enable Support for Status of Pipeline and IngestionPipeline in Elastic Search (#16023)
* - Add Ingestion Pipeline supports search

* - Add Pipeline Status in Repository

* - Add PipelineStatus update on entity pipeline status updation

* - Reindex Pipeline Usage Entities

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2024-04-30 15:12:03 +05:30
sonika-shah
432820d929
Fix #16046 : modify glossaryTerms api endpoint to support querying immediate children with childrenCount (#16056)
* Fix #16046 : modify glossaryTerms api endpoint to support querying immediate children with childrenCount

* childrenCount added in glossary_term_index_mapping.json

* childrenCount added in glossary_term_index_mapping.json
2024-04-30 00:01:07 +05:30
Mohit Yadav
fa73171dff
[GlossaryTerm] Fix Glossary Term Issue (#16068)
* - Don't update gloosary Term in case of equal tags

* - Invert Conditions

* - Mutually Exclusive cannot be updated

* - Fix Failing Tests
2024-04-29 18:16:49 +05:30
Mohit Yadav
92e5b080cf
Add more information to activity feed (#16007)
* Add more information to activity feed

* Null Or Empty Descriptions

* give owner reference

* - Entity Domain Updates

* - Fix Description

* - Custom Properties Addition
2024-04-29 18:16:21 +05:30
Mohit Yadav
4272c5ffee
- fix Reindex status in Operations (#16065)
* - fix Status

* - TODO stats print

* - Fix Stats

* - Update Error Message
2024-04-28 20:58:45 -07:00
sonika-shah
c6afcce52d
update SettingsCache and fix migration issues (#16052)
* update SettingsCache and fix migration issues

* java checkstyle change
2024-04-29 09:26:09 +05:30
Sriharsha Chintalapani
0695398a64
Fix #15490: openmetadata-ops.sh reindex does not work (#15993)
* Fix #15490: openmetadata-ops.sh reindex does not work

* add status for reindex

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-04-26 14:17:11 +02:00
Imri Paran
4ac5912d4c
MINOR: added TestCase inspection query to backend and sample data (#16003)
* added TestCase inspection query to backend and sample data

* format

* format
2024-04-26 11:49:08 +02:00
sonika-shah
a9773b9527
Add predefined color palette for UI styling assets (#15982)
* Add predefined color palette for UI styling assets - backend

* Update UI components to use customLogoConfig in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.tsx

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.test.tsx

* Remove existing customLogoConfiguration

* Refactor App.tsx and ThemeUtils.ts files

* Update AppearanceConfigSettingsPage.tsx with new imports and default theme values

* Update AppearanceConfigSettingsPage.tsx with new imports and reset functionality

* add cypress test

* Fix color update issue in CustomThemeConfig.spec.ts

* Remove existing customLogoConfiguration -migration changes

* Fix theme configuration initialization in useApplicationStore.ts

* Update CustomThemeConfig.spec.ts to check if the color is updated

* update existing customLogoConfiguration with customUiThemePreference -migration changes

* Update CustomThemeConfig.spec.ts to fix color update check

* Add unit tests for ThemeUtils

* move validateJsonSchema method to JsonUtils

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-25 22:43:02 +05:30
Pere Miquel Brull
7db1612c13
#6420 - Allow better bot role binding (#15966)
* #6420 - Allow better bot role binding

* format

* Add bot user role in data file

* Add bot user role in data file

* Trigger Build

* Trigger Build
2024-04-25 18:27:45 +02:00
Pere Miquel Brull
39dde222b2
MINOR - Migration validation list diff message (#16015)
* MINOR - Migration validation list diff message

* wording

* Trigger Build
2024-04-25 18:21:00 +02:00
Teddy
4ed87a4d08
Fix #15341 - Test Case reference as inherited field for Test Case Incident (#16027)
* fix: unique test computation to scalar_subquery

* fix: make test case reference an inherited field

* style: ran java linting

* fix: added test case resolution migration

* style: ran java linting
2024-04-25 17:31:11 +02:00
Pere Miquel Brull
8a3a9516ca
FIX - UserResource CI (#16034) 2024-04-25 14:08:07 +02:00
Mayur Singal
db8102271e
Fix #12502: Add support for lineage export (#16009) 2024-04-25 16:08:50 +05:30
Mayur Singal
b14b191416
MINOR: Add support for search index column lineage (#16032) 2024-04-25 14:15:02 +05:30
Ayush Shah
3621407642
Fixes #15732: Modify Reference for Tags to EntityName (#15938) 2024-04-25 11:53:46 +05:30
Teddy
9f49b41acf
fix: remove classifier config from openmetadata config file (#15998)
BREAKING CHANGE: if you previously used your own classifier this will remove its support
2024-04-24 11:41:40 +02:00
Pere Miquel Brull
e6c79ce810
#15901 - Non-admins cannot see users' mails (#15983)
* nullify mail

* #15901 - Non-admins cannot see users' mails

* hide masked email

* fix tests

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-24 07:28:35 +02:00
Teddy
5606e6209d
FIX #15593 - test case exc summary endpoint improvements (#15992) 2024-04-23 20:30:05 +02:00
Alexander Nedomets
e996f15a7d
ISSUE-15426 increase the server response rate GET /v1/dataQuality/testCases (#15442)
* ISSUE-15426 increase the server response rate for endpoint GET /v1/dataQuality/testCases

* fix style code

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2024-04-23 12:38:02 +02:00
Mayur Singal
85b6983eee
Fix #15062 & #14810: Fix Column level lineage overwrites pipeline Lineage & manual col lineage (#15897) 2024-04-23 09:37:43 +05:30
Teddy
449a5f2de3
FIX #11951 - ingestion logic for global profiler config (#15948)
* feat: add global metric configuration for the profiler

* style: ran java linting

* fix: renamed disable to disabled

* style: ran java linting

* feat: ometa sdk for profiler setting

* test: ingestion profiler global config tests

* fix: update metric name to use MetricType Enum

* fix: allow bot to retrieve settings

* fix: exclude GX artifacts

* feat: implement global profiler setting logic for ingestion side

* fix: exclude metrics if Metric is empty

* style: ran python linting

* style: ran python linting

* fix: skip empty metrics

* style: ran python linting

* fix: moved GET profiler config to seperate endpoint in system resource

* fix: moved compute metric filter to MetricFilter + renamed container

* fix: test failures

* fix: profiler test case
2024-04-22 22:35:37 +02:00
IceS2
08c114c340
FIXES 15626: Fix issue with not url model store (#15974)
* Changed the MLModelStore storage type to string

* fix checkstyle

* remove unused files

* Update requirements

* fix checkstyle

* Skipping MLFlow intergration on python 3.8

* Hack to allow pytest to parse the mlflow integrations test on python 3.8

* Fix checkstyle
2024-04-22 15:50:44 +02:00
Mohit Yadav
27e9012a56
Activity Feed Improvements (#15840)
* - Created Specific Schemas for different change Events
- Added Card Style for different Events
- Added Feed Info for runtime card creation and graph drawing in case test Case Result updates

* Review comment

* Test fix

* Revert unrequired changes

* Reverted changes

* Fixed Tests

* fix test

* Add Domain, Data Producs Asset addition/removal Uppdates

* Probale test case failure fix

* Followers Fields
2024-04-22 18:42:40 +05:30
Mohit Yadav
c2193a963f
fix- Allow Lineage for stored Procedure (#15397)
* Allow Lineage for stored Procedure

* Add Node in case of empty edges

* Add for OpenSearchClient
2024-04-22 17:42:39 +05:30
Imri Paran
2e1786f7b6
MINOR: use polling mechanism for bnackend tests (#15962)
* handle unstable tests

* handle retry by polling ES

* remove waitForAsyncOp

* use retry

* format

* poll elastic for 10 seconds

* fixed tag delete test with polling

* poll up to 3 seconds

* fixed tag delete test again

* fixed assertEntityReferenceFromSearch

* fixed assertEntityReferenceFromSearch

* better error messages
2024-04-22 11:10:59 +02:00
sonika-shah
7221f253bd
Fix issue in search with hierarchy for glossary terms (#15978) 2024-04-22 10:19:38 +05:30
Pere Miquel Brull
b1a4faf171
MINOR - Metadata Actions schemas (#15822)
* MINOR - Metadata Actions schemas

* add type for left side menu bar items

* fix the required fields in automatorAppConfig.json schema

* update constants

* mixin

* create new styled component to show in alerts form card

* Add new translations for ingestion pipeline and location

* Fix import statement in AlertDetailsPage.tsx

* Update automatorAppConfig.json schema to use an array for the 'type' property

* config

* move the advanced search configs to a new classBase for better customization

* Update automatorAppConfig.json schema to add "name" and "description" fields

* Update automatorAppConfig.json schema to remove "name" and "description" fields

* Add new translations for description-plural in multiple language files

* filter Automator

* revert remove

* Update pipeline actions interface and component to handle edit click event

* Update select type for advanced search fields in AdvancedSearchClassBase.ts

* Update ErrorPlaceHolderIngestion component to accept cardClassName prop

* Update FormCardSection component to accept className prop

* Update TagsContainerV2 component to include state property in tag data

* Update TagsContainerV2 component to include defaultState and defaultLabelType properties

* Update ingestionPipelineAPI.ts to include applicationType parameter in getIngestionPipelines function

* Update AdvanceSearchProvider.test.tsx to include pathname in useLocation mock

* work on comments

* fix sonar lint errors

* format

* Update useLocation mock in AdvanceSearchProvider.test.tsx to include pathname

* revert remove

* Update LogsViewerPage interfaces and component to include application_task in IngestionPipelineLogByIdInterface

* Update LeftSidebar component to import LeftSidebarItem interface and fix type annotation in children map function

* fix observability alert test

* revert the form list add method change

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-04-19 12:46:58 +02:00
Mohit Yadav
701e288512
Fix Default App Init (#15965) 2024-04-19 13:45:33 +05:30
Ayush Shah
d5b1465406
Fixes #14113 - Allow SSL file uploads (#15828) 2024-04-19 11:38:27 +05:30
Mohit Yadav
bc13659a6d
Inherit sso roles (#15903)
* Use Roles from Provider

* Add Cache For Roles From Users

* Fix Failing Test

* Revert secrets

Update Impl to use loggedInUser and CreateUser for Role assignment and resync

* Revert Expiry check
2024-04-18 20:46:04 +05:30
Mohit Yadav
d1d964867d
Move migration for apps to 1.3.3 all together (#15944) 2024-04-18 14:26:05 +05:30
sonika-shah
45bae608bd
Fixes #10724 : Search with hierarchy api for glossary terms (#15865)
* Fixes #10724 : New search with hierarchy api for glossary terms

* Added query param in existing search api - to search with hierarchy

* Created new json to manage hierarchy and other updates
2024-04-18 14:22:18 +05:30
IceS2
612bea46a6
FIXES #14772: Display Assets without Owner/Description (#15794)
* Update table_index_mapping to turn columns into a nested type

* test: added query null/empty description test

* style: ran java linting

* Add Table test for incomplete description search query

* Fix tests

* Fix linters

* Fix column search

* Increase waitForEsAsync to 500

* Defined the new calculated fields for the indexes

* Refactor search of assets with no description

* add descriptionStatus to the entities taht didn't have it

* Remove descriptionStatus where it's not needed

* Remove TestCase and TestSuite from search test

* Remove TestCase and TestSuite from search test

* Increase waitForEsAsync for checkDeleteentity 1000

* Update jp and zh mappings

* Fix syntax

* Increase waitForEsAsync for some tests to 1000

* Remove unused aggregations

* Refactor code to reduce repetition

* Fix checkstyle

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2024-04-18 10:41:58 +02:00
Teddy
8d377f5c99
Fix #11951 [backend] - Implement Profiler Global Config (#15889)
* feat: add global metric configuration for the profiler

* style: ran java linting

* fix: renamed disable to disabled

* style: ran java linting

* fix: json schema profiler config title
2024-04-16 18:56:21 +02:00
Mohit Yadav
c7e03471a1
Add Pipeline Status to Ingestion Pipeline Index (#15918) 2024-04-16 21:40:24 +05:30
Imri Paran
6349adb0ec
MINOR: added a DELETE method for failedRowsSample (#15878)
* added a DELETE method for failedRowsSample

* fixed tests

* changed required permission to EDIT_SAMPLE_DATA

* changed required permission to DELETE_SAMPLE_DATA

* changed required permission to DELETE_TEST_CASE_FAILED_ROWS_SAMPLE

* changed required permissions

addFailedRowsSample -> EDIT_TEST_CASE_FAILED_ROWS_SAMPLE
getFailedRowsSample -> VIEW_TEST_CASE_FAILED_ROWS_SAMPLE

* removed op EditTestCaseFailedRowsSample

* fixed tests

* format
2024-04-16 17:45:59 +02:00
Teddy
84469e643c
FIX #13475 - Add Parameter Validation for Min/Max tests (#15849)
* feat: add validation field to test case definition parameters

* feat: added parameter validation logic

* fix: set ES testcontainer startup attemp to 3

* style: ran java linting

* fix: added waitForEsAsyncOp in post_trigger_app_200 test
2024-04-15 07:52:43 -07:00
Karan Hotchandani
bd3838868c
disable install for preview apps (#15884)
* disable install for preview apps

* move screenshot path to base class

* MINOR - Add Marketplace preview

* format

* add unit tests

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-04-15 17:32:01 +05:30
sonika-shah
c010939072
Added payload size check logic for Elasticsearch (#15540)
* Added payload size check logic for Elasticsearch

* Added payload size check logic for Elasticsearch - made review changes

* Added payload size check logic for Elasticsearch - added env in docker files

* Optimize payload logic, fix search application error stats issue
2024-04-15 16:25:06 +05:30
Teddy
8fc4319f64
MINOR Add owner filter test case (#15853)
* feat: added owner filter to test case list from search

* refactor: deprecated `orderByLatestExecution` param

BREAKING CHANGE: `orderByLatestExecution` param from list endpoint `GET` has been removed in favor of `sortField` and `sortType` from the `search/list` endpoint

* style: ran java linting

* fix: increase es async time
2024-04-11 17:27:39 +02:00
Imri Paran
b2ce491ff1
MINOR: Add failed rows sample to test case (#15682)
* add failed sample data

* format

* fixed masking pii data in test failed rows sample

* format

* failedRowsSamples -> failedRowsSample

* failedRowsSamples -> failedRowsSample

* fixed tests

* format

* wip

* added computePassedFailedRowCount to python client

* comment for loggerLevel

* format

* fixed tests

* tests for putting / deleting failed samples

* format

* format

* added test case for pii test

* changed method name to deleteTestCaseFailedRowsSample

* added getComputePassedFailedRowCount
2024-04-10 17:00:00 +02:00
Imri Paran
d9a7ebe5e1
MINOR: fix application in corrupted state (#15830)
* fix application in corrupted state

* format

* remove old versions of the app

* format

* use name instead of fqn
2024-04-09 07:58:34 +02:00
mohitdeuex
a41dc54667 - Create Dummy Columns on empty import of table columns 2024-04-09 11:08:04 +05:30
Teddy
7bc48ffd0d
MINOR: exclude change description from testSuite key in table entity indexing (#15836)
* fix: exclude change description from testSuite key in table entity reindexing

* style: removed unecessary comment
2024-04-08 22:00:00 +00:00
Teddy
dc4c4e41b8
Fix #10354 - Implement Test Suite filter from Search (#15814) 2024-04-08 18:40:39 +02:00
Mohit Yadav
1a7314193f
Validate against PersonalAccess (#15832) 2024-04-08 17:13:19 +05:30