1189 Commits

Author SHA1 Message Date
sonika-shah
a9773b9527
Add predefined color palette for UI styling assets (#15982)
* Add predefined color palette for UI styling assets - backend

* Update UI components to use customLogoConfig in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.tsx

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.test.tsx

* Remove existing customLogoConfiguration

* Refactor App.tsx and ThemeUtils.ts files

* Update AppearanceConfigSettingsPage.tsx with new imports and default theme values

* Update AppearanceConfigSettingsPage.tsx with new imports and reset functionality

* add cypress test

* Fix color update issue in CustomThemeConfig.spec.ts

* Remove existing customLogoConfiguration -migration changes

* Fix theme configuration initialization in useApplicationStore.ts

* Update CustomThemeConfig.spec.ts to check if the color is updated

* update existing customLogoConfiguration with customUiThemePreference -migration changes

* Update CustomThemeConfig.spec.ts to fix color update check

* Add unit tests for ThemeUtils

* move validateJsonSchema method to JsonUtils

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-25 22:43:02 +05:30
Pere Miquel Brull
7db1612c13
#6420 - Allow better bot role binding (#15966)
* #6420 - Allow better bot role binding

* format

* Add bot user role in data file

* Add bot user role in data file

* Trigger Build

* Trigger Build
2024-04-25 18:27:45 +02:00
Pere Miquel Brull
39dde222b2
MINOR - Migration validation list diff message (#16015)
* MINOR - Migration validation list diff message

* wording

* Trigger Build
2024-04-25 18:21:00 +02:00
Teddy
4ed87a4d08
Fix #15341 - Test Case reference as inherited field for Test Case Incident (#16027)
* fix: unique test computation to scalar_subquery

* fix: make test case reference an inherited field

* style: ran java linting

* fix: added test case resolution migration

* style: ran java linting
2024-04-25 17:31:11 +02:00
Pere Miquel Brull
8a3a9516ca
FIX - UserResource CI (#16034) 2024-04-25 14:08:07 +02:00
Mayur Singal
db8102271e
Fix #12502: Add support for lineage export (#16009) 2024-04-25 16:08:50 +05:30
Mayur Singal
b14b191416
MINOR: Add support for search index column lineage (#16032) 2024-04-25 14:15:02 +05:30
Ayush Shah
3621407642
Fixes #15732: Modify Reference for Tags to EntityName (#15938) 2024-04-25 11:53:46 +05:30
Teddy
9f49b41acf
fix: remove classifier config from openmetadata config file (#15998)
BREAKING CHANGE: if you previously used your own classifier this will remove its support
2024-04-24 11:41:40 +02:00
Pere Miquel Brull
e6c79ce810
#15901 - Non-admins cannot see users' mails (#15983)
* nullify mail

* #15901 - Non-admins cannot see users' mails

* hide masked email

* fix tests

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-24 07:28:35 +02:00
Teddy
5606e6209d
FIX #15593 - test case exc summary endpoint improvements (#15992) 2024-04-23 20:30:05 +02:00
Alexander Nedomets
e996f15a7d
ISSUE-15426 increase the server response rate GET /v1/dataQuality/testCases (#15442)
* ISSUE-15426 increase the server response rate for endpoint GET /v1/dataQuality/testCases

* fix style code

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2024-04-23 12:38:02 +02:00
Mayur Singal
85b6983eee
Fix #15062 & #14810: Fix Column level lineage overwrites pipeline Lineage & manual col lineage (#15897) 2024-04-23 09:37:43 +05:30
Teddy
449a5f2de3
FIX #11951 - ingestion logic for global profiler config (#15948)
* feat: add global metric configuration for the profiler

* style: ran java linting

* fix: renamed disable to disabled

* style: ran java linting

* feat: ometa sdk for profiler setting

* test: ingestion profiler global config tests

* fix: update metric name to use MetricType Enum

* fix: allow bot to retrieve settings

* fix: exclude GX artifacts

* feat: implement global profiler setting logic for ingestion side

* fix: exclude metrics if Metric is empty

* style: ran python linting

* style: ran python linting

* fix: skip empty metrics

* style: ran python linting

* fix: moved GET profiler config to seperate endpoint in system resource

* fix: moved compute metric filter to MetricFilter + renamed container

* fix: test failures

* fix: profiler test case
2024-04-22 22:35:37 +02:00
IceS2
08c114c340
FIXES 15626: Fix issue with not url model store (#15974)
* Changed the MLModelStore storage type to string

* fix checkstyle

* remove unused files

* Update requirements

* fix checkstyle

* Skipping MLFlow intergration on python 3.8

* Hack to allow pytest to parse the mlflow integrations test on python 3.8

* Fix checkstyle
2024-04-22 15:50:44 +02:00
Mohit Yadav
27e9012a56
Activity Feed Improvements (#15840)
* - Created Specific Schemas for different change Events
- Added Card Style for different Events
- Added Feed Info for runtime card creation and graph drawing in case test Case Result updates

* Review comment

* Test fix

* Revert unrequired changes

* Reverted changes

* Fixed Tests

* fix test

* Add Domain, Data Producs Asset addition/removal Uppdates

* Probale test case failure fix

* Followers Fields
2024-04-22 18:42:40 +05:30
Mohit Yadav
c2193a963f
fix- Allow Lineage for stored Procedure (#15397)
* Allow Lineage for stored Procedure

* Add Node in case of empty edges

* Add for OpenSearchClient
2024-04-22 17:42:39 +05:30
Imri Paran
2e1786f7b6
MINOR: use polling mechanism for bnackend tests (#15962)
* handle unstable tests

* handle retry by polling ES

* remove waitForAsyncOp

* use retry

* format

* poll elastic for 10 seconds

* fixed tag delete test with polling

* poll up to 3 seconds

* fixed tag delete test again

* fixed assertEntityReferenceFromSearch

* fixed assertEntityReferenceFromSearch

* better error messages
2024-04-22 11:10:59 +02:00
sonika-shah
7221f253bd
Fix issue in search with hierarchy for glossary terms (#15978) 2024-04-22 10:19:38 +05:30
Pere Miquel Brull
b1a4faf171
MINOR - Metadata Actions schemas (#15822)
* MINOR - Metadata Actions schemas

* add type for left side menu bar items

* fix the required fields in automatorAppConfig.json schema

* update constants

* mixin

* create new styled component to show in alerts form card

* Add new translations for ingestion pipeline and location

* Fix import statement in AlertDetailsPage.tsx

* Update automatorAppConfig.json schema to use an array for the 'type' property

* config

* move the advanced search configs to a new classBase for better customization

* Update automatorAppConfig.json schema to add "name" and "description" fields

* Update automatorAppConfig.json schema to remove "name" and "description" fields

* Add new translations for description-plural in multiple language files

* filter Automator

* revert remove

* Update pipeline actions interface and component to handle edit click event

* Update select type for advanced search fields in AdvancedSearchClassBase.ts

* Update ErrorPlaceHolderIngestion component to accept cardClassName prop

* Update FormCardSection component to accept className prop

* Update TagsContainerV2 component to include state property in tag data

* Update TagsContainerV2 component to include defaultState and defaultLabelType properties

* Update ingestionPipelineAPI.ts to include applicationType parameter in getIngestionPipelines function

* Update AdvanceSearchProvider.test.tsx to include pathname in useLocation mock

* work on comments

* fix sonar lint errors

* format

* Update useLocation mock in AdvanceSearchProvider.test.tsx to include pathname

* revert remove

* Update LogsViewerPage interfaces and component to include application_task in IngestionPipelineLogByIdInterface

* Update LeftSidebar component to import LeftSidebarItem interface and fix type annotation in children map function

* fix observability alert test

* revert the form list add method change

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-04-19 12:46:58 +02:00
Mohit Yadav
701e288512
Fix Default App Init (#15965) 2024-04-19 13:45:33 +05:30
Ayush Shah
d5b1465406
Fixes #14113 - Allow SSL file uploads (#15828) 2024-04-19 11:38:27 +05:30
Mohit Yadav
bc13659a6d
Inherit sso roles (#15903)
* Use Roles from Provider

* Add Cache For Roles From Users

* Fix Failing Test

* Revert secrets

Update Impl to use loggedInUser and CreateUser for Role assignment and resync

* Revert Expiry check
2024-04-18 20:46:04 +05:30
Mohit Yadav
d1d964867d
Move migration for apps to 1.3.3 all together (#15944) 2024-04-18 14:26:05 +05:30
sonika-shah
45bae608bd
Fixes #10724 : Search with hierarchy api for glossary terms (#15865)
* Fixes #10724 : New search with hierarchy api for glossary terms

* Added query param in existing search api - to search with hierarchy

* Created new json to manage hierarchy and other updates
2024-04-18 14:22:18 +05:30
IceS2
612bea46a6
FIXES #14772: Display Assets without Owner/Description (#15794)
* Update table_index_mapping to turn columns into a nested type

* test: added query null/empty description test

* style: ran java linting

* Add Table test for incomplete description search query

* Fix tests

* Fix linters

* Fix column search

* Increase waitForEsAsync to 500

* Defined the new calculated fields for the indexes

* Refactor search of assets with no description

* add descriptionStatus to the entities taht didn't have it

* Remove descriptionStatus where it's not needed

* Remove TestCase and TestSuite from search test

* Remove TestCase and TestSuite from search test

* Increase waitForEsAsync for checkDeleteentity 1000

* Update jp and zh mappings

* Fix syntax

* Increase waitForEsAsync for some tests to 1000

* Remove unused aggregations

* Refactor code to reduce repetition

* Fix checkstyle

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2024-04-18 10:41:58 +02:00
Teddy
8d377f5c99
Fix #11951 [backend] - Implement Profiler Global Config (#15889)
* feat: add global metric configuration for the profiler

* style: ran java linting

* fix: renamed disable to disabled

* style: ran java linting

* fix: json schema profiler config title
2024-04-16 18:56:21 +02:00
Mohit Yadav
c7e03471a1
Add Pipeline Status to Ingestion Pipeline Index (#15918) 2024-04-16 21:40:24 +05:30
Imri Paran
6349adb0ec
MINOR: added a DELETE method for failedRowsSample (#15878)
* added a DELETE method for failedRowsSample

* fixed tests

* changed required permission to EDIT_SAMPLE_DATA

* changed required permission to DELETE_SAMPLE_DATA

* changed required permission to DELETE_TEST_CASE_FAILED_ROWS_SAMPLE

* changed required permissions

addFailedRowsSample -> EDIT_TEST_CASE_FAILED_ROWS_SAMPLE
getFailedRowsSample -> VIEW_TEST_CASE_FAILED_ROWS_SAMPLE

* removed op EditTestCaseFailedRowsSample

* fixed tests

* format
2024-04-16 17:45:59 +02:00
Teddy
84469e643c
FIX #13475 - Add Parameter Validation for Min/Max tests (#15849)
* feat: add validation field to test case definition parameters

* feat: added parameter validation logic

* fix: set ES testcontainer startup attemp to 3

* style: ran java linting

* fix: added waitForEsAsyncOp in post_trigger_app_200 test
2024-04-15 07:52:43 -07:00
Karan Hotchandani
bd3838868c
disable install for preview apps (#15884)
* disable install for preview apps

* move screenshot path to base class

* MINOR - Add Marketplace preview

* format

* add unit tests

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-04-15 17:32:01 +05:30
sonika-shah
c010939072
Added payload size check logic for Elasticsearch (#15540)
* Added payload size check logic for Elasticsearch

* Added payload size check logic for Elasticsearch - made review changes

* Added payload size check logic for Elasticsearch - added env in docker files

* Optimize payload logic, fix search application error stats issue
2024-04-15 16:25:06 +05:30
Teddy
8fc4319f64
MINOR Add owner filter test case (#15853)
* feat: added owner filter to test case list from search

* refactor: deprecated `orderByLatestExecution` param

BREAKING CHANGE: `orderByLatestExecution` param from list endpoint `GET` has been removed in favor of `sortField` and `sortType` from the `search/list` endpoint

* style: ran java linting

* fix: increase es async time
2024-04-11 17:27:39 +02:00
Imri Paran
b2ce491ff1
MINOR: Add failed rows sample to test case (#15682)
* add failed sample data

* format

* fixed masking pii data in test failed rows sample

* format

* failedRowsSamples -> failedRowsSample

* failedRowsSamples -> failedRowsSample

* fixed tests

* format

* wip

* added computePassedFailedRowCount to python client

* comment for loggerLevel

* format

* fixed tests

* tests for putting / deleting failed samples

* format

* format

* added test case for pii test

* changed method name to deleteTestCaseFailedRowsSample

* added getComputePassedFailedRowCount
2024-04-10 17:00:00 +02:00
Imri Paran
d9a7ebe5e1
MINOR: fix application in corrupted state (#15830)
* fix application in corrupted state

* format

* remove old versions of the app

* format

* use name instead of fqn
2024-04-09 07:58:34 +02:00
mohitdeuex
a41dc54667 - Create Dummy Columns on empty import of table columns 2024-04-09 11:08:04 +05:30
Teddy
7bc48ffd0d
MINOR: exclude change description from testSuite key in table entity indexing (#15836)
* fix: exclude change description from testSuite key in table entity reindexing

* style: removed unecessary comment
2024-04-08 22:00:00 +00:00
Teddy
dc4c4e41b8
Fix #10354 - Implement Test Suite filter from Search (#15814) 2024-04-08 18:40:39 +02:00
Mohit Yadav
1a7314193f
Validate against PersonalAccess (#15832) 2024-04-08 17:13:19 +05:30
Mohit Yadav
5a88d15228
[Feature] Import/Export For Table, DatabaseSchema, Databases, Service (#15816)
* - Add Import Export Separation for GlossaryTerms

* - Fixed Table Resrouce Test

* - Review Comment #2

* - GlossaryTestFix, Glossary does not allow Tier Tags

* - Database Schema Tests Fix

* - Create Database, DatabaseSchema, DatabaseService import entity if not exists

* - Fix Test for Database DatabaseSchema, Table
2024-04-07 02:21:56 +05:30
IceS2
12a4c578a2
MINOR: Fix jsonpatch operation order (#15680)
* Mantain the OperationType Order when considering the dividing groups

* Remove reordering the jsonpatch operations from the backend

* Fix checkstyle

* Fix UnitTests to comply with no reordering

* Initial idea on how to fix our current jsonpatch builder from python

* fix(JsonUtils): Change JSONPatch library used

When creating a JSONPatch by using the 'createDiff' method, the library
we are using is not returning a correct JSONPatch when removing multiple
items from an array.

Since the library doesn't provide good ways to override this behavior
and fix it, we decided to move away from it and use the json-patch
library only for this specific operation.

* Fix linters

* Add docstrings

* Refactor patch updated on ingestion framework

* Add UnitTests

* Fix linters
2024-04-05 15:52:01 +02:00
Mohit Yadav
5f9a565f3a
[AlertNotication] Fix formatting (#15800)
* - Remove sourceHash from the Activity Feed
- Update Test Case Page Uri Link

* - Remove lifeCycle

* - Add To Field Change List

* - Add asset Url to conversations and Task Updates

* - Make Redirect to Activity Feed Task Page
2024-04-04 18:28:30 +05:30
Mohit Yadav
daae3c48e3
- Do not validate bot domain (#15796) 2024-04-03 16:12:42 +05:30
Teddy
dd53f45cc6
MINOR fix test case search filtering (#15785)
* fix: test case filtering from search endpoint

* style: ran java linting

* fix: updated test case EL in list test
2024-04-03 11:01:54 +02:00
Mohit Yadav
6e8d14fad4
- Change Prompt to consent on every login (#15792) 2024-04-03 13:13:33 +05:30
Mohit Yadav
72bd411fe5
[Vulnerabilities] Update Dependencies to fix Vulnerabilities (#15775)
* [VBY] Fix POm version for affected dependency

* - add cp3 dependency

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-04-03 12:04:01 +05:30
Mohit Yadav
77bb725d5b
- add Step logs for Auth (#15786) 2024-04-03 11:15:13 +05:30
sonika-shah
4948ab3766
Fix backend bug that prevented associated tags from being removed when a glossary term is deleted. (#15676)
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-04-03 10:18:57 +05:30
Teddy
4b13403931
Fix #15594 - Implement domain support for test cases (#15736)
* feat: add list from search service logic

* fix: added test and enable ES testing

* style: ran java linting

* fix: ES container startup timeout

* style: ran java linting

* fix: removed array definition as attributes of filter class

* style: ran java linting

* fix: update list from search

* feat: added q param to list from search endpoint

* style: ran java linting

* fix: enable ES test for list from search

* fix: added include/exclude from source

* style: ran java linting

* feat: add domain support for test case

* fix: added search service check for domain inheritence

* feat: added domain query param to list from search

* style: ran java linting
2024-04-02 22:45:48 +02:00
Mohit Yadav
b38b7e3231
- added log (#15777) 2024-04-02 18:39:48 +05:30