* Feat(ui): add and edit certifications from asset page (#21344)
* added styling to certification
* added tests
* changed icons for certificatie popup
---------
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* #19529 Improvise import to include certification for Assets (Database, DatabaseSchema, DatabaseService), Introduce EDIT_CERTIFICATION policy and Fix Elastic Search Indexes on Certification (#21564)
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* Applied mvn spotless:apply
* Reused the Existing UPDATE_CERTIFICATION_SCRIPT for ElasticSearch Indexing
* Added field certification in the static String FIELDS
* fix playwright test around bulk action
* #19529 - Persisting Null or empty in the bulk import for certification
* #19529 - Persisting Null or empty in the bulk import for certification - Moved the if block to the top
* mvn spotless:apply
* Reverted an unimportant file
* mvn spotless:apply
* #19529 - Persisting the field Certification in clearFIeldsInternal
* typescript files for edit_certification
* Revert "typescript files for edit_certification"
This reverts commit f5e5514a98008cbd0b62d7cb21fefe61659e97cb.
* typescript files for edit_certification
* mvn:spotless:apply
* Removed correction
* needed typescript file for edit_certification
* Removed Unnecessary Comments
* Improved Test Cases - Added DATA_ASSET_SEARCH alias instead of GLOBAL_ALIAS
* Fixed csv values in order
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* update certification component
* Fixed Certification Indexes
* Fixed Missing Certification Label for DBSchema
* add certification changes
* fix test errors
* Remove Certification field and mapping from database service asset
* Removed certification from schema and fixed test
* fix tests
---------
Co-authored-by: Dhruv Parmar <83108871+dhruvjsx@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <81347100+yan-3005@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
* Correct FQN values will be shown to the user in case of validation failures in glossary term tags
* Applied mvn:spotless apply
* Added Test cases to verify the validation of assets
* Fix#1681 - Deleted Filter in Automations Not Working as Expected
* Revert "Fix #1681 - Deleted Filter in Automations Not Working as Expected"
This reverts commit d941ad440b6c5bcb66d4ac8bb31c321fb23cbcfc.
* Set deleted param in /search/query API to type Boolean to allow default null values
* fix AUT test failures related to user search in settings/users page
* fix AUT test failures related to user search in settings/users page
* Make `deleted` param optional for search API call in explore page
* Fix the deleted users showing on the user listing page after user search
* Fix playwright tests
* Fix playwright tests
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Fix: Remove operations and glossaryTags from bulk asset add api
* Fix: Improved the test case
* addTagToAssets schema - Removed operations field in the payload
* Replacing the request payload for validating tags in the UI
* UI changes for add and remove assets to glossary term
* Using getByName and Entity.GLOSSARY
---------
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* SCIM interface
* removed unwanted code
* remove SCIM registration in OM
* reverted scim configuration settings, as it is not needed
* Added security context to create user
* Added scimusername in user jsons
* added externalid, scimUsername in createUser
* Added security context on create and update groups
* Added jakarta imports
* Authorization added
* Added role, policy and bot for SCIM
* Fix#21653 - Tag search based on displayName
* added test
* update test to search by classification display name
---------
Co-authored-by: shrushti2000 <shrushtipolekar@gmail.com>
* refactor: removed testSuite field from CreateTestCase
BREAKING CHANGE: when creating a test case, testsuite is now derived from entityLink (fetch or created)
* feat: allow setting tags when creating a test case
* style: ran linters
* fix: compiling error
* fix: failing test case
* fix: failing tests
* removed testSuite from required filed
* fixed ui side
* style: ran java linting
* deprecation: remove testSuite param from ingestion
* fix: remove test suite filed
* fix: remove test_suite field
---------
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
* Update Classification Schema to include Governance Fields
* Removed Tags, Reviewers, Domain from Classification as they are needed and corrected tests
* Added Permission check for owners in Classification Resource Test
* Added LoadTags.ts generated from createClassificationSchema.json
* Only have my schema changes in the typescript files, ignore other changes.
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Fix Lineage/EntityRelationship mappings back to object
* Add displayName to the user when the user is created and displayName is not set
* remove UserResource related changes
---------
Co-authored-by: sonikashah <sonikashah94@gmail.com>
* Fix: Remove operations and glossaryTags from bulk asset add api
* Fix: Improved the test case
* addTagToAssets schema - Removed operations field in the payload
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Fix#21457: SearchIndexing: Limit of total fields [1000] has been exceeded
* Fix#21457: SearchIndexing: Limit of total fields [1000] has been exceeded
* Fix BulkItemException throwing back to the UI
* add upstreamLineage to all appropriate indexes
* Remove conflicting lines and clean up index mappings for api_collection and stored_procedure JSON files
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* Fix#21502: TooManyNestedClauses: Query contains too many nested clauses; maxClauseCount is set to 4096 error in table_search_index for complex search queries in Opensearch
* Fix#21502: TooManyNestedClauses: Query contains too many nested clauses; maxClauseCount is set to 4096 error in table_search_index for complex search queries in Opensearch
* Update openmetadata-service/src/test/java/org/openmetadata/service/resources/search/SearchResourceTest.java
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix mvn build
* fix checkstyle
* fix tests in SearchResourceTest
* fix tests in SearchResourceTest
* instead of wrapping baseQuery+functionScore in a new BoolQuery,use functionScore directly as the final query - avoid duplication of mult_match query
* add requestSchema fields in APIEndpointIndex, and remove array fields in searchSettings to use fuzzyfields avoiding clause explosion
* - Set fuzzyMaxExpansions/maxExpansions to 10 instead of default 50
- Turn off fuzziness (Fuzziness.ZERO) for `.ngram` fields
- Update QUERY_SYNTAX_PATTERN so only uppercase AND, OR, NOT are treated as boolean operators
* use non Fuzzy query for ngram Fields in QueryStringBuilder , remove duplication of query in should phrase
* add fuzzy/nonFuzzy Fields for each asset in searchSettings.json
* add fuzzy/nonFuzzy Fields for each asset in searchSettings.json
* update isFuzzyField function
* fix playwright
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* Add PIICategoryTags and some utilities on top of them.
* Fix static-check
* Add test for fqn representation
* Add NEREntityGeneralTags.json from Collate
* Add test to check PIICategoryTags agree with the ones used by OM server
* Add LabelExtractor
* Fix style
* Add ignore superflous-parens for pylint
* Ass comment as per PR review
* Fix not-updated PII-IT
* Remove duplicated IT test for PII
---------
Co-authored-by: Pere Menal <pere.menal@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* queryParams to the webhook request URL
* add activityThreadsRetentionPeriod in the DataRetentionApplication jsons
* clean up activity threads of type conversation
* Revert "clean up activity threads of type conversation"
This reverts commit e66ea8623be6aa1c5d07afdec04b4807eb21c7e7.
* Revert "add activityThreadsRetentionPeriod in the DataRetentionApplication jsons"
This reverts commit 4180be18a8d5e0d4fd4b3d4662df0532bb16dbed.
* deprecation: remove testCaseResults endpoint from testCaseResource
* fix: path in test e2e test
* fix: endpoint name to testCaseResults
* style: fix java linting
* Fix#21533 - Add a tool to openmetadata-ops.sh to delete orphaned relations
* Fix#21533 - Add a tool to openmetadata-ops.sh to delete orphaned relations
* Add exit 1 and make dry-run default
* Add exit 1 and make dry-run default
* Fix checkstyle
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
