* Fix: Inherited Tags from Glossary Term should be shown in put/patch column response
* Placement of the statement fixed
---------
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster
* Add exact match configs
* Add exact match configs
* Update Logic to build search source builder with exact match priority
* Revert "Update Logic to build search source builder with exact match priority"
This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.
* Revert "Add exact match configs"
This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.
* revert display name change
* make boost mode as sum by defaul
* add more fqnparts for schema and database
* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue
* use fqn split
* refactor fqn parsing
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster
* Add exact match configs
* Add exact match configs
* Update Logic to build search source builder with exact match priority
* Revert "Update Logic to build search source builder with exact match priority"
This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.
* Revert "Add exact match configs"
This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.
* revert display name change
* make boost mode as sum by defaul
* add more fqnparts for schema and database
* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue
* use fqn split
* Refactor FQN Parts
---------
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* add owners field in classification
* fix tests
* add owners field in tags page
* Tag Inherits owners from classification
* Added loadTags.ts for owners field
* add owners for tags page
* fix tests
* review comments
* add owners field in create form
* fix domain test
---------
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Feat(ui): add and edit certifications from asset page (#21344)
* added styling to certification
* added tests
* changed icons for certificatie popup
---------
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* #19529 Improvise import to include certification for Assets (Database, DatabaseSchema, DatabaseService), Introduce EDIT_CERTIFICATION policy and Fix Elastic Search Indexes on Certification (#21564)
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* Applied mvn spotless:apply
* Reused the Existing UPDATE_CERTIFICATION_SCRIPT for ElasticSearch Indexing
* Added field certification in the static String FIELDS
* fix playwright test around bulk action
* #19529 - Persisting Null or empty in the bulk import for certification
* #19529 - Persisting Null or empty in the bulk import for certification - Moved the if block to the top
* mvn spotless:apply
* Reverted an unimportant file
* mvn spotless:apply
* #19529 - Persisting the field Certification in clearFIeldsInternal
* typescript files for edit_certification
* Revert "typescript files for edit_certification"
This reverts commit f5e5514a98008cbd0b62d7cb21fefe61659e97cb.
* typescript files for edit_certification
* mvn:spotless:apply
* Removed correction
* needed typescript file for edit_certification
* Removed Unnecessary Comments
* Improved Test Cases - Added DATA_ASSET_SEARCH alias instead of GLOBAL_ALIAS
* Fixed csv values in order
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* update certification component
* Fixed Certification Indexes
* Fixed Missing Certification Label for DBSchema
* add certification changes
* fix test errors
* Remove Certification field and mapping from database service asset
* Removed certification from schema and fixed test
* fix tests
---------
Co-authored-by: Dhruv Parmar <83108871+dhruvjsx@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <81347100+yan-3005@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
* Correct FQN values will be shown to the user in case of validation failures in glossary term tags
* Applied mvn:spotless apply
* Added Test cases to verify the validation of assets
* Fix#1681 - Deleted Filter in Automations Not Working as Expected
* Revert "Fix #1681 - Deleted Filter in Automations Not Working as Expected"
This reverts commit d941ad440b6c5bcb66d4ac8bb31c321fb23cbcfc.
* Set deleted param in /search/query API to type Boolean to allow default null values
* fix AUT test failures related to user search in settings/users page
* fix AUT test failures related to user search in settings/users page
* Make `deleted` param optional for search API call in explore page
* Fix the deleted users showing on the user listing page after user search
* Fix playwright tests
* Fix playwright tests
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Fix: Remove operations and glossaryTags from bulk asset add api
* Fix: Improved the test case
* addTagToAssets schema - Removed operations field in the payload
* Replacing the request payload for validating tags in the UI
* UI changes for add and remove assets to glossary term
* Using getByName and Entity.GLOSSARY
---------
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* SCIM interface
* removed unwanted code
* remove SCIM registration in OM
* reverted scim configuration settings, as it is not needed
* Added security context to create user
* Added scimusername in user jsons
* added externalid, scimUsername in createUser
* Added security context on create and update groups
* Added jakarta imports
* Authorization added
* Added role, policy and bot for SCIM
* refactor: removed testSuite field from CreateTestCase
BREAKING CHANGE: when creating a test case, testsuite is now derived from entityLink (fetch or created)
* feat: allow setting tags when creating a test case
* style: ran linters
* fix: compiling error
* fix: failing test case
* fix: failing tests
* removed testSuite from required filed
* fixed ui side
* style: ran java linting
* deprecation: remove testSuite param from ingestion
* fix: remove test suite filed
* fix: remove test_suite field
---------
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
* Update Classification Schema to include Governance Fields
* Removed Tags, Reviewers, Domain from Classification as they are needed and corrected tests
* Added Permission check for owners in Classification Resource Test
* Added LoadTags.ts generated from createClassificationSchema.json
* Only have my schema changes in the typescript files, ignore other changes.
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Fix: Remove operations and glossaryTags from bulk asset add api
* Fix: Improved the test case
* addTagToAssets schema - Removed operations field in the payload
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Fix#21457: SearchIndexing: Limit of total fields [1000] has been exceeded
* Fix#21457: SearchIndexing: Limit of total fields [1000] has been exceeded
* Fix BulkItemException throwing back to the UI
* add upstreamLineage to all appropriate indexes
* Remove conflicting lines and clean up index mappings for api_collection and stored_procedure JSON files
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* Fix#21502: TooManyNestedClauses: Query contains too many nested clauses; maxClauseCount is set to 4096 error in table_search_index for complex search queries in Opensearch
* Fix#21502: TooManyNestedClauses: Query contains too many nested clauses; maxClauseCount is set to 4096 error in table_search_index for complex search queries in Opensearch
* Update openmetadata-service/src/test/java/org/openmetadata/service/resources/search/SearchResourceTest.java
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* fix mvn build
* fix checkstyle
* fix tests in SearchResourceTest
* fix tests in SearchResourceTest
* instead of wrapping baseQuery+functionScore in a new BoolQuery,use functionScore directly as the final query - avoid duplication of mult_match query
* add requestSchema fields in APIEndpointIndex, and remove array fields in searchSettings to use fuzzyfields avoiding clause explosion
* - Set fuzzyMaxExpansions/maxExpansions to 10 instead of default 50
- Turn off fuzziness (Fuzziness.ZERO) for `.ngram` fields
- Update QUERY_SYNTAX_PATTERN so only uppercase AND, OR, NOT are treated as boolean operators
* use non Fuzzy query for ngram Fields in QueryStringBuilder , remove duplication of query in should phrase
* add fuzzy/nonFuzzy Fields for each asset in searchSettings.json
* add fuzzy/nonFuzzy Fields for each asset in searchSettings.json
* update isFuzzyField function
* fix playwright
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* queryParams to the webhook request URL
* add activityThreadsRetentionPeriod in the DataRetentionApplication jsons
* clean up activity threads of type conversation
* Revert "clean up activity threads of type conversation"
This reverts commit e66ea8623be6aa1c5d07afdec04b4807eb21c7e7.
* Revert "add activityThreadsRetentionPeriod in the DataRetentionApplication jsons"
This reverts commit 4180be18a8d5e0d4fd4b3d4662df0532bb16dbed.
* deprecation: remove testCaseResults endpoint from testCaseResource
* fix: path in test e2e test
* fix: endpoint name to testCaseResults
* style: fix java linting
* Fix#21533 - Add a tool to openmetadata-ops.sh to delete orphaned relations
* Fix#21533 - Add a tool to openmetadata-ops.sh to delete orphaned relations
* Add exit 1 and make dry-run default
* Add exit 1 and make dry-run default
* Fix checkstyle
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Adding Data Insights Migrations to remove Stale Objects
* Adding Data Insights Migrations to remove Stale Objects
* Adding dettachment to ES ILM policy from Index and template
* Adding dettachment to ES ILM policy from Index and template
* Adding dettachment to ES ILM policy from Index and template
* Fix Checkstyle
* Moved migrations to 172
* Removed code that was left over on 171 migration
* Improve SearchClient error handling
* Add logic to handle WorkflowContext on Ingestion
* Revert base.py changes
* Removed comment
* Fix basedpyright complaints
* Make ContextManager automatically add its context to the PipelineStatus
* Small changes
* Only dump non-null keys
* Add Method to Filter Ingestion Pipeline based on Metadata
* Reduce the scope to filter only specifically on metadata->workflow->serviceName
