* Fix#21908: Add Persona configuration per User
* Add default persona, add tests for User defaultPersona
* Add default persona, add tests for User defaultPersona
* Fix tests
* Fix#20621: User Status Tracking in the System
* Fix#20621: User Status Tracking in the System
* Fix#20621: Add Playwright tests
* Update generated TypeScript types
* fix import
* remove unused imports and fix broken ui for settings user page
* revert path change for PersonaDetailsPage
* fix unit tests
* fix tests
* Add separate api for last login time
* update last login time api
* remove last login time patch
* remove updateLoginTime
---------
Co-authored-by: shrushti2000 <shrushtipolekar@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* allow changing of glossary term
* Draft ES Index Update for Glossary term move
* ES Upates in a more faster one shot bulk update
* Remove validations on Parent Glossary Hierarchy and Include Circular Glossary Term Validations
* Introduced a new PUT api for glossaryTerms move operation
* Optimised moveGlossaryTerm
* Added Transaction Scope in moveAndStore in updater
* Remove Consolidate and revert changes for move operation
* Improved Change Description and Versioning for move operation
* Improvise request payload to have only parent and optimise the flow
* Implement async functionality of Glossary Terms move
* mvn spotless:apply
* MoveGlossaryTermChannel, Get only the required fields
* Fix: Update Parent for Glossary Term during Bulk Import
* allow glossary term to be moved across glossaries
* update locales
* move socket listening to hierarchy dialog
* Added fqn to move message for the websocket response
* fix tests
* add glossary tests
* cleanup
* fix tests
* Refactor code: Resolve Conflicts
* fix tests
* fix permissions
* cleanup
* Update generated TypeScript types
* Abstraction Logic in place of instanceOf logic for bulkImportExportGlossary to update the parent
* Update generated TypeScript types
---------
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch
* Use native bulk processors
* lower the batch size, add composite index for app_runs table
* while updated secrets, if there is entityNotFoundException ignore and move on
* Fix deadlock
* Fix deadlock
* Fix gzip headers
* remove unnecessary code to check compression, batch sizes etc..
* Improved performance
* Improved performance
* fix merge error
* fix styling
* fix tests
* Fix build issues
* improve queue size recommendations
* put max limits on threads
* fix checkstyle
* Feat: add support of adding tags in test case form
* fix: don't override test case tags with entity tags
* feat: enhance test case form to support glossary terms and improve tag handling
* feat: implement tags and glossary terms functionality in test case forms and enhance related tests
* fixed failing unit test
* feat: add tier tag filtering and preservation in test case forms and related tests
* feat: add tags to addTestCaseAction
---------
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* feat: added endpoint to programatically retrieve index mapping
* feat: added endpoint to programatically retrieve index mapping
* refactor: extracted JsonUtils from server sub module
* refactor: move IndexMapping to spec
* style: ran java linting
* refactor: moved IndexMapping to singleton class
* fix: JsonUtils import
* style: ran java linting
* fix: init indexmapper in operations
* fix: moved ES index json files to spec
* style: ran java linting
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* fix: remove recursive extraction of nested fields for entityReference and entityReferenceList type custom properties
* get displayName field along with other fields in /customProperties api
* show display name instead of name for custom props
* show display name instead of name for custom props
* fix extension type and add tests
* fix tests
* fix data model tests
---------
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* Update Profile Workflow to allow engine configuration
* Add ui generated schemas
* Add Repository Override mechanism based on annotations
* Implement logic to use the ProcessingEngine configuration
* Update SparkEngine to use remote and not master
* Extended relationshipCleanup command to clean up broken entities across all
OpenMetadata service hierarchies (Database, Dashboard, API, Messaging, Pipeline,
Storage, ML Model, and Search services), not just database services.
Key changes:
- Added performServiceHierarchyCleanup() method in OpenMetadataOperations
- Added --skip-hierarchy-cleanup option for backward compatibility
- Added getBrokenRelationFromParentToChild() and deleteBrokenRelationFromParentToChild() methods in CollectionDAO
- Support for detecting and cleaning orphaned entities across any parent-child relationship
- Maintains dry-run and actual cleanup modes with detailed reporting
Usage: relationshipCleanup [--delete] [--skip-hierarchy-cleanup] [--batch-size N
* refactor: extract reusable cleanup utilities and eliminate code duplication
Changes:
- Add ServiceHierarchyCleanup.java for reusable hierarchy cleanup
- Add ComprehensiveCleanupUtil.java for combined cleanup operations
- Refactor OpenMetadataOperations to use reusable utilities
- Refactor DataRetention to use reusable utilities
- Single source of truth for all service hierarchy definitions
* - missing commit
* - Data Retention App test
* Fix query aggregation to use checksum not query text
* Fix query aggregation to use checksum not query text
* fix query text
* fix opensearch class
---------
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
* Fix: Inherited Tags from Glossary Term should be shown in put/patch column response
* Placement of the statement fixed
---------
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster
* Add exact match configs
* Add exact match configs
* Update Logic to build search source builder with exact match priority
* Revert "Update Logic to build search source builder with exact match priority"
This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.
* Revert "Add exact match configs"
This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.
* revert display name change
* make boost mode as sum by defaul
* add more fqnparts for schema and database
* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue
* use fqn split
* refactor fqn parsing
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster
* Add exact match configs
* Add exact match configs
* Update Logic to build search source builder with exact match priority
* Revert "Update Logic to build search source builder with exact match priority"
This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.
* Revert "Add exact match configs"
This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.
* revert display name change
* make boost mode as sum by defaul
* add more fqnparts for schema and database
* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue
* use fqn split
* Refactor FQN Parts
---------
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* add owners field in classification
* fix tests
* add owners field in tags page
* Tag Inherits owners from classification
* Added loadTags.ts for owners field
* add owners for tags page
* fix tests
* review comments
* add owners field in create form
* fix domain test
---------
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Feat(ui): add and edit certifications from asset page (#21344)
* added styling to certification
* added tests
* changed icons for certificatie popup
---------
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* #19529 Improvise import to include certification for Assets (Database, DatabaseSchema, DatabaseService), Introduce EDIT_CERTIFICATION policy and Fix Elastic Search Indexes on Certification (#21564)
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* Applied mvn spotless:apply
* Reused the Existing UPDATE_CERTIFICATION_SCRIPT for ElasticSearch Indexing
* Added field certification in the static String FIELDS
* fix playwright test around bulk action
* #19529 - Persisting Null or empty in the bulk import for certification
* #19529 - Persisting Null or empty in the bulk import for certification - Moved the if block to the top
* mvn spotless:apply
* Reverted an unimportant file
* mvn spotless:apply
* #19529 - Persisting the field Certification in clearFIeldsInternal
* typescript files for edit_certification
* Revert "typescript files for edit_certification"
This reverts commit f5e5514a98008cbd0b62d7cb21fefe61659e97cb.
* typescript files for edit_certification
* mvn:spotless:apply
* Removed correction
* needed typescript file for edit_certification
* Removed Unnecessary Comments
* Improved Test Cases - Added DATA_ASSET_SEARCH alias instead of GLOBAL_ALIAS
* Fixed csv values in order
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* update certification component
* Fixed Certification Indexes
* Fixed Missing Certification Label for DBSchema
* add certification changes
* fix test errors
* Remove Certification field and mapping from database service asset
* Removed certification from schema and fixed test
* fix tests
---------
Co-authored-by: Dhruv Parmar <83108871+dhruvjsx@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <81347100+yan-3005@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
* Correct FQN values will be shown to the user in case of validation failures in glossary term tags
* Applied mvn:spotless apply
* Added Test cases to verify the validation of assets
* Fix#1681 - Deleted Filter in Automations Not Working as Expected
* Revert "Fix #1681 - Deleted Filter in Automations Not Working as Expected"
This reverts commit d941ad440b6c5bcb66d4ac8bb31c321fb23cbcfc.
* Set deleted param in /search/query API to type Boolean to allow default null values
* fix AUT test failures related to user search in settings/users page
* fix AUT test failures related to user search in settings/users page
* Make `deleted` param optional for search API call in explore page
* Fix the deleted users showing on the user listing page after user search
* Fix playwright tests
* Fix playwright tests
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
