* Update Profile Workflow to allow engine configuration
* Add ui generated schemas
* Add Repository Override mechanism based on annotations
* Implement logic to use the ProcessingEngine configuration
* Update SparkEngine to use remote and not master
* add owners field in classification
* fix tests
* add owners field in tags page
* Tag Inherits owners from classification
* Added loadTags.ts for owners field
* add owners for tags page
* fix tests
* review comments
* add owners field in create form
* fix domain test
---------
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Feat(ui): add and edit certifications from asset page (#21344)
* added styling to certification
* added tests
* changed icons for certificatie popup
---------
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* #19529 Improvise import to include certification for Assets (Database, DatabaseSchema, DatabaseService), Introduce EDIT_CERTIFICATION policy and Fix Elastic Search Indexes on Certification (#21564)
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* #19529 - Adding certification field in json schema for databaseService.json, databaseServiceCsvDocumentation.json for bulk import facility, Implemented bulk import for non-recursive databaseService import async
* #19529 -Bulk async import - databaseSchema entity
* #19529 -Bulk async import - database entity
* #19529 - Initial Implementation of Search Indexes for Certificate during bulk Import
* #19529 - Edit Certification Policy Implementation, Attaching it to DataStewards and DataConsumers, also adding it to Migration
* #19529 - Updated ElasticSearch Index mapping name
* #19529 - Updated the CSV indices for recursive import
* #19529 - Single Test working as expected but not as a test suite.
* #19529 - Fixed DatabaseSchemaResourceTest
* #19529 - Fixed DatabaseResource Test for exportImportCSV
* #19529 - Fixed DatabaseServiceResource Test for exportImportCSV
* #19529 - Updated and Improvised ElasticSearch Certification functionality
* #19529 - Added postgres migration as well along with mysql migration
* #19529 - Removed migration scripts from v160 and placed it at v180, Included mappings for database_service in zh and jp,
Applied mvn spotless:apply
* Applied mvn spotless:apply
* Reused the Existing UPDATE_CERTIFICATION_SCRIPT for ElasticSearch Indexing
* Added field certification in the static String FIELDS
* fix playwright test around bulk action
* #19529 - Persisting Null or empty in the bulk import for certification
* #19529 - Persisting Null or empty in the bulk import for certification - Moved the if block to the top
* mvn spotless:apply
* Reverted an unimportant file
* mvn spotless:apply
* #19529 - Persisting the field Certification in clearFIeldsInternal
* typescript files for edit_certification
* Revert "typescript files for edit_certification"
This reverts commit f5e5514a98008cbd0b62d7cb21fefe61659e97cb.
* typescript files for edit_certification
* mvn:spotless:apply
* Removed correction
* needed typescript file for edit_certification
* Removed Unnecessary Comments
* Improved Test Cases - Added DATA_ASSET_SEARCH alias instead of GLOBAL_ALIAS
* Fixed csv values in order
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
* update certification component
* Fixed Certification Indexes
* Fixed Missing Certification Label for DBSchema
* add certification changes
* fix test errors
* Remove Certification field and mapping from database service asset
* Removed certification from schema and fixed test
* fix tests
---------
Co-authored-by: Dhruv Parmar <83108871+dhruvjsx@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <81347100+yan-3005@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: System Administrator <root@192.168.1.4>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
* Fix#1681 - Deleted Filter in Automations Not Working as Expected
* Revert "Fix #1681 - Deleted Filter in Automations Not Working as Expected"
This reverts commit d941ad440b6c5bcb66d4ac8bb31c321fb23cbcfc.
* Set deleted param in /search/query API to type Boolean to allow default null values
* fix AUT test failures related to user search in settings/users page
* fix AUT test failures related to user search in settings/users page
* Make `deleted` param optional for search API call in explore page
* Fix the deleted users showing on the user listing page after user search
* Fix playwright tests
* Fix playwright tests
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Fix: Remove operations and glossaryTags from bulk asset add api
* Fix: Improved the test case
* addTagToAssets schema - Removed operations field in the payload
* Replacing the request payload for validating tags in the UI
* UI changes for add and remove assets to glossary term
* Using getByName and Entity.GLOSSARY
---------
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* SCIM interface
* removed unwanted code
* remove SCIM registration in OM
* reverted scim configuration settings, as it is not needed
* Added security context to create user
* Added scimusername in user jsons
* added externalid, scimUsername in createUser
* Added security context on create and update groups
* Added jakarta imports
* Authorization added
* Added role, policy and bot for SCIM
* refactor: removed testSuite field from CreateTestCase
BREAKING CHANGE: when creating a test case, testsuite is now derived from entityLink (fetch or created)
* feat: allow setting tags when creating a test case
* style: ran linters
* fix: compiling error
* fix: failing test case
* fix: failing tests
* removed testSuite from required filed
* fixed ui side
* style: ran java linting
* deprecation: remove testSuite param from ingestion
* fix: remove test suite filed
* fix: remove test_suite field
---------
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
* Update Classification Schema to include Governance Fields
* Removed Tags, Reviewers, Domain from Classification as they are needed and corrected tests
* Added Permission check for owners in Classification Resource Test
* Added LoadTags.ts generated from createClassificationSchema.json
* Only have my schema changes in the typescript files, ignore other changes.
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* Fix: Remove operations and glossaryTags from bulk asset add api
* Fix: Improved the test case
* addTagToAssets schema - Removed operations field in the payload
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* queryParams to the webhook request URL
* add activityThreadsRetentionPeriod in the DataRetentionApplication jsons
* clean up activity threads of type conversation
* Revert "clean up activity threads of type conversation"
This reverts commit e66ea8623be6aa1c5d07afdec04b4807eb21c7e7.
* Revert "add activityThreadsRetentionPeriod in the DataRetentionApplication jsons"
This reverts commit 4180be18a8d5e0d4fd4b3d4662df0532bb16dbed.
* Update the file upload option input for service config
* Separate out the field rendering for 'uiFieldType' as fileOrInput and file.
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* Add logic to handle WorkflowContext on Ingestion
* Revert base.py changes
* Removed comment
* Fix basedpyright complaints
* Make ContextManager automatically add its context to the PipelineStatus
* Small changes
* Issue-19251: Upgrade dropwizard to 4.x and Jetty to 11.x
* keep jdbi3.37.1 version for ConnectionAware functionality
* Code builds
* Merge with main and fix MCP
* Fix to make server run
* Tests fix
* JsonArray causing issue
* Fix WebSockets
* fix json2schema using jakarta instead of javax
* Fix patch tests
* Fix tests
* Register auditing subtypes in OpenmeteadataOperations
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* feat: prep for collate support
- Added support for delayed background jobs. This is to delete the support user after a while.
- added ScheduleType.OnlyManual for applications to support only manual triggers
- Added a four hour token expiration
- Allow a null trigger type in job data (not sure what it's for)
* added test for delayed job
* added type checking for enumCleanupArgs in navbar
* - added a support toten type
- added job handler for delete token
* fixed whitelist links
* patch OnlyManual apps
* Update openmetadata-spec/src/main/resources/json/schema/entity/applications/app.json
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* added indexes for runAt
* use NO_JOB_SLEEP_SECONDS in background job test
* Fixed NavBar test
* removed backticls
* fix indexes
* change for support app
* fix tests
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* wip
* feat: trigger external apps with override config
- Added in openmetadata-airflow-apis functionality to trigger DAG with feature.
- Modified openmetadata-airflow-apis application runner to accept override config from params.
- Added overloaded runPipeline with `Map<String,Object> config` to allow triggering apps with configuration. We might want to expand this to all ingestion pipelines. For now its just for apps.
- Implemented an example external app that can be used to test functionality of external apps. The app can be enabled by setting the `ENABLE_APP_HelloPipelines=true` environment variable.
* fix class doc for application
* fixed README for airflow apis
* fixes
* set HelloPipelines to disabeld by default
* fixed basedpywright errros
* fixed app schema
* reduced airflow client runPipeline to an overload with null config
removed duplicate call to runPipeline in AppResource
* Update openmetadata-docs/content/v1.7.x-SNAPSHOT/developers/applications/index.md
Co-authored-by: Matias Puerta <matias@getcollate.io>
* deleted documentation file
---------
Co-authored-by: Matias Puerta <matias@getcollate.io>
* Adds followers to services and database schema
* Add delete and followers fields
* Add missing fields
* Fix failing test for get entity with different fields
* Fix ingestion test case failure
