* Initial implementation for Governance Workflows
* Remove file that shouldn't be there
* Initial implementation for the governance workflows
* Refactor JSONSchema for Governance Workflows
* Refactored WorkflowInstanceStatus to keep the stage properly
* Fix Migration for MySQL for the WorkflowInstanceStates change
* Update WorkflowInstanceState to track all the states propertly
* Initial code tro migrate Request type of Tasks
* copy governance schemas into json directory
* update schema names
* Extract WorkflowInstance into its own resource
* Update WorkflowInstance to have the workflow starting variables
* Changed relatedEntity to be an EntityLink
* First iteration on checkEntityAttributes JSONLogic
* Refactored Workflow into TriggerWorkfow and MainWorkflow
* Fixed WorkflowInstance and WorkflowInstanceState
* Fixed WorkflowInstance and WorkflowInstanceState when process is terminated
* Fix UserTask
* json logic changes to workflow
* Fix Patch request. Add exclude filter to EventBasedEntityTrigger
* update query builder
* Fix Migrations
* Fix Migrations
* modify query builder widget
* fix search props
* update widget and use default fields
* Update edges to always have a true/false condition
* made changes to provider for fields
* push progress
* fix grouping of array field
* Remove duplicated migrations
* Fix migrations
* Fix migrations
* Fix Tests and commit some improvements suggestion
* fix unit tests
* Add Certification Workflow
* Fix Tests
* Fix Tests
* fix tests
* add delay in pagination tests
* fix minor config changes for json logic
* change the glossary page
* Remove Table Certification Workflow seed Data
* add json logic and elastic search parsers
* minor schedule interval changes for workflows
* minor fixes to the parsing logic
* certification ui
* fix tests
* fix tests
* add other search fields
* Add Certification to search index
* Update index mapping for certification
* add quick filter for certification
* update locales
* Fix index mapping
* Add missing index updates
* Fix checkstyle
* increase timeout
* skipping user test
* skipping team test
---------
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
* Search RBAC: process multiple view policies that can grant access to different set of entities
* Search RBAC: process multiple view policies that can grant access to different set of entities
* Search RBAC: process multiple view policies that can grant access to different set of entities
* Improve parallelization in SearchIndexApp
* Improve parallelization in SearchIndexApp
* add countdown latch
* typo
* Status issues - #1
* Improve search indexing
* cleanup the code
* schemas
* Status Updates Via WebSockets
* Add Updates
* Make Logs Debug
* show duration in table
* add searchIndexApp params to openmetadata-ops.sh reindex commandline
* Fix code style
* Add Parallel Entity Level Reader
* Make code more readable
* Send App Run Record instead of jobData
* Sned App Run Record WebSocket for Data Insights
* ui: use socket connection to get the real-time updates of application data
* Send WebSocket Final Update
* ui: remove failedRecords, totalRecords and successRecordsfrom entities record
* Invert Condition
---------
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
* fix(redshift-system): redshift return type
* fixed bigquery profiler
* fixed snowflake profiler
* job id action does not support matrix. using plain action summary.
* reverted gha change
* feat(data-quality): use sampling config in data diff
- get the table profiling config
- use hashing to sample deterministically the same ids from each table
- use dirty-equals to assert results of stochastic processes
* - reverted missing md5
- added missing database service type
* - use a custom substr sql function
* fixed nounce
* added failure for mssql with sampling because it requires a larger change in the data-diff library
* fixed unit tests
* updated range for sampling
* Add default ChangeEvent template and Slack destination test template
* fix conflict
* initial ingestionPipeline template
* clean templates.
* clean templates
* teams templates.
* add getBoldWithSpace()
* java check style
* google chat templates.
* Refactor GChat and Teams message for template data handling for DQ_Template_Section and General_Template_Section.
* Refactor slack message for template data handling for DQ_Template_Section and General_Template_Section.
* improvements gChatMessageDecorator
* improvements slackMessageDecorator
* improve MSTeamsMessageDecorator
* Call templates per entityType and refactor code
* Update EmailPublisher to use testEmail template instead of changeEvent template for sendTestMessage
* add parameterValues sections for teams
* Update SlackMessage to have attachments.
Update DQ template.
* Update dq templates, build dq template data in MessageDecorator
* remove IngestionPipeline template
* move SlackCallbackResource into a separate class. Fix tests.
* simulate timeout and 300.
* remove the validation of messages.
* fix teams MessageDecorator
* remove unused code from SlackMessageDecorator
* fix owners and tags data population in the template
* fix: changes in test case and test case result should load different templates.
* fix: list records from latest to earliest.
* refactor: remove listing unprocessed events from listEvents API.
* refactor: EventSubscriptionDiagnosticInfo.
* Add export apis back and new apis should be called exportAsync
* ui: use exportAsync endpoint for export assets
* fix build failure
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
* fix getSuccessfullySentChangeEventsForAlert
* migrations.
* offset-based pagination
* add startingOffset in EventSubscriptionDiagnosticInfo
* fix missing paginationOffset
* Record successfully sent change events in a separate table, clean up associated successful and failed records on alert deletion.
* Remove oldest 30 records if record count reaches 100 for successful_sent_change_events.
* list unprocessed changeEvents based on alert subscription rules, ensuring only relevant unprocessed events are listed.
* refactor: enhance eventSubscriptionDiagnosticInfo
Added new fields to the eventSubscriptionDiagnosticInfo schema to support detailed diagnostics:
- `relevantUnprocessedEventsCount`: Count of relevant unprocessed events based on alert filtering rules.
- `relevantUnprocessedEventsList`: List of relevant unprocessed events based on alert filtering rules.
- `totalUnprocessedEventsCount`: Count of all unprocessed events.
- `totalUnprocessedEventsList`: List of all unprocessed events.
* Refactor: delete 30 oldest records per alert when the count reaches 100 records
* fix: table creation issue in PostgreSQL
* fix tests.
* scheduleCleanupJob
* remove in-line deletion of records.
* fix postgres table structure issue.
* feat(statistics-profiler): use statistics tables to profile trino tables
- implemented the collaborative root class
- added the "useStatistics" profiler parameter
- added the "supportsStatistics" database connection property
- implemented the ProfilerWithStatistics and StoredStatisticsSource to add this functionality to specific profilers
- implemented TrinoStoredStatisticsSource for specific trino statistics logic
* added ABC to terminal classes in collaborative root
* fixed docstring for TestSuiteInterface
* reverted unintended changes
* typo
