1831 Commits

Author SHA1 Message Date
Sriharsha Chintalapani
a924064c09
Fix #17723: Generate Incremental Change Events even when consolidation of events applied (#19550)
* Fix #17723: Generate Incremental Change Events even when consolidation of events applied

* Fix #17723: Generate Incremental Change Events even when consolidation of events applied

* fix tests

* Fix tests

* clean policy tests

* update search methods to use incrementalChangeDescription part-1

* Fix the version page playwrights

* update search methods to use incrementalChangeDescription part-2

* introduce new field incrementalChangeDescription for search part-3

* fix mvn endpoint test

* fix followers and page search test

* fix following of assets

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2025-02-20 10:23:08 +05:30
Imri Paran
3b5218b923
MINOR: feat(event-subscriptions): event consumer extension (#19591)
* feat(event-subscriptions): event consumer extension

- implemented DI for event consumers
- added custom config for event consumers
- initialize the event scheduler along with application
- added workflow custom type

* - removed the ConsumerService as it is specific for each consumer implementation
- added a "custom" alert type

* wip

* cleanup

* reverted pom.xml changes

* reverted pom.xml changes

* reverted pom.xml changes

* removed abstract layer AbstractOmAppJobListener

* fixed import AbstractOmAppJobListener -> OmAppJobListener

* fixed constructor

* - add event subscription if it doesnt exist (set operation)
- add deleteEventSubscription

* - use patch mode when updating app.eventSubscriptions

* added docs for DIContainer

* aligned py tests with new automation schema

* added AppRunRecord.properties

* format

* changes after merge

* generated missing ts

* use openmetadata config builder instead of the config

* add json logic fixes for reverse metadata

* ref: reverted changes ro workflows

they will be part of https://github.com/open-metadata/OpenMetadata/pull/19723

* ref: reverted changes to workflows

they will be part of https://github.com/open-metadata/OpenMetadata/pull/19723

* reverted ts changes

* reverted ts changes

* reverted ts changes

* fix: initialization sequence for event subscription scheduler

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-19 16:23:46 +01:00
IceS2
9d51add739
MINOR: Add run app task (#19841)
* Add Run App Task

* Work on Run App Task

* Rollback task changes

* Fixing type convertion

* Fix Run External App wait

* Fix Run App Task

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-19 13:49:46 +01:00
sonika-shah
9c59f81234
improve log message for registering new type in db (#19868) 2025-02-19 10:51:02 +05:30
Mohit Yadav
e674669431
Fix Deleted filter (#19835)
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-19 00:33:38 +05:30
Mohit Yadav
250e130e61
Fix tests (#19866)
* Update expected values from TestconnectionDefinition test

* Allow get dao to be injected

---------

Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2025-02-19 00:19:59 +05:30
Mohit Yadav
fce358eaa2
Remove and read alert on sync offset (#19862) 2025-02-18 18:36:58 +05:30
Mayur Singal
546a5583f3
MINOR: Optimise Pipeline Lineage Extraction (#19608) 2025-02-18 11:32:44 +05:30
Sriharsha Chintalapani
eb1dfca8bc
Fix json smart version (#19826) 2025-02-17 07:35:05 -08:00
sonika-shah
63eb9c3074
Fix : Add typeRegistry steps when reindexing by ops (#19820)
* Fix : Add typeRegistry steps when reindexing by ops

* change to log.error in catch
2025-02-17 14:47:24 +05:30
Sriharsha Chintalapani
3ae2ff1c1c
Fix install appplication (#19819) 2025-02-17 14:00:26 +05:30
Mohit Yadav
37002220c8
Alert Fixes (#19574)
* Fix Alert and Mapper

* Update to current latest

* Add Event Subscription Exception

* Removed unnecessary calls to database

* Add api for syncing offset

* Add Sync option for alerts in operations
2025-02-16 20:48:59 -08:00
Akash Jain
9111d98eec
fix: openmetadata-service/pom.xml to reduce vulnerabilities (#19773)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-8707740
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-8707739

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2025-02-16 12:00:17 -08:00
IceS2
2cb51f71cd
Small refactor to be able to print correct message when reviewer is invalid (#19810) 2025-02-16 11:58:55 -08:00
IceS2
fa2d9eb81d
MINOR: Update Glossary Approval Workflow (#19787)
* Update Glossary Approval Workflow

* Update Default Workflows

* Add small Fix to PeriodicBatchEntity variables
2025-02-14 17:57:51 +01:00
Gyowanny P. Queiroz
9426a8a87f
MINOR - Fixes relationship between IngestionPipeline and IngestionAgent entities (#19790)
* MINOR - Fix relationship between IngestionPipeline and IngestionAgent entities

* Add null check

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-02-14 16:08:37 +01:00
Pere Miquel Brull
ea1d2c700d
FIX #19768 - Delete pipelines from logical suites at deletion (#19792)
* FIX #19768 - Delete pipelines from logical suites at deletion

* FIX #19768 - Delete pipelines from logical suites at deletion

* FIX #19768 - Delete pipelines from logical suites at deletion

* FIX #19768 - Delete pipelines from logical suites at deletion
2025-02-14 14:33:01 +01:00
IceS2
46948601db
MINOR: Add create ingestion pipeline task (#19753)
* Use namespaced variables

* Fix test

* Add LOG, remove uneeded catch

* Add Log, Refactor VariableHandler

* Add Validation at Definition time

* Add migration

* Remove comments

* Add Create Ingestion Pipeline Task

* Fix migrations by not setting deployed if workflowHandler is not initialized

* Add RunIngestionPipeline Task

* Add new flowable nodes

* Add Job Lock Time configuration default - 5 mins

* fix checkstyle

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-13 13:10:17 +01:00
dechoma
0e7010a154
fix announcement owner or follower (#19548) 2025-02-13 10:23:13 +05:30
mgorsk1
337108e212
🎉 Init (#19377) 2025-02-13 09:00:09 +05:30
Gyowanny P. Queiroz
2559c79edd
MINOR - Hybrid Ingestion Agent - Link ingestion agents to ingestion pipelines (#19751)
* Hybrid Ingestion Agent - Link ingestion agents to ingestion pipelines

* Store relationship as HAS types

* Java checkstyle

* Trigger CI
2025-02-12 14:31:51 +01:00
Pere Miquel Brull
ede8108a5a
MINOR - Validate basic suites do have basicEntityRef (#19763)
* MINOR - Validate basic suites do have basicEntityRef

* add test
2025-02-12 14:07:16 +01:00
Mayur Singal
e873ba0f1b
Fix #19201: Fix connection password masking (#19749) 2025-02-12 16:16:25 +05:30
IceS2
b120748656
MINOR: Use namespaced variables (#19648)
* Use namespaced variables

* Fix test

* Add LOG, remove uneeded catch

* Add Log, Refactor VariableHandler

* Add Validation at Definition time

* Add migration

* Remove comments

* Fix migrations by not setting deployed if workflowHandler is not initialized
2025-02-11 17:26:32 +05:30
sonika-shah
c0eb7d08de
GEN -19588 Sort Enum type Custom Property Values (#19637)
* GEN -19588 Sort Enum type Custom Property Values

* fix py-tests

* use streams for sorting
2025-02-11 14:29:01 +05:30
IceS2
7e6fc02585
Remove description check for columnDescriptionStatus (#19715) 2025-02-11 12:43:25 +05:30
Mohit Yadav
08de322e34
Fixes Email Issue in system repository being null (#19733) 2025-02-11 12:36:12 +05:30
Sriharsha Chintalapani
7e7691df15
Remove unused json-smart dependency (#19701)
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-02-10 20:36:10 -08:00
Imri Paran
5f73dd717b
Fix 19499: feat(apps): add app config in run record (#19544)
* feat(apps): add app config in run record

- changed returned schema type of "status" to "AppRunRecord"!
- added a "config" field in the app record
- added a "pending" stage so that "external" app status can be mapped ti internal app status

* added note about schema compatiblity

* added app config for external apps

* show config for every app run

* add tests

* revert open change for now

* Revert "revert open change for now"

This reverts commit e1492a5c92d1967b3d47230a25df4c2c3d399de2.

* update dependency

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-07 15:21:58 +01:00
IceS2
c34b612df3
Fix followers for Data Insights index (#19693) 2025-02-07 13:47:48 +01:00
Sriharsha Chintalapani
be27f3e8fc
User Permissions on TestSuite and TestCases (#19072)
* issue-11002: user with Delete permission not able to delete testCases

* Fix creation of testSuite if there are conditions in policies

* issue-19072: User Permissions on TestSuite and TestCases

* replace `executable` with `basic`

* Improvement: Revamp the permissions for testCase & testSuite details page (#19137)

* Feat: revamp the permissions in test case and test suite details page

* minor fixes

* added permission checks for test case details page

* Add multiple permission check to Authorizer

* fixed failing unit test

* fix: incident and results permission

* added permission checks for incident manager

* fix: failing java tests

* allow edit the test case if table has edit permission for test case

* fix: test case detail + test suite resource

* fix: test suite PUT and PATCH permission

* style: ran java linting

* style: ran java linting

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2025-02-05 14:05:53 +01:00
Imri Paran
74c69a157a
chore(apps): reset error triggers on schedule (#19606) 2025-02-05 10:21:52 +01:00
dechoma
2308d4a4f3
remove object aliasing from buildReceiversListFromActions (#19495)
* remove object aliasing from buildReceiversListFromActions

* fix linting

---------

Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2025-02-04 10:39:04 +05:30
Mohit Yadav
6fba495daf
[MINOR] Fixes String value length (20054016) exceeds the maximum allowed (20000000, (#19643)
* Fixes Caused by: com.fasterxml.jackson.databind.JsonMappingException: String value length (20054016) exceeds the maximum allowed (20000000, from

* Update log message
2025-02-04 10:24:33 +05:30
mohitdeuex
04cab8e71a Revert "- Update lineage model and api"
This reverts commit c5cce2b0c7c1be7ba654bfc2677da0d05215a895.
2025-02-03 12:11:07 +05:30
mohitdeuex
03f492ad1d Revert "- Adds updatedAt and updatedBy"
This reverts commit c6b462f0ab5905fe339c08a05ab819ca940dd06d.
2025-02-03 12:11:07 +05:30
Mohit Yadav
78123db16c
Merge branch 'lineage-work' into main 2025-02-03 11:43:01 +05:30
sonika-shah
d067121fa3
Fix #17059 - Task cannot be closed by the non-admin user who created it (#19528)
* Fix #17059 - Task cannot be closed by the non-admin user who created it

* uncommented playwright test around the flow

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-02-03 11:04:50 +05:30
mohitdeuex
c6b462f0ab - Adds updatedAt and updatedBy 2025-02-02 23:13:15 +05:30
mohitdeuex
c5cce2b0c7 - Update lineage model and api 2025-02-02 22:46:28 +05:30
sonika-shah
773982210d
fix : aggregation issue in opensearch while fetching domain hierarchy (#19594) 2025-01-29 19:14:41 +05:30
Mohit Yadav
8c5eee8f07
Add prometheus counter for search and database (#19561)
* Add prometheus counter for search and dartabase

* chenge env

* typo - review comments
2025-01-29 00:08:45 +05:30
Mohit Yadav
58d5988a16
Add Onboarding SQL (#19525)
* Add Primary Key Migration

* Update queries and conflict on apps data store

* Update queries and conflict on apps data store

* add index on isBot
2025-01-28 11:45:58 +05:30
Teddy
36d8519b09
feat: add entityFQN query param for test case (#19541) 2025-01-27 17:51:06 +01:00
Mayur Singal
eea4b65b80
Fix #19198: di fix custom property filter (#19531) 2025-01-27 15:54:41 +05:30
sonika-shah
5b63c36ef1
Fix #19116 : Support for domain hierarchy listing (#19191)
* Backend support for domain hierarchy listing

* use linkedHashmap to maintain order of results in hierarchy

* Revert changes to generated files to match main

* show all domains for non admin user

* change select domain layout

* fix multiple save

* fix playwright

* cleanup

* fix domain tests

* fix domain tests

* fix icon styling

* show icons on navbar domain list

* update tests

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-01-27 15:43:42 +05:30
Imri Paran
5482b94a57
feat(schedule-application): enable scheduling application using config file (#19470)
- added schedule to app private config
- reschedule a schedule application job in quartz if it exists.
2025-01-27 13:39:46 +05:30
sonika-shah
82198f8126
GEN -19341 : Improvement - Avoid usage of CONCAT in WHERE clause (#19510)
* Improvement : Avoid usage of CONCAT in WHERE clause

* fix tests

* remove usage of @BindFqnConcat

* remove usage of @BindFqnConcat

* fix test

* fix test
2025-01-27 11:20:25 +05:30
Siddhant
473688648b
Fix #18834: Introduce new setting OpenMetadataBaseUrlConfiguration to set the server endpoint (#19366)
* Refactor: Introduce openMetadataBaseUrlConfiguration to set up the server endpoint and remove OpenMetadataUrl from smtpSettings.json

* migrations.

* refactor(cli): add setOpenMetadataUrl command to store OpenMetadata URL

* refactor(cli): add configureEmailSettings command to manage SMTP/Email configuration

* load initialData when config is not present in the db

* add om url settings page

* update locales

* add e2e

* load omBaseUrl with default host and port: localhost:8585 when not present

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2025-01-27 10:20:04 +05:30
Imri Paran
a6670af500
MINOR: Use interface schemas for governance workflows (#19411)
* feat(apps): support event subscriptions

- added support for apps with event subscriptions.
- added support for custom consumers on event subscriptions.
- use native application methods instead of reflection in ApplicationHandler

* removed runMethodFromApplication reflection and use concrete methods from AbstractNativeApplication

* format

* fix(governance-workflows): defined interface in workflow schema

- defined types for workflow nodes
- fixed updateEdge method in WorkflowDefinitionRepository

* ref(governance-workflows): use explicit types in schema

use explicit interface type in json schemas and use annotations to infer the concrete classes when deserializing

* - implemented CustomSignal trigger
- use JsonTypeInfo and JsonSubTypes to infer types in jsonschema for nodes and triggers
- Implemented JsonLogicFilter task
- implemented placeholder noop task

* - simplified trigger type names
- removed governance workflow root "type"
- added "deployed" field to governance workflow
- applied changes to existing gov-workflows
- migrations

* fixed migrations
2025-01-24 09:08:06 +01:00