2141 Commits

Author SHA1 Message Date
Imri Paran
97fad806a2
Fixes 19755: Publish app config with status (#19754)
* feat(app): add config to status

add config to the reported status of the ingestion pipeline

* added separate pipeline service client call for external apps

* fix masking of pydantic model

* - overload model_dump to mask secrets instead of a separate method
- moved tests to test_custom_pydantic.py

* fix: execution time

* fix: mask secrets in dump json

* fix: for python3.8

* fix: for python3.8

* fix: use mask_secrets=False when dumping a model for create

* format

* fix: update mask_secrets=False for workflow configurations

* fix: use context directly when using model_dump_json

* fix: default behavior when dumping json

* format

* fixed tests
2025-02-25 16:51:49 +00:00
Keshav Mohta
df193bd0c4
Feature: Reverse Metadata Ingestion (#19723) 2025-02-25 18:41:08 +05:30
IceS2
a0e91f3dea
MINOR: Fix Certification Comparisson (#19947)
* Used .equals() instead of ==

* fix comparison

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
2025-02-25 14:04:13 +01:00
Ayush Shah
8550954150
Fix: Update summary and description for addCustomMetric operation (#19964) 2025-02-25 16:56:21 +05:30
sonika-shah
054d317a8d
Minor : get domain deletion activity feed update (#19961) 2025-02-25 14:48:08 +05:30
Sriharsha Chintalapani
a41c14c546
Add cmd line user ops (#19959)
* Add create-user/update-password commands

* Fix #19958: Add create user option to openmetadata-ops.sh
2025-02-24 18:51:40 -08:00
IceS2
2ac8670c98
MINOR: Add day one experience app (#19883)
* Add Day One Experience App JSONSchema

* Add 'uninstall' method to applications, that should run on deleteApp

* Fix JSONSchemas

* Treating the case where we are trying to suspend or resume a workflow already in that state

* Fix JsonSchema
2025-02-24 13:53:23 +01:00
Pere Miquel Brull
c207c8d4f3
MINOR - Fix collate app yaml reader with unwanted dropwizard java props (#19908) 2025-02-21 07:34:07 +01:00
sonika-shah
b341d5e1cb
fix - 17995 Avoid duplicate teams in team listing hierarchy (#19844)
* fix - 17995 Avoid duplicate teams in team listing hierarchy

* added check in playwright to see if the team is not repeated

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-02-20 16:44:22 +05:30
sonika-shah
aefc36b596
GEN 16908 - Support pagination for children field (#19650)
* GEN 16908 - Support pagination for children field

* Fix tests - Support pagination for children field

* move children pagination listing to separate api

* added pagination support from UI

* added playwright test for the pagination test

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-02-20 16:43:30 +05:30
Sriharsha Chintalapani
a924064c09
Fix #17723: Generate Incremental Change Events even when consolidation of events applied (#19550)
* Fix #17723: Generate Incremental Change Events even when consolidation of events applied

* Fix #17723: Generate Incremental Change Events even when consolidation of events applied

* fix tests

* Fix tests

* clean policy tests

* update search methods to use incrementalChangeDescription part-1

* Fix the version page playwrights

* update search methods to use incrementalChangeDescription part-2

* introduce new field incrementalChangeDescription for search part-3

* fix mvn endpoint test

* fix followers and page search test

* fix following of assets

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2025-02-20 10:23:08 +05:30
Imri Paran
3b5218b923
MINOR: feat(event-subscriptions): event consumer extension (#19591)
* feat(event-subscriptions): event consumer extension

- implemented DI for event consumers
- added custom config for event consumers
- initialize the event scheduler along with application
- added workflow custom type

* - removed the ConsumerService as it is specific for each consumer implementation
- added a "custom" alert type

* wip

* cleanup

* reverted pom.xml changes

* reverted pom.xml changes

* reverted pom.xml changes

* removed abstract layer AbstractOmAppJobListener

* fixed import AbstractOmAppJobListener -> OmAppJobListener

* fixed constructor

* - add event subscription if it doesnt exist (set operation)
- add deleteEventSubscription

* - use patch mode when updating app.eventSubscriptions

* added docs for DIContainer

* aligned py tests with new automation schema

* added AppRunRecord.properties

* format

* changes after merge

* generated missing ts

* use openmetadata config builder instead of the config

* add json logic fixes for reverse metadata

* ref: reverted changes ro workflows

they will be part of https://github.com/open-metadata/OpenMetadata/pull/19723

* ref: reverted changes to workflows

they will be part of https://github.com/open-metadata/OpenMetadata/pull/19723

* reverted ts changes

* reverted ts changes

* reverted ts changes

* fix: initialization sequence for event subscription scheduler

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-19 16:23:46 +01:00
IceS2
9d51add739
MINOR: Add run app task (#19841)
* Add Run App Task

* Work on Run App Task

* Rollback task changes

* Fixing type convertion

* Fix Run External App wait

* Fix Run App Task

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-19 13:49:46 +01:00
sonika-shah
9c59f81234
improve log message for registering new type in db (#19868) 2025-02-19 10:51:02 +05:30
Mohit Yadav
e674669431
Fix Deleted filter (#19835)
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-19 00:33:38 +05:30
Mohit Yadav
250e130e61
Fix tests (#19866)
* Update expected values from TestconnectionDefinition test

* Allow get dao to be injected

---------

Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2025-02-19 00:19:59 +05:30
Mohit Yadav
fce358eaa2
Remove and read alert on sync offset (#19862) 2025-02-18 18:36:58 +05:30
Mayur Singal
546a5583f3
MINOR: Optimise Pipeline Lineage Extraction (#19608) 2025-02-18 11:32:44 +05:30
Sriharsha Chintalapani
eb1dfca8bc
Fix json smart version (#19826) 2025-02-17 07:35:05 -08:00
sonika-shah
63eb9c3074
Fix : Add typeRegistry steps when reindexing by ops (#19820)
* Fix : Add typeRegistry steps when reindexing by ops

* change to log.error in catch
2025-02-17 14:47:24 +05:30
Sriharsha Chintalapani
3ae2ff1c1c
Fix install appplication (#19819) 2025-02-17 14:00:26 +05:30
Mohit Yadav
37002220c8
Alert Fixes (#19574)
* Fix Alert and Mapper

* Update to current latest

* Add Event Subscription Exception

* Removed unnecessary calls to database

* Add api for syncing offset

* Add Sync option for alerts in operations
2025-02-16 20:48:59 -08:00
Akash Jain
9111d98eec
fix: openmetadata-service/pom.xml to reduce vulnerabilities (#19773)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-8707740
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-8707739

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2025-02-16 12:00:17 -08:00
IceS2
2cb51f71cd
Small refactor to be able to print correct message when reviewer is invalid (#19810) 2025-02-16 11:58:55 -08:00
IceS2
fa2d9eb81d
MINOR: Update Glossary Approval Workflow (#19787)
* Update Glossary Approval Workflow

* Update Default Workflows

* Add small Fix to PeriodicBatchEntity variables
2025-02-14 17:57:51 +01:00
Gyowanny P. Queiroz
9426a8a87f
MINOR - Fixes relationship between IngestionPipeline and IngestionAgent entities (#19790)
* MINOR - Fix relationship between IngestionPipeline and IngestionAgent entities

* Add null check

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-02-14 16:08:37 +01:00
Pere Miquel Brull
ea1d2c700d
FIX #19768 - Delete pipelines from logical suites at deletion (#19792)
* FIX #19768 - Delete pipelines from logical suites at deletion

* FIX #19768 - Delete pipelines from logical suites at deletion

* FIX #19768 - Delete pipelines from logical suites at deletion

* FIX #19768 - Delete pipelines from logical suites at deletion
2025-02-14 14:33:01 +01:00
IceS2
46948601db
MINOR: Add create ingestion pipeline task (#19753)
* Use namespaced variables

* Fix test

* Add LOG, remove uneeded catch

* Add Log, Refactor VariableHandler

* Add Validation at Definition time

* Add migration

* Remove comments

* Add Create Ingestion Pipeline Task

* Fix migrations by not setting deployed if workflowHandler is not initialized

* Add RunIngestionPipeline Task

* Add new flowable nodes

* Add Job Lock Time configuration default - 5 mins

* fix checkstyle

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-13 13:10:17 +01:00
dechoma
0e7010a154
fix announcement owner or follower (#19548) 2025-02-13 10:23:13 +05:30
mgorsk1
337108e212
🎉 Init (#19377) 2025-02-13 09:00:09 +05:30
Gyowanny P. Queiroz
2559c79edd
MINOR - Hybrid Ingestion Agent - Link ingestion agents to ingestion pipelines (#19751)
* Hybrid Ingestion Agent - Link ingestion agents to ingestion pipelines

* Store relationship as HAS types

* Java checkstyle

* Trigger CI
2025-02-12 14:31:51 +01:00
Pere Miquel Brull
ede8108a5a
MINOR - Validate basic suites do have basicEntityRef (#19763)
* MINOR - Validate basic suites do have basicEntityRef

* add test
2025-02-12 14:07:16 +01:00
Mayur Singal
e873ba0f1b
Fix #19201: Fix connection password masking (#19749) 2025-02-12 16:16:25 +05:30
IceS2
b120748656
MINOR: Use namespaced variables (#19648)
* Use namespaced variables

* Fix test

* Add LOG, remove uneeded catch

* Add Log, Refactor VariableHandler

* Add Validation at Definition time

* Add migration

* Remove comments

* Fix migrations by not setting deployed if workflowHandler is not initialized
2025-02-11 17:26:32 +05:30
sonika-shah
c0eb7d08de
GEN -19588 Sort Enum type Custom Property Values (#19637)
* GEN -19588 Sort Enum type Custom Property Values

* fix py-tests

* use streams for sorting
2025-02-11 14:29:01 +05:30
IceS2
7e6fc02585
Remove description check for columnDescriptionStatus (#19715) 2025-02-11 12:43:25 +05:30
Mohit Yadav
08de322e34
Fixes Email Issue in system repository being null (#19733) 2025-02-11 12:36:12 +05:30
Sriharsha Chintalapani
7e7691df15
Remove unused json-smart dependency (#19701)
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-02-10 20:36:10 -08:00
Imri Paran
5f73dd717b
Fix 19499: feat(apps): add app config in run record (#19544)
* feat(apps): add app config in run record

- changed returned schema type of "status" to "AppRunRecord"!
- added a "config" field in the app record
- added a "pending" stage so that "external" app status can be mapped ti internal app status

* added note about schema compatiblity

* added app config for external apps

* show config for every app run

* add tests

* revert open change for now

* Revert "revert open change for now"

This reverts commit e1492a5c92d1967b3d47230a25df4c2c3d399de2.

* update dependency

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-02-07 15:21:58 +01:00
IceS2
c34b612df3
Fix followers for Data Insights index (#19693) 2025-02-07 13:47:48 +01:00
Sriharsha Chintalapani
be27f3e8fc
User Permissions on TestSuite and TestCases (#19072)
* issue-11002: user with Delete permission not able to delete testCases

* Fix creation of testSuite if there are conditions in policies

* issue-19072: User Permissions on TestSuite and TestCases

* replace `executable` with `basic`

* Improvement: Revamp the permissions for testCase & testSuite details page (#19137)

* Feat: revamp the permissions in test case and test suite details page

* minor fixes

* added permission checks for test case details page

* Add multiple permission check to Authorizer

* fixed failing unit test

* fix: incident and results permission

* added permission checks for incident manager

* fix: failing java tests

* allow edit the test case if table has edit permission for test case

* fix: test case detail + test suite resource

* fix: test suite PUT and PATCH permission

* style: ran java linting

* style: ran java linting

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2025-02-05 14:05:53 +01:00
Imri Paran
74c69a157a
chore(apps): reset error triggers on schedule (#19606) 2025-02-05 10:21:52 +01:00
dechoma
2308d4a4f3
remove object aliasing from buildReceiversListFromActions (#19495)
* remove object aliasing from buildReceiversListFromActions

* fix linting

---------

Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2025-02-04 10:39:04 +05:30
Mohit Yadav
6fba495daf
[MINOR] Fixes String value length (20054016) exceeds the maximum allowed (20000000, (#19643)
* Fixes Caused by: com.fasterxml.jackson.databind.JsonMappingException: String value length (20054016) exceeds the maximum allowed (20000000, from

* Update log message
2025-02-04 10:24:33 +05:30
mohitdeuex
04cab8e71a Revert "- Update lineage model and api"
This reverts commit c5cce2b0c7c1be7ba654bfc2677da0d05215a895.
2025-02-03 12:11:07 +05:30
mohitdeuex
03f492ad1d Revert "- Adds updatedAt and updatedBy"
This reverts commit c6b462f0ab5905fe339c08a05ab819ca940dd06d.
2025-02-03 12:11:07 +05:30
Mohit Yadav
78123db16c
Merge branch 'lineage-work' into main 2025-02-03 11:43:01 +05:30
sonika-shah
d067121fa3
Fix #17059 - Task cannot be closed by the non-admin user who created it (#19528)
* Fix #17059 - Task cannot be closed by the non-admin user who created it

* uncommented playwright test around the flow

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-02-03 11:04:50 +05:30
mohitdeuex
c6b462f0ab - Adds updatedAt and updatedBy 2025-02-02 23:13:15 +05:30
mohitdeuex
c5cce2b0c7 - Update lineage model and api 2025-02-02 22:46:28 +05:30