202 Commits

Author SHA1 Message Date
Sriharsha Chintalapani
0162f42a1d
Fix #20621: User Status Tracking in the System (#21911)
* Fix #20621: User Status Tracking in the System

* Fix #20621: User Status Tracking in the System

* Fix #20621: Add Playwright tests

* Update generated TypeScript types

* fix import

* remove unused imports and fix broken ui for settings user page

* revert path  change for PersonaDetailsPage

* fix unit tests

* fix tests

* Add separate api for last login time

* update last login time api

* remove last login time patch

* remove updateLoginTime

---------

Co-authored-by: shrushti2000 <shrushtipolekar@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2025-07-02 14:37:36 -07:00
Sriharsha Chintalapani
db8e2fa548
Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch (#21931)
* Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch

* Use native bulk processors

* lower the batch size, add composite index for app_runs table

* while updated secrets, if there is entityNotFoundException ignore and move on

* Fix deadlock

* Fix deadlock

* Fix gzip headers

* remove unnecessary code to check compression, batch sizes etc..

* Improved performance

* Improved performance

* fix merge error

* fix styling

* fix tests

* Fix build issues

* improve queue size recommendations

* put max limits on threads

* fix checkstyle
2025-06-30 07:40:25 -07:00
Mohit Yadav
5b6e25999e
Disabled bot creating activity feeds (#21773)
* - Disabled bot creating feeds

* - nit
2025-06-14 19:21:00 +05:30
Mohit Yadav
6124c5573e
fix tests (#21751) 2025-06-12 20:47:41 -07:00
Mohit Yadav
4309f79453
Fix Reset Link Template (#21747)
* Fix Reset Link Template

* 1.7.3
2025-06-13 08:57:18 +05:30
Sriharsha Chintalapani
18bd4689a0
Add Data Contracts Specification and APIs (#21164)
* Add Data Contracts Specification and APIs

* Fix checkstyle

* Add @Repository

* Add @Repository

* Address YAML APIs

* Fix tests

* Fix tests

* Address comments

* Fix tests for checking table/topic fields do exists for a data contract

* migrate to jakarta

* migrate to jakarta

* add semantics

---------

Co-authored-by: Harsha <harsha@Harshas-MacBook-Air.local>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-06-04 06:36:28 +02:00
Pere Miquel Brull
e58802b659
FIX - Automation Workflows should not be updated by the SM & cleanup migration (#21435)
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2025-06-03 12:17:14 +02:00
Imri Paran
baee931b85
MINOR: App changes for collate support (#21068)
* feat: prep for collate support

- Added support for delayed background jobs. This is to delete the support user after a while.
- added ScheduleType.OnlyManual for applications to support only manual triggers
- Added a four hour token expiration
- Allow a null trigger type in job data (not sure what it's for)

* added test for delayed job

* added type checking for enumCleanupArgs in navbar

* - added a support toten type
- added job handler for delete token

* fixed whitelist links

* patch OnlyManual apps

* Update openmetadata-spec/src/main/resources/json/schema/entity/applications/app.json

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* added indexes for runAt

* use NO_JOB_SLEEP_SECONDS in background job test

* Fixed NavBar test

* removed backticls

* fix indexes

* change for support app

* fix tests

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-05-26 12:02:14 +02:00
Mohit Yadav
1ee36d41c6
Escape ? to causing issues in jdbi binding (#21381) 2025-05-23 17:13:45 +05:30
Mohit Yadav
94ceb7792a
Failing Postgres sql (#21378) 2025-05-23 16:39:56 +05:30
IceS2
6f4204731f
MINOR: Fix app marketplace migrations (#21360)
* Fix app marketplace migrations

* Fix test
2025-05-22 15:13:00 +02:00
IceS2
04bfb7b85d
Improve Workflow Logs and Reduce Concurrency (#21345) 2025-05-21 16:22:28 +02:00
IceS2
b761efbfd3
MINOR: Add missing workflow config status migrations (#21286)
* Add missing workflow config status migrations

* Moved to Java based migrations to redeploy pipeline

* Moved migrations to Java. Updated Listener based on config

* Improved update on the migrations
2025-05-20 08:02:02 +02:00
Imri Paran
07a6530b5b
fix: move autopilot runtime migrations to 1.7.1 (#21267) 2025-05-19 08:54:57 +00:00
Suman Maharana
f81ee52ec4
Chore Ingestion Tableau library change (#21076) 2025-05-15 17:48:39 +05:30
Imri Paran
112dc4be1f
MINOR fix: autopilot marketplace runtime (#21134) 2025-05-13 15:45:50 +05:30
Ayush Shah
49de0119cf
fix: Migrations - add operator parameter to test_definition for custom SQL queries in MySQL and Postgres (#21044) 2025-04-30 14:57:20 +05:30
Mohit Yadav
e0eac43636
Add cleanup apps_extension_time_series (#20857) 2025-04-16 14:54:11 +05:30
Teddy
98c6b6e4ef
migration: fix duplicate param key insertion (#20802) 2025-04-15 14:10:51 +02:00
Keshav Mohta
0796c6274b
Fixes: Databricks httpPath Required (#20611)
* fix: made databricks httpPath required and added a migration file for the same

* fix: added sql migration in postDataMigration file and fix databricks tests

* fix: added httpPath in test_source_connection.py and test_source_parsing.py files

* fix: added httpPath in test_databricks_lineage.py

* fix: table name in postgres migration
2025-04-07 13:33:55 +05:30
IceS2
c9182016db
MINOR: Update DayOneWorkflow Agent Names (#20473)
* Update DayOneWorkflow Agent Names

* Migrations for CollateAI DisplayName Update

* Remove wrong migrations
2025-03-28 14:01:42 +01:00
IceS2
deca9f9574
Fix Workflow State not Updating on Exception (#20395) 2025-03-24 09:26:05 -07:00
Teddy
5a3fc2cfa4
feat: add created by for test case entities (#20304)
* feat: add created by for test case entities

* migration: createdBy sql migration
2025-03-24 13:13:01 +01:00
Imri Paran
451506cb08
MINOR: chore: added missing timestamp indexes for time series tables (#20373)
* chore: added index for apps_extension_time_series.timestamp

* - added timestamp index for workflow_instance_time_series
- added timestamp index for workflow_instance_state_time_series
2025-03-24 07:43:07 +01:00
Imri Paran
277e602482
fix: workflow definition migrations (#20357) 2025-03-20 14:28:23 +01:00
IceS2
ce497e020e
MINOR: Expose Workflow Stage Status and Bring OneDayApp to OSS (#20123)
* Expose Workflow Stage Status and Bring OneDayApp to OSS

* Fix Migrations

* Change Bot and add comment

* Fix checkstyle
2025-03-10 09:25:32 +01:00
Mayur Singal
46947277c4
MINOR: Fix cost analysis build failing for postgres (#20122) 2025-03-06 21:30:28 +00:00
Mayur Singal
c1592b54c5
Capture Query Cost (#19991) 2025-03-06 19:49:59 +05:30
IceS2
fd7b00d246
MINOR: Fix Migrations (#19925)
* Fix Migrations

* Fix WorkflowHandler initialization

* Refactor how to fix the migrations by passing the Config object itself

* Fix Migrations

* Remove comments

* Remove comments
2025-02-26 12:53:08 +05:30
Mohit Yadav
cb7942f4b6
Cleanup App data (#19571) 2025-01-28 19:22:33 +05:30
Imri Paran
76c9b5a0ae
fix(migrations): use double question-mark (#19557)
Use double question mark to escape parametrization by jdbi.
https://jdbc.postgresql.org/documentation/query/#using-the-statement-or-preparedstatement-interface
2025-01-28 18:07:23 +05:30
Mohit Yadav
58d5988a16
Add Onboarding SQL (#19525)
* Add Primary Key Migration

* Update queries and conflict on apps data store

* Update queries and conflict on apps data store

* add index on isBot
2025-01-28 11:45:58 +05:30
Imri Paran
5fb7abb820
MINOR: fix syntax in migration (#19540) 2025-01-27 18:03:54 +00:00
Imri Paran
a0fcf67a4e
fix(governance-workflows): fix migrations for 1.7.0 (#19533)
eventBasedEntityWorkflow -> eventBasedEntity
periodicBatchEntityWorkflow -> periodicBatchEntity
2025-01-27 14:29:15 +01:00
Siddhant
473688648b
Fix #18834: Introduce new setting OpenMetadataBaseUrlConfiguration to set the server endpoint (#19366)
* Refactor: Introduce openMetadataBaseUrlConfiguration to set up the server endpoint and remove OpenMetadataUrl from smtpSettings.json

* migrations.

* refactor(cli): add setOpenMetadataUrl command to store OpenMetadata URL

* refactor(cli): add configureEmailSettings command to manage SMTP/Email configuration

* load initialData when config is not present in the db

* add om url settings page

* update locales

* add e2e

* load omBaseUrl with default host and port: localhost:8585 when not present

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2025-01-27 10:20:04 +05:30
sonika-shah
3267b71217
Fix postgres query added in pr #19411 (#19512) 2025-01-24 16:56:07 +05:30
Imri Paran
a6670af500
MINOR: Use interface schemas for governance workflows (#19411)
* feat(apps): support event subscriptions

- added support for apps with event subscriptions.
- added support for custom consumers on event subscriptions.
- use native application methods instead of reflection in ApplicationHandler

* removed runMethodFromApplication reflection and use concrete methods from AbstractNativeApplication

* format

* fix(governance-workflows): defined interface in workflow schema

- defined types for workflow nodes
- fixed updateEdge method in WorkflowDefinitionRepository

* ref(governance-workflows): use explicit types in schema

use explicit interface type in json schemas and use annotations to infer the concrete classes when deserializing

* - implemented CustomSignal trigger
- use JsonTypeInfo and JsonSubTypes to infer types in jsonschema for nodes and triggers
- Implemented JsonLogicFilter task
- implemented placeholder noop task

* - simplified trigger type names
- removed governance workflow root "type"
- added "deployed" field to governance workflow
- applied changes to existing gov-workflows
- migrations

* fixed migrations
2025-01-24 09:08:06 +01:00
sonika-shah
5ccc0c5a2a
Improvement #19065 : Support removing existing enumKeys (for enum type custom property) (#19054)
* Improvement : Support removing existing enumKeys (for enum type custom property)

* initial implementation of background job job runner for enum cleanup

* fix postgres query

* ui: allow existing enum value updates in EditCustomPropertyModal

* create BackgroundJob to use jsonSchema

* feat: add socket event handling for background job part1

* fix(i18n): remove trailing spaces in Galician language JSON file

* refactor to use jsonSchema

* feat(ui): show browser notification for custom property update

* feat(utils): add getCustomPropertyEntityPathname function and tests

* change enumCleanup trigger condition

* feat(i18n): add enum property update message to multiple language JSON files

* enhancement : improve execution flow and better exception handling with test

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-01-07 19:25:59 -08:00
Pere Miquel Brull
e56f477a4a
Fix #19147 - Executable Test Suites (#19221)
* backend

* format & tests

* rename backend

* migrations and ingestion

* format & tests

* format & tests

* tests

* format & tests

* tests

* updated ui side of changes

* addressing comment

* fixed failing unit test

* fix test list

* added e2e test, and fixed existing test

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2025-01-07 17:59:54 +01:00
Teddy
bd68d959f3
fix: added index on timestamp field for reindex (#19101) 2024-12-16 20:57:42 +01:00
Mohit Yadav
3578a4b32d
Domain Policy Update to be non-system (#19060) 2024-12-15 01:18:12 +05:30
Siddhant
4985496051
fix: Postgresql migrations for eventSubscriptionOffset. (#18980) 2024-12-09 20:19:39 -08:00
Siddhant
e770ba8e0e
FIX: Combined index on offset and eventTime in changeEvent table (#18922)
* fix: combined index on offset and eventTime in changeEvent table

* fix: correct syntax error in Postgres query.

* fix: correct syntax error in Postgres query.

* fix: correct syntax error in Postgres query.
2024-12-04 15:14:31 -08:00
IceS2
f939e7d3c0
Reduce amount of flowable created tables (#18882)
* Reduce the number of flowable tables on postgresql

* Reduce the amount of Flowable Created Tables
2024-12-04 19:34:33 +01:00
Siddhant
28f551d20a
GEN-Activity Feed Filtering (#18873)
* fix: Update configuration for activity feed.

* fix: Migrations

* remove the customProperty and testSuite feed beautification case from the feed

* fix: Remove custom properties from the source

* fix: add metric in the matchAnySource.

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-12-04 16:56:21 +05:30
Pere Miquel Brull
613fd331e0
MINOR - Clean up configs & add auto classification docs (#18907)
* MINOR - Clean up configs & add auto classification docs

* deprecation notice
2024-12-04 09:32:25 +01:00
harshsoni2024
cb33f274fc
Connector: rename microstrategy connector (#18604) 2024-11-28 18:50:42 +05:30
Suman Maharana
9a21e77e15
Added dbt cloud multi projects and jobs filter (#18801)
* Added dbt cloud multi project and jobs filter

* added tests

* change to array type

* updated yaml config

* added migrations
2024-11-28 16:10:34 +05:30
Pere Miquel Brull
c68a45e7d8
Create new Auto Classification Workflow (#18610) 2024-11-19 08:10:45 +01:00
Aniket Katkar
bfc12feafc
GEN-1976: Alert improvements (#18448)
* Move the alert components to the components folder from pages folder

* auto generated localization changes and linting changes

* Fix the imports

* Add AlertConfigDetails component

* Add tabs in the alert details page

* auto generated localization changes

* Apply permissions for alert and add owner update functionality

* Change the alert update method from PUT to PATCH

* Add the recent events tab

* auto generated localization changes

* Improve the recent events tab for alert details

* Implement the logic to show alert events count on the alert details page

* auto generated localization and linting changes

* Add the pagination logic for the recent events list on the alert details page

* Fix the logic for previous click in NextPreviousWithOffset

* Localization change for other languages

* Hide pagination when not required

* Add pagination and permissions to API; create eventsRecord API; fix timestamp issue

* fix: remove publisherName from test destination message.

* Fix the recent events tab error

* fix: Resolve PATCH API issue with destination ID creation.

* fix: Delete associated alert metrics upon alert deletion

* feat: Detailed HTTP Response for Destination Test

* Add the test external destination functionality

* localization changes for other languages

* fix: Set status for testing destination in EmailPublisher

* Add advanced configurations for webhook destinations

* Localization changes for other languages

* Fix the destination status not showing
Add the POST and PUT radio button in advanced configs

* Localization changes for other languages

* Update the API to fetch the alert event diagnostic counts

* Improve the AlertConfigDetails component to show the alert configs properly

* fix: Display displayName instead of fullyQualifiedName in templates

* Fix the test destination result not showing the first time

* Fix the test destination status not showing for already existing destinations

* Fix the unit tests

* Fix playwright tests

* refactor: Separate sendMessage and sendTestMessage methods; add statusContext

* fix: Use name or fqn as fallback if displayName is null or empty

* fix: upsert successfulChangeEvent

* eliminate the GovernanceWorkflowChangeEvent option from external destination

* Add code comments

* Fix the unit test

---------

Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
Co-authored-by: Siddhanttimeline <siddhanttimeline9@gmail.com>
2024-11-14 17:39:45 +05:30