2333 Commits

Author SHA1 Message Date
Mayur Singal
a5dc332c1c
Fix #1751: Include Column Constraint in DI Charts (#22199)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-07-08 15:09:34 -07:00
Teddy
80cd12fb39
pass test case result ts as event time (#22190) 2025-07-08 11:14:43 -07:00
Ajith Prasad
38e0da5a05
Schema Change Alert Not Triggering for Team Owner with Slack Destination (#22215) 2025-07-08 06:53:15 -07:00
Ram Narayan Balaji
85b7328e95
Minor fix: Relevant fields are pulled for bulk import (#22154)
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-07-08 16:09:56 +05:30
Mohit Yadav
0b2321e976
Added Session Age for Cookies (#22166)
* - Added Session Age for Cookies

* Make OIDC Session Expiry Configurable

* Update generated TypeScript types

* Updated Docker Files

* Update Session to 7 days

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-07-08 15:07:52 +05:30
sonika-shah
56b1719494
Fix #22067 : Domain inheritance issue after team removal (#22146)
* fix Domain inheritance issue after team removal

* add backend test and fix logic

* added playwright test

* update domain on reload

* fix test

* fix test

---------

Co-authored-by: Shrushti Polekar <shrushtipolekar@gmail.com>
2025-07-08 12:26:19 +05:30
Sriharsha Chintalapani
d21c0b0f2e
Add DataContract status result API; add reviewers for data contract (#22176)
* Add DataContract status result API; add reviewers for data contract

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-07-07 18:30:21 +02:00
Sriharsha Chintalapani
02a5a5fcc2
Bulk load fields for paginations and indexing (#21771)
* Add db max connections to used while auto tune search index app

* Add more logging

* Add batch reading of entities

* Fix bulk loading issues

* Fix bulk loading issues

* Fix bulk loading issues

* Increase the default max concurrent requests

* Remove entitynotfound from errors

* Add higher no.of threads

* Fix KPI, TestCase, Policy repository fixes

* Fix Merge Issues

* Remove column tests

* Add bulk load for Tags

* Fix tests

* Fix tests for TestSuiteRepository

* Improve bulk loading in Table, DatabaseSchema

* Improve bulk loading in all entities

* fix merge error

* Improve indexing for entities

* Improve performance while pagination

* fix tests

* fix tests

* fix tests

* fix markdown test

* Address issues with bulkloading owners,inherited owners in database schema, tables, roles

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2025-07-07 07:46:42 -07:00
Sriharsha Chintalapani
cec7dd31ad
Fix #21662: Optional Redis Write-Through Cache for Database Queries (#21667)
* Fix #21662: Optional Redis Write-Through Cache for Database Queries

* Fix #21662: Optional Redis Write-Through Cache for Database Queries

* Fix #21662: Optional Redis Write-Through Cache for Database Queries

* Add lazy load, add test profile

* Fix stylecheck

* Fix tests, sonar comments

* Fix tests, sonar comments

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2025-07-04 23:21:20 -07:00
Ram Narayan Balaji
b80b6101e6
Fix:#21536 Column lineages are not getting updated/deleted when columns are updated/deleted (#22106) 2025-07-03 22:16:47 +05:30
Sriharsha Chintalapani
d2cff8ff52
Fix #21908: Add Persona configuration per User (#21909)
* Fix #21908: Add Persona configuration per User

* Add default persona, add tests for User defaultPersona

* Add default persona, add tests for User defaultPersona

* Fix tests
2025-07-03 14:15:07 +05:30
Sriharsha Chintalapani
0162f42a1d
Fix #20621: User Status Tracking in the System (#21911)
* Fix #20621: User Status Tracking in the System

* Fix #20621: User Status Tracking in the System

* Fix #20621: Add Playwright tests

* Update generated TypeScript types

* fix import

* remove unused imports and fix broken ui for settings user page

* revert path  change for PersonaDetailsPage

* fix unit tests

* fix tests

* Add separate api for last login time

* update last login time api

* remove last login time patch

* remove updateLoginTime

---------

Co-authored-by: shrushti2000 <shrushtipolekar@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2025-07-02 14:37:36 -07:00
Sriharsha Chintalapani
15f9cd115e
Fix LocalJwksProvider lookup, add MultiUrlJwkProvider tests (#22102)
* Fix LocalJwksProvider lookup, add MultiUrlJwkProvider tests

* fix style check
2025-07-02 12:40:48 -07:00
sonika-shah
5a124873e1
add name tag in pom.xml of all modules (#22104) 2025-07-02 22:48:11 +05:30
Ram Narayan Balaji
b82b4f23e5
Feature #20137 Move Glossary Terms to any level including other glossaries (#21867)
* allow changing of glossary term

* Draft ES Index Update for Glossary term move

* ES Upates in a more faster one shot bulk update

* Remove validations on Parent Glossary Hierarchy and Include Circular Glossary Term Validations

* Introduced a new PUT api for glossaryTerms move operation

* Optimised moveGlossaryTerm

* Added Transaction Scope in moveAndStore in updater

* Remove Consolidate and revert changes for move operation

* Improved Change Description and Versioning for move operation

* Improvise request payload to have only parent and optimise the flow

* Implement async functionality of Glossary Terms move

* mvn spotless:apply

* MoveGlossaryTermChannel, Get only the required fields

* Fix: Update Parent for Glossary Term during Bulk Import

* allow glossary term to be moved across glossaries

* update locales

* move socket listening to hierarchy dialog

* Added fqn to move message for the websocket response

* fix tests

* add glossary tests

* cleanup

* fix tests

* Refactor code: Resolve Conflicts

* fix tests

* fix permissions

* cleanup

* Update generated TypeScript types

* Abstraction Logic in place of instanceOf logic for bulkImportExportGlossary to update the parent

* Update generated TypeScript types

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-07-02 15:44:01 +05:30
Sriharsha Chintalapani
b461eeb881
Move MCP into separate maven module (#22043)
* Move MCP into separate maven module

* Fix styling

* Add Tests, upgrade to 0.10 mcp-sdk version

* Fix tests

* revert npm package and related files

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-07-01 12:02:29 -07:00
Mohit Yadav
e5e5017393
Issue in Os Lineage (#22069) 2025-07-01 17:55:59 +05:30
Akash Verma
bd25c6123e
feature:ssas-connector (#22028)
* feature:ssas-connector

* ui and ts files

* ui errors

* minor change

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-07-01 15:51:58 +05:30
Mohit Yadav
ffd50a7fbd
Update Servlet Holders (#22055) 2025-07-01 14:46:37 +05:30
Sriharsha Chintalapani
52a8ec0489
Fix: recreate fails if we can't find one index for a given entity (#22047) 2025-06-30 17:38:31 -07:00
Sriharsha Chintalapani
db8e2fa548
Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch (#21931)
* Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch

* Use native bulk processors

* lower the batch size, add composite index for app_runs table

* while updated secrets, if there is entityNotFoundException ignore and move on

* Fix deadlock

* Fix deadlock

* Fix gzip headers

* remove unnecessary code to check compression, batch sizes etc..

* Improved performance

* Improved performance

* fix merge error

* fix styling

* fix tests

* Fix build issues

* improve queue size recommendations

* put max limits on threads

* fix checkstyle
2025-06-30 07:40:25 -07:00
Suman Maharana
e36e5da26e
Added Databricks pipeline Lineage (#22014) 2025-06-30 10:41:22 +05:30
sonika-shah
ae3c6893eb
Fix Playwright failures: handle JsonParsingException after JsonUtils refactor (#22020) 2025-06-29 09:47:33 -07:00
Shailesh Parmar
62ab5689d2
Feat: add support of adding tags in test case form (#22007)
* Feat: add support of adding tags in test case form

* fix: don't override test case tags with entity tags

* feat: enhance test case form to support glossary terms and improve tag handling

* feat: implement tags and glossary terms functionality in test case forms and enhance related tests

* fixed failing unit test

* feat: add tier tag filtering and preservation in test case forms and related tests

* feat: add tags to addTestCaseAction

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2025-06-28 21:55:01 +02:00
Teddy
d36878409a
MINOR: Extracted indexMapping into spec module (#21882)
* feat: added endpoint to programatically retrieve index mapping

* feat: added endpoint to programatically retrieve index mapping

* refactor: extracted JsonUtils from server sub module

* refactor: move IndexMapping to spec

* style: ran java linting

* refactor: moved IndexMapping to singleton class

* fix: JsonUtils import

* style: ran java linting

* fix: init indexmapper in operations

* fix: moved ES index json files to spec

* style: ran java linting

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-06-27 14:47:36 +02:00
Teddy
d5b01f991e
MINOR: throw error on query filter parsing + error on column test case … (#21967)
* fix: throw error on query filter parsing + error on column test case with incorrect entityLink

* fix: failing tests

* fix: failing TableResourceTest
2025-06-27 11:17:39 +02:00
sonika-shah
971225dbce
Fix #21414: Custom properties display issue in advanced search (#21873)
* fix: remove recursive extraction of nested fields for entityReference and entityReferenceList type custom properties

* get displayName field along with other fields in /customProperties api

* show display name instead of name for custom props

* show display name instead of name for custom props

* fix extension type and add tests

* fix tests

* fix data model tests

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-06-26 19:35:45 +05:30
IceS2
94cf3e0fd6
MINOR: Extend profile workflow config to allow engine configuration (#21840)
* Update Profile Workflow to allow engine configuration

* Add ui generated schemas

* Add Repository Override mechanism based on annotations

* Implement logic to use the ProcessingEngine configuration

* Update SparkEngine to use remote and not master
2025-06-26 19:11:26 +05:30
Mohit Yadav
b8afbb4ed2
FIx failing Maven CI Tests (#21975)
* Fix Glossary Test Failures

* Fix Soft Deletion Causing issues
2025-06-26 16:03:36 +05:30
Mohit Yadav
3e3f1d8f3b
feat: implement entity lifecycle event delegation (#21921)
* feat: implement entity lifecycle event delegation framework

* added tests

* added reindex method to search clients

* Use virtual thread pool
2025-06-25 23:26:45 +05:30
Mohit Yadav
0c6ded3ae3
Soft-Deleted Entities are not removed from system, so relations are still valid if there is entry in db (#21957) 2025-06-25 21:54:02 +05:30
Mohit Yadav
2c696bb1e2
Remove Default: null from columns and let json to pojo handle (#21934)
* Remove Default: null from columns and let json to pojo handle

* Remove Setting table columns to null

* Fix Failing Tests

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-06-25 21:53:32 +05:30
Mohit Yadav
89fcac6316
feat: enhance relationshipCleanup with service hierarchy cleanup and Data Retention App (#21904)
* Extended relationshipCleanup command to clean up broken entities across all
  OpenMetadata service hierarchies (Database, Dashboard, API, Messaging, Pipeline,
  Storage, ML Model, and Search services), not just database services.

  Key changes:
  - Added performServiceHierarchyCleanup() method in OpenMetadataOperations
  - Added --skip-hierarchy-cleanup option for backward compatibility
  - Added getBrokenRelationFromParentToChild() and deleteBrokenRelationFromParentToChild() methods in CollectionDAO
  - Support for detecting and cleaning orphaned entities across any parent-child relationship
  - Maintains dry-run and actual cleanup modes with detailed reporting

  Usage: relationshipCleanup [--delete] [--skip-hierarchy-cleanup] [--batch-size N

* refactor: extract reusable cleanup utilities and eliminate code duplication

 Changes:
  - Add ServiceHierarchyCleanup.java for reusable hierarchy cleanup
  - Add ComprehensiveCleanupUtil.java for combined cleanup operations
  - Refactor OpenMetadataOperations to use reusable utilities
  - Refactor DataRetention to use reusable utilities
  - Single source of truth for all service hierarchy definitions

* - missing commit

* - Data Retention App test
2025-06-25 14:19:22 +05:30
Mohit Yadav
16aa38da8f
Fix Cache To have immutable entities (#21933) 2025-06-24 16:40:32 -07:00
Sriharsha Chintalapani
97cf131d19
Fix query cost aggregation (#21907)
* Fix query aggregation to use checksum not query text

* Fix query aggregation to use checksum not query text

* fix query text

* fix opensearch class

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2025-06-23 14:02:58 -07:00
sonika-shah
cf6eaf8a1a
Revert : query_cost_record_search_index Search reindexing issue (#21906) 2025-06-23 22:11:29 +05:30
sonika-shah
594ae851f5
Fix : query_cost_record_search_index Search reindexing issue (#21900) 2025-06-23 06:51:37 -07:00
Ajith Prasad
b8887e4df1
Scim Bot can update display name (#21879)
* Scim Bot can update display name

* Updated by Bot checked fir editdisplayname

* format

* Fixed TC

* Fixed policy tests

* comment

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-06-22 17:44:46 -07:00
Teddy
337ce90797
MINOR: Alert test message type (#21866)
* fix: align message with Content-Type for test webhook alert

* fix: align message with Content-Type for test webhook alert
2025-06-20 14:39:11 +02:00
Keshav Mohta
73ea60b898
Refactor: Unity Catalog (#21801) 2025-06-20 16:04:34 +05:30
Ajith Prasad
2f7c6ef05e
Swagger response code corrected for scim (#21871) 2025-06-20 15:49:18 +05:30
Ram Narayan Balaji
e7c0216b25
Fix:#21647 Inherited Tags from Glossary Term should be shown in put column response (#21832)
* Fix: Inherited Tags from Glossary Term should be shown in put/patch column response

* Placement of the statement fixed

---------

Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2025-06-19 15:13:20 +05:30
Ajith Prasad
33711766e2
commons-beanutils vulnerability reported in security scan (#21830) 2025-06-19 08:43:19 +05:30
Sriharsha Chintalapani
802438f0ea
Fix default boost score, improve fqn parsing (#21854)
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster

* Add exact match configs

* Add exact match configs

* Update Logic to build search source builder with exact match priority

* Revert "Update Logic to build search source builder with exact match priority"

This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.

* Revert "Add exact match configs"

This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.

* revert display name change

* make boost mode as sum by defaul

* add more fqnparts for schema and database

* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue

* use fqn split

* refactor fqn parsing

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2025-06-18 18:56:11 -07:00
Sriharsha Chintalapani
8adda4955c
Revert "Issues in Search Relevancy (#21841)" (#21853)
This reverts commit f388e570c1dac5b9eee31364870fb66e42715f18.
2025-06-18 16:43:34 -07:00
Mohit Yadav
f388e570c1
Issues in Search Relevancy (#21841)
* Fix explain turned by default, use dfs_query_then_fetch in cases of sharding of search cluster

* Add exact match configs

* Add exact match configs

* Update Logic to build search source builder with exact match priority

* Revert "Update Logic to build search source builder with exact match priority"

This reverts commit 175a2e9c6b67ee90d4b2a35af89bb035e8c45131.

* Revert "Add exact match configs"

This reverts commit 3fd52606610bbb97a676170004cab6d7adc31a0d.

* revert display name change

* make boost mode as sum by defaul

* add more fqnparts for schema and database

* revert DFS_QUERY_THEN_FETCH since sharding wasn the issue

* use fqn split

* Refactor FQN Parts

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-06-18 16:33:46 -07:00
Karan Hotchandani
631c6f58fe
Feat(ui): add owners field in tags and classification (#21757)
* add owners field in classification

* fix tests

* add owners field in tags page

* Tag Inherits owners from classification

* Added loadTags.ts for owners field

* add owners for tags page

* fix tests

* review comments

* add owners field in create form

* fix domain test

---------

Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2025-06-18 16:52:53 +05:30
sonika-shah
9e281d0ee5
fix #21394 Custom properties bug update with existing wrong values (#21825)
* fix #21394 allow only updated extension fields to be validated

* fix #21394 add tests

* fix #21394 add tests
2025-06-18 14:03:45 +05:30
Sriharsha Chintalapani
fb5e45e6cd
Fix TableResourceTest failures for column updates (#21826) 2025-06-17 17:48:01 -07:00
Sriharsha Chintalapani
c90138501f
Fix #21822: OpenSearch by default limits the number of characters it will analyze for highlighting to 1,000,000 characters. If your description field is very large (e.g. Markdown docs, embedded HTML, or verbose documentation), this limit gets exceeded. (#21821)
* Add sample data

* Fix index mappings to optimize the highlighter for OpenSearch
2025-06-17 14:22:11 -07:00