2196 Commits

Author SHA1 Message Date
Mohit Yadav
b48a02d94e
Fix Column Lineage Validation issue (#22308)
* Fix Column Lineage Validation issue

* Add test for validations

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-07-14 19:25:49 +05:30
mohitdeuex
5e7fb80d05 Fix App Uninstall Flow 2025-07-14 18:26:45 +05:30
Ram Narayan Balaji
4af8ac065d
Fix: Extract Proper Parent FQN for column fqn (#22300)
* Fix: Extract Proper Parent FQN for column fqn

* Fix tests
2025-07-14 17:22:58 +05:30
Sriharsha Chintalapani
0b82e2e2a2
Fix EntityRelationshipCleanupTest, failing intermittently (#22322) 2025-07-13 15:00:13 -07:00
Sriharsha Chintalapani
d1c92fbe05
Fix #22990: Add ETag Support for Optimistic Concurrency Control in PATCH Operations (#22291)
* Fix #22990: Add ETag Support for Optimistic Concurrency Control in PATCH Operations

* Fix tests

* Add package.json back
2025-07-13 14:59:49 -07:00
Sriharsha Chintalapani
2c43cfe854
Add improvements to storing tags in PATCH or PUT (#22297)
* Add improvements to storing tags in PATCH or PUT

* Add bulk updates to owners, domains etc..

* Fix styling

* Optimize tag comparison performance from O(n²) to O(n)

Replace nested stream().anyMatch() operations with Set-based lookups in
updateTags() method. This reduces time complexity for tag operations from
O(n²) to O(n) by using createTagKey() and createTagKeySet() helper methods.

Performance impact: For 1000 tags, reduces operations from ~1M to ~1K.

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>

* Fix stylecheck

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2025-07-13 07:57:56 -07:00
Suresh Srinivas
2c1ccca730
Remove unnecessary exception throws, Redundant lambda code and javadoc (#22321)
* Remove unnecessary exception throws, Rredundant labmda code and javadoc

* Fix java checkstyle issues
2025-07-12 17:26:00 -07:00
Sriharsha Chintalapani
58cd7daf7e
Add drive as service (#22159)
* Fix #22157: Add Drive Service Support to OpenMetadata

* Fix #22157: Add Drive Service Support to OpenMetadata

* Update generated TypeScript types

* Fix tests

* Update generated TypeScript types

* Fix tests

* Tests are passing

* Tests are passing

* Update generated TypeScript types

* Address comments

* Update generated TypeScript types

* Add sample data and ingestion for drives, directories, spreadsheets, worksheets, files

* Fix postgres migrations

* Fix postgres tests, optimize queries

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2025-07-11 10:00:54 -07:00
Sriharsha Chintalapani
e835c1074f
Revamp metrics, Add Request Latencies, breakdown database, search latencies (#22259)
* Add cleaner metrics

* Revamp Metrics; Add Request Latencies, breakdown database, search latencies

* Remove MicrometerBundleSingleton.java

* Fix tests
2025-07-10 11:34:30 -07:00
sonika-shah
9a65ae1a50
Fix : Activity Feed Not Showing Column-Level Metadata Changes (#22245)
* Fix : Activity Feed Not Showing Column-Level Metadata Changes

* Add backend test

* fix checkstyle

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-07-09 15:36:42 -07:00
Sriharsha Chintalapani
91ae7c4882
Fix Promtheus endpoint (#22233) 2025-07-09 15:36:16 -07:00
Mohit Yadav
331cc07221
Fix ElasticSearchSink and OpeSearchSink (#22257) 2025-07-09 15:34:30 -07:00
Sriharsha Chintalapani
f1b73aeaec
Add Permission Debugger to understand roles & permissions for a user (#22207)
* Add debug api endpoints for roles & policies

* Add Roles & Policy Debugger

* fix errors

* Address comments

* Address comments

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-07-08 18:55:51 -07:00
Mayur Singal
a5dc332c1c
Fix #1751: Include Column Constraint in DI Charts (#22199)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-07-08 15:09:34 -07:00
Teddy
80cd12fb39
pass test case result ts as event time (#22190) 2025-07-08 11:14:43 -07:00
Ajith Prasad
38e0da5a05
Schema Change Alert Not Triggering for Team Owner with Slack Destination (#22215) 2025-07-08 06:53:15 -07:00
Ram Narayan Balaji
85b7328e95
Minor fix: Relevant fields are pulled for bulk import (#22154)
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-07-08 16:09:56 +05:30
Mohit Yadav
0b2321e976
Added Session Age for Cookies (#22166)
* - Added Session Age for Cookies

* Make OIDC Session Expiry Configurable

* Update generated TypeScript types

* Updated Docker Files

* Update Session to 7 days

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-07-08 15:07:52 +05:30
sonika-shah
56b1719494
Fix #22067 : Domain inheritance issue after team removal (#22146)
* fix Domain inheritance issue after team removal

* add backend test and fix logic

* added playwright test

* update domain on reload

* fix test

* fix test

---------

Co-authored-by: Shrushti Polekar <shrushtipolekar@gmail.com>
2025-07-08 12:26:19 +05:30
Sriharsha Chintalapani
d21c0b0f2e
Add DataContract status result API; add reviewers for data contract (#22176)
* Add DataContract status result API; add reviewers for data contract

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-07-07 18:30:21 +02:00
Sriharsha Chintalapani
02a5a5fcc2
Bulk load fields for paginations and indexing (#21771)
* Add db max connections to used while auto tune search index app

* Add more logging

* Add batch reading of entities

* Fix bulk loading issues

* Fix bulk loading issues

* Fix bulk loading issues

* Increase the default max concurrent requests

* Remove entitynotfound from errors

* Add higher no.of threads

* Fix KPI, TestCase, Policy repository fixes

* Fix Merge Issues

* Remove column tests

* Add bulk load for Tags

* Fix tests

* Fix tests for TestSuiteRepository

* Improve bulk loading in Table, DatabaseSchema

* Improve bulk loading in all entities

* fix merge error

* Improve indexing for entities

* Improve performance while pagination

* fix tests

* fix tests

* fix tests

* fix markdown test

* Address issues with bulkloading owners,inherited owners in database schema, tables, roles

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2025-07-07 07:46:42 -07:00
Sriharsha Chintalapani
cec7dd31ad
Fix #21662: Optional Redis Write-Through Cache for Database Queries (#21667)
* Fix #21662: Optional Redis Write-Through Cache for Database Queries

* Fix #21662: Optional Redis Write-Through Cache for Database Queries

* Fix #21662: Optional Redis Write-Through Cache for Database Queries

* Add lazy load, add test profile

* Fix stylecheck

* Fix tests, sonar comments

* Fix tests, sonar comments

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2025-07-04 23:21:20 -07:00
Ram Narayan Balaji
b80b6101e6
Fix:#21536 Column lineages are not getting updated/deleted when columns are updated/deleted (#22106) 2025-07-03 22:16:47 +05:30
Sriharsha Chintalapani
d2cff8ff52
Fix #21908: Add Persona configuration per User (#21909)
* Fix #21908: Add Persona configuration per User

* Add default persona, add tests for User defaultPersona

* Add default persona, add tests for User defaultPersona

* Fix tests
2025-07-03 14:15:07 +05:30
Sriharsha Chintalapani
0162f42a1d
Fix #20621: User Status Tracking in the System (#21911)
* Fix #20621: User Status Tracking in the System

* Fix #20621: User Status Tracking in the System

* Fix #20621: Add Playwright tests

* Update generated TypeScript types

* fix import

* remove unused imports and fix broken ui for settings user page

* revert path  change for PersonaDetailsPage

* fix unit tests

* fix tests

* Add separate api for last login time

* update last login time api

* remove last login time patch

* remove updateLoginTime

---------

Co-authored-by: shrushti2000 <shrushtipolekar@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2025-07-02 14:37:36 -07:00
Sriharsha Chintalapani
15f9cd115e
Fix LocalJwksProvider lookup, add MultiUrlJwkProvider tests (#22102)
* Fix LocalJwksProvider lookup, add MultiUrlJwkProvider tests

* fix style check
2025-07-02 12:40:48 -07:00
sonika-shah
5a124873e1
add name tag in pom.xml of all modules (#22104) 2025-07-02 22:48:11 +05:30
Ram Narayan Balaji
b82b4f23e5
Feature #20137 Move Glossary Terms to any level including other glossaries (#21867)
* allow changing of glossary term

* Draft ES Index Update for Glossary term move

* ES Upates in a more faster one shot bulk update

* Remove validations on Parent Glossary Hierarchy and Include Circular Glossary Term Validations

* Introduced a new PUT api for glossaryTerms move operation

* Optimised moveGlossaryTerm

* Added Transaction Scope in moveAndStore in updater

* Remove Consolidate and revert changes for move operation

* Improved Change Description and Versioning for move operation

* Improvise request payload to have only parent and optimise the flow

* Implement async functionality of Glossary Terms move

* mvn spotless:apply

* MoveGlossaryTermChannel, Get only the required fields

* Fix: Update Parent for Glossary Term during Bulk Import

* allow glossary term to be moved across glossaries

* update locales

* move socket listening to hierarchy dialog

* Added fqn to move message for the websocket response

* fix tests

* add glossary tests

* cleanup

* fix tests

* Refactor code: Resolve Conflicts

* fix tests

* fix permissions

* cleanup

* Update generated TypeScript types

* Abstraction Logic in place of instanceOf logic for bulkImportExportGlossary to update the parent

* Update generated TypeScript types

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-07-02 15:44:01 +05:30
Sriharsha Chintalapani
b461eeb881
Move MCP into separate maven module (#22043)
* Move MCP into separate maven module

* Fix styling

* Add Tests, upgrade to 0.10 mcp-sdk version

* Fix tests

* revert npm package and related files

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-07-01 12:02:29 -07:00
Mohit Yadav
e5e5017393
Issue in Os Lineage (#22069) 2025-07-01 17:55:59 +05:30
Akash Verma
bd25c6123e
feature:ssas-connector (#22028)
* feature:ssas-connector

* ui and ts files

* ui errors

* minor change

---------

Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-07-01 15:51:58 +05:30
Mohit Yadav
ffd50a7fbd
Update Servlet Holders (#22055) 2025-07-01 14:46:37 +05:30
Sriharsha Chintalapani
52a8ec0489
Fix: recreate fails if we can't find one index for a given entity (#22047) 2025-06-30 17:38:31 -07:00
Sriharsha Chintalapani
db8e2fa548
Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch (#21931)
* Improve indexing auto tune based on heap mem, parse custer metrics for new versions of OpenSearch

* Use native bulk processors

* lower the batch size, add composite index for app_runs table

* while updated secrets, if there is entityNotFoundException ignore and move on

* Fix deadlock

* Fix deadlock

* Fix gzip headers

* remove unnecessary code to check compression, batch sizes etc..

* Improved performance

* Improved performance

* fix merge error

* fix styling

* fix tests

* Fix build issues

* improve queue size recommendations

* put max limits on threads

* fix checkstyle
2025-06-30 07:40:25 -07:00
Suman Maharana
e36e5da26e
Added Databricks pipeline Lineage (#22014) 2025-06-30 10:41:22 +05:30
sonika-shah
ae3c6893eb
Fix Playwright failures: handle JsonParsingException after JsonUtils refactor (#22020) 2025-06-29 09:47:33 -07:00
Shailesh Parmar
62ab5689d2
Feat: add support of adding tags in test case form (#22007)
* Feat: add support of adding tags in test case form

* fix: don't override test case tags with entity tags

* feat: enhance test case form to support glossary terms and improve tag handling

* feat: implement tags and glossary terms functionality in test case forms and enhance related tests

* fixed failing unit test

* feat: add tier tag filtering and preservation in test case forms and related tests

* feat: add tags to addTestCaseAction

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2025-06-28 21:55:01 +02:00
Teddy
d36878409a
MINOR: Extracted indexMapping into spec module (#21882)
* feat: added endpoint to programatically retrieve index mapping

* feat: added endpoint to programatically retrieve index mapping

* refactor: extracted JsonUtils from server sub module

* refactor: move IndexMapping to spec

* style: ran java linting

* refactor: moved IndexMapping to singleton class

* fix: JsonUtils import

* style: ran java linting

* fix: init indexmapper in operations

* fix: moved ES index json files to spec

* style: ran java linting

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-06-27 14:47:36 +02:00
Teddy
d5b01f991e
MINOR: throw error on query filter parsing + error on column test case … (#21967)
* fix: throw error on query filter parsing + error on column test case with incorrect entityLink

* fix: failing tests

* fix: failing TableResourceTest
2025-06-27 11:17:39 +02:00
sonika-shah
971225dbce
Fix #21414: Custom properties display issue in advanced search (#21873)
* fix: remove recursive extraction of nested fields for entityReference and entityReferenceList type custom properties

* get displayName field along with other fields in /customProperties api

* show display name instead of name for custom props

* show display name instead of name for custom props

* fix extension type and add tests

* fix tests

* fix data model tests

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-06-26 19:35:45 +05:30
IceS2
94cf3e0fd6
MINOR: Extend profile workflow config to allow engine configuration (#21840)
* Update Profile Workflow to allow engine configuration

* Add ui generated schemas

* Add Repository Override mechanism based on annotations

* Implement logic to use the ProcessingEngine configuration

* Update SparkEngine to use remote and not master
2025-06-26 19:11:26 +05:30
Mohit Yadav
b8afbb4ed2
FIx failing Maven CI Tests (#21975)
* Fix Glossary Test Failures

* Fix Soft Deletion Causing issues
2025-06-26 16:03:36 +05:30
Mohit Yadav
3e3f1d8f3b
feat: implement entity lifecycle event delegation (#21921)
* feat: implement entity lifecycle event delegation framework

* added tests

* added reindex method to search clients

* Use virtual thread pool
2025-06-25 23:26:45 +05:30
Mohit Yadav
0c6ded3ae3
Soft-Deleted Entities are not removed from system, so relations are still valid if there is entry in db (#21957) 2025-06-25 21:54:02 +05:30
Mohit Yadav
2c696bb1e2
Remove Default: null from columns and let json to pojo handle (#21934)
* Remove Default: null from columns and let json to pojo handle

* Remove Setting table columns to null

* Fix Failing Tests

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-06-25 21:53:32 +05:30
Mohit Yadav
89fcac6316
feat: enhance relationshipCleanup with service hierarchy cleanup and Data Retention App (#21904)
* Extended relationshipCleanup command to clean up broken entities across all
  OpenMetadata service hierarchies (Database, Dashboard, API, Messaging, Pipeline,
  Storage, ML Model, and Search services), not just database services.

  Key changes:
  - Added performServiceHierarchyCleanup() method in OpenMetadataOperations
  - Added --skip-hierarchy-cleanup option for backward compatibility
  - Added getBrokenRelationFromParentToChild() and deleteBrokenRelationFromParentToChild() methods in CollectionDAO
  - Support for detecting and cleaning orphaned entities across any parent-child relationship
  - Maintains dry-run and actual cleanup modes with detailed reporting

  Usage: relationshipCleanup [--delete] [--skip-hierarchy-cleanup] [--batch-size N

* refactor: extract reusable cleanup utilities and eliminate code duplication

 Changes:
  - Add ServiceHierarchyCleanup.java for reusable hierarchy cleanup
  - Add ComprehensiveCleanupUtil.java for combined cleanup operations
  - Refactor OpenMetadataOperations to use reusable utilities
  - Refactor DataRetention to use reusable utilities
  - Single source of truth for all service hierarchy definitions

* - missing commit

* - Data Retention App test
2025-06-25 14:19:22 +05:30
Mohit Yadav
16aa38da8f
Fix Cache To have immutable entities (#21933) 2025-06-24 16:40:32 -07:00
Sriharsha Chintalapani
97cf131d19
Fix query cost aggregation (#21907)
* Fix query aggregation to use checksum not query text

* Fix query aggregation to use checksum not query text

* fix query text

* fix opensearch class

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2025-06-23 14:02:58 -07:00
sonika-shah
cf6eaf8a1a
Revert : query_cost_record_search_index Search reindexing issue (#21906) 2025-06-23 22:11:29 +05:30
sonika-shah
594ae851f5
Fix : query_cost_record_search_index Search reindexing issue (#21900) 2025-06-23 06:51:37 -07:00