14340 Commits

Author SHA1 Message Date
Mayur Singal
c13ae7269d
Fix #24318: Dbt Nonetype has no attr len error (#24320) 2025-11-13 17:52:05 +05:30
sonika-shah
e53a98f6c0
Fix socket timeout connection issue in Mysql AUT 2 (#24313)
* Fix socket timeout connection issue in Mysql AUT 2

* update connect time
2025-11-13 17:28:04 +05:30
IceS2
a3238edb66
Fix Validators (#24310) 2025-11-13 11:18:15 +01:00
Ashish Gupta
274db6dd7c
fix the tab still visible, even if it has been removed from persona in Charts (#24315) 2025-11-13 14:49:50 +05:30
Bhanu Agrawal
2fb34a9762
Migrate/es os maven hlrc removal (#24283)
* Setup new client for ES/OS

* Migrated createIndex and addIndexAlias methods

* Migrated createAliases method to new ES/OS client

* Migrated updateIndex and deleteIndex methods

* Updated indexExists methods

* refactor: extract index management operations into dedicated manager classes

* Add tests for ElasticSearch and OpenSearch index managers

* chore: fix code style issues"

* Added integeration tests for ES/OS index manager

* Fix log level

* Fixed priority key mapping for test_case_result_index

* fix test to use field name in the query as domains.id not domain.id

* Migrated createEntity and createEntities method

* Fixed failing tests

* Set headers so 8.x client can work with 7.17x or higher server for ES

* Added OpenSearch compatible stemmer configuration

* Fix java code style

* Fixed java checkstyle issue

* Added support for 7.17.x backward compatibility

* Fixed failing tests

* Migrated createTimeSeriesEntity and deleteEntity methods

* Refactor search client architecture with entity management abstraction

* Updated log level to ERROR form WARN

* deleteEntity method impl removed from ES client

* migrated deleteEntityByFields method to new es/os client

* migrated deleteEntityByFQNPrefix method to use new ES/OS client

* deleteEntityByFQNPrefix method removed

* Migrated deleteByScript method to new ES/OS client

* Removed deleteByScript method from SearchClient

* Migrated softDeleteOrRestoreEntity method to new ES/OS client

* Reverted BulkResponse static import

* Migrated softDeleteOrRestoreChildren method to new ES/OS client

* Migrated updateEntity method to new ES/OS client

* Migrated updateChildren method to new ES/OS client

* Removed unused methods

* Migrated getDocByID method to new ES/OS client

* Added info logging

* Fixed parsing issue while passing string doc

* Migrated updateEntityRelationship method to new ES/OS client

* Migrated reindexWithEntityIds method to new ES/OS client

* Added log to show error message for updateEntityRelationship

* Refactored entity manager methods

* Added async client, fixed null obj parsing issue

* Refactoring

* Refactored duplicate methods createEntity and createTimeSeriesEntity

* Added integration tests for ES/OS entity managers

* Updated client availability checks in both index manager classes to use the consistent !isClientAvailable pattern

* Removed comment

* chore: Fix line formatting in Playwright test files

- Adjusted line length in Domains.spec.ts for better readability
- Fixed indentation and formatting in Lineage.spec.ts
- Added missing newline at end of files

* Added new line

* Migrated other index related methods and add tests

* MIgrated updateByFqnPrefix and updateLineage and deleteByRangeQuery methods

* migrated deleteByRangeAndTerm method

* Added integration tests

* Migrated updateColumnsInUpstreamLineage and deleteColumnsInUpstreamLineage methods

* Migrated updateGlossaryTermByFqnPrefix method

* Removed unused method and updated error handling

* Migrated reindexEntities method to new ES/OS client

* Added integration tests for reindexEntities method

* Added interface and impl for generic methods migration

* added default impl

* Migrated deleteDataStream method

* Migrated deleteILMPolicy method

* Migrated deleteIndexTemplate method

* Migrated deleteComponentTemplate emthod

* migrated dettachIlmPolicyFromIndexes method

* migrated removeILMFromComponentTemplate method

* Migrated cluster metric methods

* Removed from ESClient/OSClient to GenericManager

* Migrated getSearchHealthStatus

* Add integration tests

* Migrated buildDIChart and fetchDIChartFields methods

* Migrated listDataInsightChartResult emthod

* Migrated getQueryCostRecords

* Migrated getSchemaEntityRelationship

* Added data insight aggregator tests

* Fixed failing tests

* Migrated aggregate method

* Migrated genericAggregation and aggregate methods

* Fixed failing data insight query

* Updated ref for OS aggregation manager

* Fixed issues with aggregations migration

* Fixed tests

* Migrated lineage methods

* fix aggregate api response

* Fixed aggregation response structure

* Migrated searchBySourceUrl and searchByField methods

* Migrated getEntityTypeCounts, listWithOffset and listWithDeepPagination methods

* Fix failing tests

* Fix tests

* Migrated searchWithDirectQuery method

* Migrated search and previewSearch methods

* Fix tests

* Migrated searchWithNLQ method

* Migrated searchEntityRelationship, searchDataQualityLineage and searchSchemaEntityRelationship methods

* Removed unused fields and methods

* Migrated bulk ES/OS methods

* Migrated DataInsightProcess ES/OS

* Migrated ES/OS bulk and index sinks

* Remove unused search methods

* Removed deprecated HLRC for ES/OS

* Removed maven HLRC ES/OS client

* Fixed collate compile issues

* Fixed collate compile issues

* Fix nested should

* Fix for failing playwright tests

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2025-11-13 14:21:35 +05:30
IceS2
d13f24a87c
Refactor Median Dimensionality since it was not working (#24305) 2025-11-13 09:04:03 +01:00
Rohit Jain
97a0a53c70
fix: Fixed the classification UI flickering issue (#24304) 2025-11-13 11:10:34 +05:30
Copilot
90e93949cf
Add support for classification tags in dbt meta field (#23671)
* Initial plan

* Add support for classification tags in dbt meta field

- Update DbtMetaGlossaryTier model to include tags field
- Add processing logic in process_dbt_meta for classification tags
- Add unit test for classification tags functionality
- Support tag FQNs in format 'classification.tag'

Co-authored-by: harshach <38649+harshach@users.noreply.github.com>

* Add comprehensive test for combined dbt meta tags

- Add test_dbt_combined_meta_tags to verify all tag types work together
- Test glossary terms, tier, and classification tags in one meta field
- Verify all three types of tags are processed correctly

Co-authored-by: harshach <38649+harshach@users.noreply.github.com>

* Add edge case tests for classification tags

- Test empty tags list returns empty result
- Test invalid tag format (missing classification) is skipped
- Test None tags is handled gracefully
- Ensures robust error handling

Co-authored-by: harshach <38649+harshach@users.noreply.github.com>

* Fix pycheck

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: harshach <38649+harshach@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Keshav Mohta <68001229+keshavmohta09@users.noreply.github.com>
2025-11-13 00:59:55 +05:30
Adrià Manero
20e041abf1
Prevent horizontal scroll and emoji line breaks in OpenMetadata email notifications (#24280)
* Fix email envelope style on Gmail

* Fix code styling

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2025-11-12 17:20:39 +01:00
Akash Verma
766fd639bd
Fixes #23102: Add consoleEndpointURL support for S3-compatible services (#23998) 2025-11-12 18:04:31 +05:30
Rohit Jain
ddba1ce89d
fix: Added pagiantion playwright test in a seperate file and fix code smells (#24293) 2025-11-12 17:56:08 +05:30
Adrià Manero
25eb9f028e
Implement Handlebars helpers metadata API endpoint (#24271)
* Implement Handlebars helpers metadata API endpoint

* Add cursorOffset and fix Handlebars helper syntax
2025-11-12 12:15:54 +01:00
IceS2
b27036bf0c
Fix ColumnValuesToBeUnique (#24284) 2025-11-12 12:09:43 +01:00
Mohit Yadav
df11eceeb7
Fixes for Data Retention (#24289) 2025-11-12 16:24:31 +05:30
sonika-shah
bde04680b4
Fix socket timeout connection issue in Mysql AUT (#24291)
* Fix socket timeout connection issue in Mysql AUT

* Fix socket timeout connection issue in Mysql AUT

* Fix socket timeout connection issue in Mysql AUT
2025-11-12 16:04:01 +05:30
Mohit Yadav
fbf8cdd05b
Fix Build to use client 1.44 (#24299) 2025-11-12 15:26:06 +05:30
Rohit Jain
cdbae7713e
fixes 23991: Fixed the currentPage reset issue in all the pages (#24278)
* fix: Fixed the currentPage reset issue in all the pages

* fix: fixed the currentPage reset issue in serviceVersion page

* fix: fixed the service details page test

* playwright: Added test for currentPage reset changes

* fix: fixed the unit tests

* fix: addressed pr comments
2025-11-12 11:02:51 +05:30
Anujkumar Yadav
60248fac56
feat: Add common DeleteModal component (#24279)
* feat: Add common DeleteModal component

* Add additional style support

* Add data test id for test
2025-11-12 09:35:18 +05:30
Chirag Madlani
bc7e9be98b
chore(ui): improve lineage control styling with new design (#23770)
* chore(ui): improve lineage control styling with new design

* minor fixes as per design

* update

* fix unit tests

* fix tests

* WIP lineage changes

* fix unit tests

* revert lineage sidebar changes

* fix permission specs

* fix multiple permission tests

* fix permission spec

* fix lineage spec

* fix tests

* fix lineage spec failure

* fix playwright and update platform lineage styles

* update platform view with new layout

* fix tests

* update translations
2025-11-11 18:27:08 +05:30
Chirag Madlani
11566a90b2
chore(ui): bump vite-plugin-dts to latest to resolve vulnerabilities (#24276) 2025-11-11 17:06:33 +05:30
Chirag Madlani
8a52b19fe3
chore(ui): bump Axios resolve vulnerabilities (#24272) 2025-11-11 16:48:35 +05:30
Keshav Mohta
107a3c4946
Fixes #24132: Airbyte Cloud Support (#24261)
* feat: airbyte cloud support - added apis, refactor ingestion code

* fix: added tite in oneOf json & logLink /

* test: airbyte cloud tests

* test: airbyte cloud test json

* fix: airbyte test checkstyle

* fix: api expires_in instead of 3 minutes and typehinting
2025-11-11 16:24:09 +05:30
Eugenio
ef8b19142f
Create documentation resources for Data Quality as Code (closes #23800) (#24169)
* Brief documentation of installation requirements

* Minor fix to run tests only defined in OpenMetadata

* Add full example to Data Quality as Code

* Install `griffe2md` and fix docstrings

* Remove local openmetadata reference

* Fix writing, grammar and typos

* Fix test

* Fix formatting
2025-11-11 10:25:42 +00:00
Chirag Madlani
56656d68cd
chore(ui): bump Vite package to resolve vulnerabilities (#24273) 2025-11-11 14:41:04 +05:30
Aniket Katkar
1906a2a2f7
refactor: remove duplicate directory custom property from GlobalSettingsClassBase (#24265) 2025-11-11 13:51:40 +05:30
Mohit Tilala
79f04bda65
Fix workflow python version picking with quotes (#24263) 2025-11-11 13:24:22 +05:30
Adrià Manero
3e5facdb71
Add testCase description to TestCase notifications (#24252) 2025-11-11 08:39:44 +01:00
Ram Narayan Balaji
5bcf792aa4
Fix: #24100 Implementation for Change Events for Workflow Generated Changes and Manual Task Resolutions (#24108)
* Initial Implementation for Change Events for Workflow Generated Changes and Manual Task Resolutions

* Test case

* Add impersonatedBy to changeEvent schema, changeEvent generations, workflow consumer filtering, always override impersonatedBy even if not null

* Update generated TypeScript types

* Improved debug logging in WorkflowEventConsumer and tests

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-11 10:28:39 +05:30
Chirag Madlani
8514b967e4
chore(ui): remove checker from vite dev mode due to memory issues (#24247) 2025-11-11 10:25:48 +05:30
Mohit Yadav
301d97be9e
[Fix-24250] issue with big param list in findByIds and Names (#24259)
* Fix issue with big param list in findByIds and Names

* Remove table queries from set fields
2025-11-11 10:19:08 +05:30
IceS2
bbd0ce1334
Implement API to retrieve all dimensional test results for a dimensional column (#24255) 2025-11-11 00:09:28 +01:00
Adrià Manero
9ad6783a99
Enable notification template preview and test send (#24229)
* Add NotificationTemplate sending and rendering endpoints with mock data

* Setup entity fixtures for mock notifications

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-10 17:06:26 +01:00
Anujkumar Yadav
679ad3daf8
Fix: Filter out dataProduct and domain from knowledge center data asset selection (#24214) 2025-11-10 21:06:28 +05:30
IceS2
1492539234
Feature/dimensionality column values to be at expected location (#24239)
* Initial implementation for Dimensionality on Data Quality Tests

* Fix ColumnValuesToBeUnique and create TestCaseResult API

* Refactor dimension result

* Initial E2E Implementation without Impact Score

* Dimensionality Thin Slice

* Update generated TypeScript types

* Update generated TypeScript types

* Removed useless method to use the one we already had

* Fix Pandas Dimensionality checks

* Remove useless comments

* Implement PR comments, fix Tests

* Improve the code a bit

* Fix imports

* Implement Dimensionality for ColumnMeanToBeBetween

* Removed useless comments and improved minor things

* Implement UnitTests

* Fixes

* Moved import pandas to type checking

* Fix Min/Max being optional

* Fix Unittests

* small fixes

* Fix Unittests

* Fix Issue with counting total rows on mean

* Improve code

* Fix Merge

* Removed unused type

* Refactor to reduce code repetition and complexity

* Fix conflict

* Rename method

* Refactor some metrics

* Implement Dimensionality to ColumnLengthToBeBetween

* Implement Dimensionality for ColumnMedianToBeBetween in Pandas

* Implement Median Dimensionality for SQL

* Add database tests

* Fix median metric

* Implement Dimensionality SumToBeBetween

* Implement dimensionality for Column Values not In Set

* Implement Dimensionality for ColumnValuestoMatchRegex and ColumnValuesToNotMatchRegex

* Implement NotNull and MissingCount dimensionality

* Implement columnValuesToBeBetween dimensionality

* Fix test

* Implement Pandas Dimensionality for ColumnValueStdDevToBeBetween

* Implement Dimensionality for ColumnValuesStdDevToBeBetween

* Implement dimensionality for column values to be at expected location

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-10 15:00:06 +01:00
Keshav Mohta
011e61de69
dep: setuptools version to 78.1.1 (#24206) 2025-11-10 18:43:05 +05:30
Chirag Madlani
f66c2e6eb0
Feat(ui): support contract tab for other entities (#23079)
* Feat(ui): support contract tab for other entities

* fix the tab being showed in entities along with the next and previous button with the functionality

* modify the contract badge status to all entities in data asset header

* supported topic,dashboardDataModel,ApiEndpoint in schema tab with the json changes

* Update generated TypeScript types

* fix the contract.ts file being deleted

* re-push the missing data in dataContract.json

* supported the contract tab in persona changes and also supported for Chart Entity

* fix the unit test and type error in schema for contract tab

* added playwright test for dataContract for other entities

* fix the sonar test

* added databas and schema contract test along with the previous failure

* added Database and DatabaseSchema contract test and fix other test issue due to merge conflict

* commeted out the tab support for charts now, since BE is still pending on it

* code polish, render schema table on condition with test and schema form default expand all keys

* Update generated TypeScript types

* fix the button props naming

* fix the buttons labels

* modify the playwright and include support for all entities checking

* fix the schema fields for other entities

* added support for fiel, directory, spreadsheet and worksheet and fix some playwright cases

* fix unit and playwright test

* fix some sonar issue

* fix some code smell and minor improvement

* fix unit test

* remove unwanted file

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-10 18:15:12 +05:30
IceS2
dddec06143
Feature/dimensionality column values stddev to be between (#24235)
* Initial implementation for Dimensionality on Data Quality Tests

* Fix ColumnValuesToBeUnique and create TestCaseResult API

* Refactor dimension result

* Initial E2E Implementation without Impact Score

* Dimensionality Thin Slice

* Update generated TypeScript types

* Update generated TypeScript types

* Removed useless method to use the one we already had

* Fix Pandas Dimensionality checks

* Remove useless comments

* Implement PR comments, fix Tests

* Improve the code a bit

* Fix imports

* Implement Dimensionality for ColumnMeanToBeBetween

* Removed useless comments and improved minor things

* Implement UnitTests

* Fixes

* Moved import pandas to type checking

* Fix Min/Max being optional

* Fix Unittests

* small fixes

* Fix Unittests

* Fix Issue with counting total rows on mean

* Improve code

* Fix Merge

* Removed unused type

* Refactor to reduce code repetition and complexity

* Fix conflict

* Rename method

* Refactor some metrics

* Implement Dimensionality to ColumnLengthToBeBetween

* Implement Dimensionality for ColumnMedianToBeBetween in Pandas

* Implement Median Dimensionality for SQL

* Add database tests

* Fix median metric

* Implement Dimensionality SumToBeBetween

* Implement dimensionality for Column Values not In Set

* Implement Dimensionality for ColumnValuestoMatchRegex and ColumnValuesToNotMatchRegex

* Implement NotNull and MissingCount dimensionality

* Implement columnValuesToBeBetween dimensionality

* Fix test

* Implement Pandas Dimensionality for ColumnValueStdDevToBeBetween

* Implement Dimensionality for ColumnValuesStdDevToBeBetween

* Fixed tests due to sqlite now supporting stddev

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-10 12:13:27 +01:00
Adrià Manero
c56edc3df1
Create ChangeEvents for SSO user operations to enable EventSubscription notifications (#24209) 2025-11-10 11:22:04 +01:00
Eugenio
c885a29a8c
Fix Snowflake and BigQuery CLI e2e tests (#24217)
* Create function to normalize PEM strings

* Normalize PEM strings where needed

Also fix a docstring

* Use ingestion bot when running tests
2025-11-10 10:16:20 +01:00
IceS2
ca5945c5be
Feature/dimensionality column values to be between (#24215)
* Initial implementation for Dimensionality on Data Quality Tests

* Fix ColumnValuesToBeUnique and create TestCaseResult API

* Refactor dimension result

* Initial E2E Implementation without Impact Score

* Dimensionality Thin Slice

* Update generated TypeScript types

* Update generated TypeScript types

* Removed useless method to use the one we already had

* Fix Pandas Dimensionality checks

* Remove useless comments

* Implement PR comments, fix Tests

* Improve the code a bit

* Fix imports

* Implement Dimensionality for ColumnMeanToBeBetween

* Removed useless comments and improved minor things

* Implement UnitTests

* Fixes

* Moved import pandas to type checking

* Fix Min/Max being optional

* Fix Unittests

* small fixes

* Fix Unittests

* Fix Issue with counting total rows on mean

* Improve code

* Fix Merge

* Removed unused type

* Refactor to reduce code repetition and complexity

* Fix conflict

* Rename method

* Refactor some metrics

* Implement Dimensionality to ColumnLengthToBeBetween

* Implement Dimensionality for ColumnMedianToBeBetween in Pandas

* Implement Median Dimensionality for SQL

* Add database tests

* Fix median metric

* Implement Dimensionality SumToBeBetween

* Implement dimensionality for Column Values not In Set

* Implement Dimensionality for ColumnValuestoMatchRegex and ColumnValuesToNotMatchRegex

* Implement NotNull and MissingCount dimensionality

* Implement columnValuesToBeBetween dimensionality

* Fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-10 08:03:07 +01:00
Chirag Madlani
727560e88f
fix(test): playwright to create data once instead on each tests (#23756)
* fix(test): playwright to create data once instead on each tests

* fix issue with name conflicts on classes

* sync setup data

* fix customize details page

* fix persistence data issue

* fix test failures

* avoid deleting file

* fix tests

* fix tests with class conflicts

* fix version tests

* fix failing tests

* fix my task failure

* fix failing tests

* fix tests

* revert user list component changes
2025-11-08 12:32:35 +05:30
Ajith Prasad
65f42109e5
Initialize JWT token generator before websocket setup (#24230)
* Initialize JWT token generator before websocket setup

* Added tests for secure socket

* Checkstyle corrected
2025-11-07 20:43:14 -08:00
sonika-shah
c8b759609e
Fix fetchAndSetService for database (#24232)
* Fix fetchAndSetService for service

* add test

* add test
2025-11-08 09:44:01 +05:30
Teddy
f3ef29a117
fix: improve query and minimize snflk <-> OM backand and forth (#24228) 2025-11-07 14:54:37 +01:00
IceS2
4e398d003b
Feature/dimensionality column values to be not null (#24211)
* Initial implementation for Dimensionality on Data Quality Tests

* Fix ColumnValuesToBeUnique and create TestCaseResult API

* Refactor dimension result

* Initial E2E Implementation without Impact Score

* Dimensionality Thin Slice

* Update generated TypeScript types

* Update generated TypeScript types

* Removed useless method to use the one we already had

* Fix Pandas Dimensionality checks

* Remove useless comments

* Implement PR comments, fix Tests

* Improve the code a bit

* Fix imports

* Implement Dimensionality for ColumnMeanToBeBetween

* Removed useless comments and improved minor things

* Implement UnitTests

* Fixes

* Moved import pandas to type checking

* Fix Min/Max being optional

* Fix Unittests

* small fixes

* Fix Unittests

* Fix Issue with counting total rows on mean

* Improve code

* Fix Merge

* Removed unused type

* Refactor to reduce code repetition and complexity

* Fix conflict

* Rename method

* Refactor some metrics

* Implement Dimensionality to ColumnLengthToBeBetween

* Implement Dimensionality for ColumnMedianToBeBetween in Pandas

* Implement Median Dimensionality for SQL

* Add database tests

* Fix median metric

* Implement Dimensionality SumToBeBetween

* Implement dimensionality for Column Values not In Set

* Implement Dimensionality for ColumnValuestoMatchRegex and ColumnValuesToNotMatchRegex

* Implement NotNull and MissingCount dimensionality

* Fix test

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-07 14:44:58 +01:00
Ashish Gupta
14e6b7ccc4
fix some flaky playwright test (#24224) 2025-11-07 18:42:43 +05:30
Pere Miquel Brull
2058b45001
FIX - Logs Ingestion Pipeline tests (#24225) 2025-11-07 13:41:27 +01:00
Mohit Tilala
9f6c07cb1c
Upgrade github workflows to use python 3.10 (#24203) 2025-11-07 16:43:04 +05:30
Adrià Manero
15dcfe7e39
Add bidirectional EventSubscription-NotificationTemplate relationship (#24204)
* Add bidirectional EventSubscription-NotificationTemplate relationship

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-07 12:08:36 +01:00
Harshit Shah
c8a7ded8b8
fix(test): Should display persona dropdown with pagination (#24221)
* fix(test): Should display persona dropdown with pagination

* minor fix
2025-11-07 15:18:13 +05:30