14385 Commits

Author SHA1 Message Date
Mohit Yadav
7f5328f48c
Reapply "Fix Slack Config to be read from UI (#24396)" (#24405)
* Reapply "Fix Slack Config to be read from UI (#24396)"

This reverts commit 437a266ce130eb7b658060bc14ed492aa7d94743.

* Revert schema deletion
2025-11-18 17:15:28 +05:30
Ushran Gouhar
fd7816e690
Fix edges position when node columns are expanded (#24408) 2025-11-18 17:02:45 +05:30
Ashish Gupta
8c9ca5aeca
remove beta tags from DataContract (#24409)
* remove beta tags from DataContract

* fix unit test
2025-11-18 11:28:26 +00:00
Shailesh Parmar
851fdaf482
Feat: Dimensionality ui feature (#24167)
* Feat: Dimensionality ui feature

* Add dimensionColumns to updatedTestCase and Select component

* feat: add Dimensionality Heatmap component with mock data support

- Implemented DimensionalityHeatmap component to visualize test case results over time.
- Added utility functions for data transformation and date range generation.
- Created mock data generator for testing without API.
- Integrated DimensionalityHeatmap into DimensionalityTab with loading state management.
- Updated localization files for new labels and messages.
- Enhanced IncidentManager to include dimensionality tab based on available data.

* Add dimension-related translations to multiple language files

- Added translations for "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality" in German, Spanish, French, Galician, Hebrew, Japanese, Korean, Dutch, Portuguese (Brazil and Portugal), Russian, Thai, Turkish, and Chinese (Simplified and Traditional).
- Included translations for "failed-rows", "no-data", "passed-rows", and "select-dimension" in the respective language files.

* feat: enhance test case dimensions functionality and routing

- Refactor AuthenticatedAppRouter to dynamically handle test case detail routes.
- Update DimensionalityTab to link to dimension detail pages.
- Modify IncidentManagerPageHeader to display dimension key when applicable.
- Adjust TestSummary to support dimension results and update API calls accordingly.
- Introduce new utility function for generating dimension detail page paths.
- Update constants for new route placeholders related to dimensions.

* feat: update DimensionalityTab layout and enhance select component styling

* feat: update DimensionalityHeatmap component and related files for improved status handling and UI enhancements

* feat: add 'entity-text-table' localization key across multiple language files

* Fix PUT/PATCH API for Dimensionality

* feat: Enhance Dimensionality UI with improved localization and tooltip support

* feat: Add BetaBadge component and integrate beta indicators in selection cards and tabs

* fix: Encode FQN in getTestCaseDimensionResultsByFqn API call

* fixed failing unit test

* fix: Update dimension description messages in multiple languages

* feat: Add dimension column selection to EditTestCaseModal

* feat: Add dimension editing functionality to Incident Manager

* feat: Enhance EditTestCaseModal and IncidentManagerDetailPage for improved dimension handling

* feat: Add dimension-level testing functionality and enhance UI components for better data quality analysis

* feat: Implement dimensionality heatmap with tooltip and scroll indicator functionality

* feat: Update dimensionality heatmap cell styles with borders and new background colors

* feat: Enhance dimensionality heatmap with responsive placeholders and dynamic width calculation

* feat: Update HeatmapCellTooltip tests to include MUI theme provider for proper context

* Update translations for dimension-related terms in multiple languages

- Updated Persian (pr-pr) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".
- Updated Brazilian Portuguese (pt-br) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".
- Updated European Portuguese (pt-pt) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".
- Updated Russian (ru-ru) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".
- Updated Thai (th-th) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".
- Updated Turkish (tr-tr) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".
- Updated Simplified Chinese (zh-cn) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".
- Updated Traditional Chinese (zh-tw) translations for dimension-related terms, including "dimension-entity", "dimension-plural", "dimension-value", and "dimensionality".

* feat: Add tests for DimensionalityHeatmap component and enhance utils

- Created a comprehensive test suite for the DimensionalityHeatmap component, covering various rendering states, date range headers, dimension rows, legends, scroll indicators, responsive behavior, data transformations, and ResizeObserver integration.
- Enhanced DimensionalityHeatmap component with accessibility features, including ARIA roles and labels.
- Improved utility functions for generating date ranges and mapping statuses to heatmap statuses, ensuring robust error handling and performance optimizations.
- Updated constants for heatmap layout and tooltip styling for better clarity and maintainability.
- Refactored tests for CustomControls and LogsViewerPage to include theme provider for consistent styling.

* fix: Update color values in defaultColors and variables, enhance DimensionalityHeatmap styles

* feat: Enhance Dimensionality Heatmap with Impact Score and Scroll Indicators

- Added impact score column to the DimensionalityTab component.
- Improved the DimensionalityHeatmap component layout with scroll indicators for better navigation.
- Updated tooltip to display detailed information including impact score.
- Enhanced styles for scroll indicators and status badges.
- Localized new strings for impact score in multiple languages.

* feat: Update Dimensionality UI with scroll indicators and new test cases

* feat: Enhance Dimensionality UI with scroll indicators and impact score helper tooltip

* feat: Update TestCaseForm to include Dimension Level for granular quality analysis

* fix: Refactor DataContracts tests for improved element interaction and response handling

---------

Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2025-11-18 12:06:45 +01:00
Karan Hotchandani
c334689150
feat(ui): domain treeview (#23963)
* initial implementation of domain treeview

* commit progress

* fix: improved the domain treeview UI (#24086)

* fix: improved the domain treeview UI

* fix: Fixed the add button height

* fix: Added search feature in domain tree view

* fix: Implemented new empty states UI for domains Tabs

* fix: added new text in the lang json files

* fix: Fixed the tree view empty state UI

* fix: rename prop name

* fix: extended the error placeholder component with new mui create UI

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* resolve merge conflicts

* PLAYWRIGHT: Domain treeview playwright (#24105)

* fix: Implemented new empty states UI for domains Tabs

* fix: extended the error placeholder component with new mui create UI

* Playwright: Added playwright test for domain treeview

* fix: Immediately display newly added data product in the list without requiring refresh (#24072)

* fix: Immediately display newly added data product in the list without requiring refresh

* Playwright: Added playwright test for data product list

* fix: reverted the skip test

* fix: Fixed minor bugs in domain treeview (#24134)

* fix: fixed domain treeview playwright test (#24138)

* fix: Fixed code smells (#24143)

* fix: Fixed glossary playwright test (#24151)

* Support pagination in domain hierarchy api and add new childrenCount field for domain

* Update generated TypeScript types

* fix total field count in domain hierarchy api

* fix domain/dataProduct filter

* feat: Implemented infinite scrolling in the domain treeview (#24212)

* feat: Implemented infinte scrolling in domain treeview

* fix: Fixed the code smell

* fix: rotate icon on expand and collapse of tree items

* fix: fixed the count mismatch on subdomain add/delete

* fix: removed unwanted field for hierarchy fetch api

* feat: implemented infinte scrolling in navbar domain dropdown

* fix: fixed the search query count bug

* fix: showing the actual children count in treeview

* fix: removed the domain count api from navbar domain dropdown

* feat: added pagination for subdomains in domain treeview

* feat: added pagination in subdomain in navbar domain dropdown

* feat: implemented the filters in domains treeview

* fix: fixed the tags query filter

* fix: hide filter for domain treeview

* fix: assigning single domain to entity

* fix: addressed PR comments

* fix: fixed the domainSelectableTree issue

* minor fix

* fix: fixed entity spec for domain

---------

Co-authored-by: Rohit Jain <60229265+Rohit0301@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Rohit0301 <rj03012002@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-11-18 16:35:06 +05:30
Teddy
3c0d758b80
chore: don't deserialize None field on model_dump (#24407) 2025-11-18 11:12:38 +01:00
Bhanu Agrawal
2362bb21f4
Added missing migration sql files [1.8.5 and 1.10.2] (#24399)
* Added missing migration sql files [1.8.5 and 1.10.2]

* Updated comment
2025-11-18 08:02:35 +01:00
mohitdeuex
437a266ce1 Revert "Fix Slack Config to be read from UI (#24396)"
This reverts commit 421df519a6c8a4f5888d82e82c54036e2bdc703c.
2025-11-18 11:36:54 +05:30
Ayush Shah
8f837c34cc
Fixes 22920: Enhance connection class import handling (#24093) 2025-11-18 11:23:39 +05:30
Mohit Yadav
782a67cc43
Ignore bulk sink exe (#24378)
* Ignore Exception from buildSearchIndex

* Ignore entity Exception for addentity and addTimeSeriesentity

---------

Co-authored-by: Bhanu Agrawal <bhanuagrawal2018@gmail.com>
2025-11-18 11:17:52 +05:30
Mohit Yadav
421df519a6
Fix Slack Config to be read from UI (#24396)
* Fix Slack Config

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-18 11:11:16 +05:30
Rohit Jain
3a9918d23d
Fixes 23497: Added Search and Sort Functionality at Schema Level (#23907)
* Feat: Added search functionality for schema tables and stored procedures

* Fix: memoize the searchProps of the table component

* Fix: fixed the table pagination issue

* Fix: Added search funtionality in service page

* Playwright: Added test for service page search

* Fix: Fixed the falling unit tests

* Fix: Fixed code smells

* fix: Fixed the current page reset issue

* fix: removed the getSearchPlaceholderKey util function

* fix: fixed the merge conflicts

* "fix: Added sorter on name column in all the tables"

* fix: Added search field in data models and api endpoint tab

* fix: Added search field in files and spreadsheets table

* playwright: Added e2e test for table search

* fix: fixed the failed unit test

* add new field service.fullyQualifiedName.keyword in index mapping

* fix: fixed the table search and sort playwright

* fix: fixed the api endpoint schema playwright test

* fix: fixed tha table search and sort playwright

* fix: removed serial from the table search and sort tests

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2025-11-18 10:51:56 +05:30
Ram Narayan Balaji
d1d2e69d3d
Fix #24302: Tasks should be assigned to newly patched reviewers, Metadata fields API should return response for tag, classification, Nitpicks removing the default value in the entity schema (#24275)
* Clean the default value of entityStatus

* Adding reviewers should assign Approval Tasks to them

* Adding om comment to the schema and SchemaExtractor

* Test Cases

* revert local fix

* Add schema field extractor for page, Defensive condition for NPEs

* Changed the subdirectory path to data for page

* Reverted the jsonLinters

* Reverted the jsonLinters in worksheet.json

* Added update Reviewers to Metric Repository. Polling in WorkflowDefinitionResourceTest

* Fix the timeouts of the test case

* Increase polling durations

* Increase Polling Time

* Fix Test cases, remove workflow definitions after the test for approval workflows
2025-11-18 10:05:49 +05:30
Ushran Gouhar
2ad760bbb7
Revamp lineage node (#24251)
* Revamp lineage node

* Fix node handles positioning

* Handle columns toggling

* Using useCallback hook for handlers

* Remove mapping icon

* Fixed unit tests for CustomNode.utils

* Use divs instead of material box to prevent test failing

* Add unit test for columns dropdown button

* Add test to check expand and expand all button present

* Add unit test for expand and expand all btn click

* Add test for observability layer applied

* Remove Show more columns test as that button is removed

* Fix dropdown label text

* Rename from columns to children

* Update test for children dropdown

* Updated styles for column hover and selected

* Updated styles for column hover and selected for dq failed node

* Column styles

* Fix playwright tests for Lineage.spec.ts

* Fix failing playwright test for lineage

* Fix failing UT
2025-11-18 09:18:11 +05:30
Chirag Madlani
299ce12029
fix(test): mentions tests failure (#24391) 2025-11-18 09:15:54 +05:30
Ashish Gupta
d755cb3c2f
chore(ui): fix playwright flaky test on main (#24390) 2025-11-17 22:52:54 +05:30
Harsh Vador
f57d1eb932
Fix search bar typing race conditions (#24300)
* fix(ui): resolve SearchBar typing issues and race conditions

* fix test

* reverted the check on empty response, as we need to show the empty state if user is not found

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-11-17 22:51:51 +05:30
IceS2
8c0215a353
Fix Row Counts for LengthsToBeBetweena nd ValueToBeBetween (#24359) 2025-11-17 15:19:41 +01:00
Mayur Singal
615cb8ec50
Fix #2324: Enable Parallel Execution for Query Runner App (#24256) 2025-11-17 18:42:22 +05:30
Keshav Mohta
6bb0bdb2c2
fix: sample data config in airflow extended sample data (#24381) 2025-11-17 13:47:28 +01:00
Chirag Madlani
2878ac4b01
fix(ui): lineage paging and rendering issues (#24351)
* fix(ui): lineage pagination behavior

* fix pagination issues

* update nodeDepth in case of loadChild

* un-skip cycle lineage spec

* fix unit tests

* Add Direction Based Count

* fix paging issues

* fix util tests

* update tests

* fix cycle lineage test

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2025-11-17 16:58:43 +05:30
Himanshu Khairajani
e857a076e7
Queryrunner clean (#24328)
* config merge draft commit

* fields addition for query-runner

* fields addition for query-runner

* add registry at applications provider

* minor fix

* added password field in form utils

* commit progress

* add delete modal with mui

* supportsQueryRunner flag

* trino test

* Update generated TypeScript types

* add workflow api

* fix tests

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2025-11-17 16:05:25 +05:30
Eugenio
a49b720482
Improvements to accuracy of auto-classification (#24337)
* Add test cases

* Change preprocessing for autoclassification

* Change patterns of recognizers

Affected recognizers:
- `CreditCardRecognizer`: also included a sanitation step before analyzing
- `UsLicenseRecognizer`: pattern that avoids version strings like `V1`
- `AuTfnNumber`: be more specific with patterns

* Remove normalization in processor

The goal is to remove a normalization that doesn't make much sense in an environment where we may get only one result.

Use `tolerance` to filter noise and depend on `confidence_threshold` to remove low score results

* Fix tests
2025-11-17 10:48:12 +01:00
Ashish Gupta
6b3a95f252
allocate 1 worflow to dataAssetRule in nightly (#24368) 2025-11-17 14:14:05 +05:30
Adrià Manero
741556cede
Fix email configuration templates default value from 'collate' to 'openmetadata' (#24352) 2025-11-17 08:39:41 +01:00
Ram Narayan Balaji
ea5eaf885d
Fix: #24372 Field LastDocumentsInBatch to indicate the final Search Documents in the corresponding batch (#24360)
* Field LastDocumentsInBatch to indicate the final Search Documents in the corresponding batch

* fix the wildcard and removed the commented code

* fix the wildcard and removed the commented code

---------

Co-authored-by: Bhanu Agrawal <bhanuagrawal2018@gmail.com>
2025-11-17 13:02:34 +05:30
Ajith Prasad
494d148e6b
Ensure SAML timestamp compatibility with Azure AD validation (#24357) 2025-11-17 12:57:46 +05:30
Keshav Mohta
4e2d63be70
Fix: metadata-rest sink configuration for sample data ingestion (#24371)
* fix: metadata-rest sink configuration for sample data ingestion

* fix: sink config changes in sample_data.json
2025-11-17 07:53:42 +01:00
Harsh Vador
3321dd4bac
ui: Enhance project explorer card (#24281)
* ui: handle empty states in project explorer card

* add locale files

* fix failing tests

* fix failing test

* fix failing test

* fix test

* fix spacing

* add searchbar in tabs & remove view all from custom properties

* add locale files

* fix failing tests

* address feedbacks & code cleanup

* remove unwanted code

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-11-17 10:34:59 +05:30
Pere Miquel Brull
bad44db3fa
MINOR - Schemas to AI Agents (#24345)
* add agents

* ESResponse

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-16 20:08:36 +01:00
Harsh Vador
7745cc1330
fix(ui): Fix duplicate owners & tier field in project explorer card (#24297)
* fix(ui): fix duplicate owner field in overview section in explore card

* nit

* nit

* hide tier row in overview section

* refactor code
2025-11-15 16:29:13 +05:30
Ashish Gupta
e18c848851
Data Asset rule enforcement on ui (#24049)
* data asset rule enforcement on ui

* added multi user and multi team support at same time

* fix the multi user case which wasn't working correctly and clean the ui as well

* fix the rule condition for glossary

* fix the glossary dropdown getting closed on select select after selection where we should wait for footer dropdown button action

* added playwright test for data asset rules table

* fix unit test and supported playwright test for table entity

* change the method and variable name for better readability

* naming convention fix around the glossary validation rule

* supported playwright test for all the entities

* supported in service entities

* spearate the test of rules enable and disable state and added supported in glossaryand teams page of rules owner

* Update generated TypeScript types

* fix the unit test and supported rule in tags page

* fix the generated type issue

* fix he domain single select where de-selecting throwing error and fix type

* modify the playwrigth config so dataAssetRules should be run after the chromium is finish executing so the global run changes won't interrupt other test

* fix all unit test

* fix the test failing in dataAssetHeader

* trying the chromium parallel test running

* modifyed yaml files to run the data asset rules after all machine runs is completed

* change the test running order in playwright config

* reverted the playwright config

* run the data asset rules in only one runner

* added support for multiple user and team combination in ownerLabel compoennt

* fix the placemnet issue, reduce the team font to 12 and minor cleanup

* modify many owner component cases as per ui

* clear ruleEnforcementProvider and useEntityRules hooks

* change the dropdown on owner user to mui component

* fix unit test and make chnages in playwright postgress around test running

* fix unit test failing

* fix some playwright test due to rule enforcement and made changes in CI playeright config to only run dataasset on container 6

* reverted the baseurl

* fix the version view for owner, fix some failing playwright test and re-modify the postgress script to run test

* supported the placement option for owner in case of user and team both and postgress e2e change

* remove the other e2e file changes and fix playwright around rules changes

* fix the e2e command

* fix the failing test around disabled rule and based on comments made fix

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-14 16:19:45 +01:00
IceS2
99bd82ee54
Two fixes for Dimensionality Validators (#24339)
* Fixes

* Rollback pandas changes
2025-11-14 15:18:56 +01:00
Rohit Jain
33380d6ee8
fix: Fixed Bot page pagination playwright test (#24341) 2025-11-14 17:29:14 +05:30
Adrià Manero
780a96b20b
Implement permission system for NotificationTemplates (#24301)
* Implement permission system for NotificationTemplate with field-level controls

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-14 10:14:38 +01:00
Chirag Madlani
3fa90b11ae
fix user tests to have unique IDs in name (#24312)
* fix user tests to have unique IDs in name

* fix tests

* fix user naming issues

* fix query entity
2025-11-14 14:34:04 +05:30
Teddy
678b054fa5
chore: reduce constraints db calls for redshift (#24336) 2025-11-14 08:25:47 +01:00
Sriharsha Chintalapani
587c4058ee
Bulk update APIs for Data Assets (#24262)
* Bulk APIs for Data Assets

* Update generated TypeScript types

* Bulk apis enablement on ingestion

* Ingestion status progress

* Update generated TypeScript types

* Bulk APIs for create/update data assets

* Bulk APIs for create/update data assets

* Update generated TypeScript types

* remove python changes

* Update bulk apis for other entities

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-14 08:24:28 +01:00
Ajith Prasad
91583db2f6
Improve Okta public key URL validation to correctly handle both /oauth2/v1/keys and /oauth2/{authServerId}/v1/keys patterns (#24326) 2025-11-14 11:12:23 +05:30
Suman Maharana
0249368d3a
Fix #23891 : dbt <> snowflake column case issue (#23910) 2025-11-14 10:02:40 +05:30
Ram Narayan Balaji
e831c7dd2a
TRUNCATE Flowable History Tables in both 1.10.5 and 1.10.7 Migration (#24323)
* Revert deleting Old Deployments for Periodic batch Workflows

* Revert "Revert deleting Old Deployments for Periodic batch Workflows"

This reverts commit 7bd1be5a813c209ff9493eedfdb68c918fa68706.

* TRUNCATE FLOWABLE history tables in both 1.10.5 and 1.10.7 migrations

* TRUNCATE FLOWABLE history tables in both 1.10.5 and 1.10.7 migrations
2025-11-13 21:05:31 +00:00
Bhanu Agrawal
509087c8b5
Fix/python fail test postgres (#24316)
* Setup new client for ES/OS

* Migrated createIndex and addIndexAlias methods

* Migrated createAliases method to new ES/OS client

* Migrated updateIndex and deleteIndex methods

* Updated indexExists methods

* refactor: extract index management operations into dedicated manager classes

* Add tests for ElasticSearch and OpenSearch index managers

* chore: fix code style issues"

* Added integeration tests for ES/OS index manager

* Fix log level

* Fixed priority key mapping for test_case_result_index

* fix test to use field name in the query as domains.id not domain.id

* Migrated createEntity and createEntities method

* Fixed failing tests

* Set headers so 8.x client can work with 7.17x or higher server for ES

* Added OpenSearch compatible stemmer configuration

* Fix java code style

* Fixed java checkstyle issue

* Added support for 7.17.x backward compatibility

* Fixed failing tests

* Migrated createTimeSeriesEntity and deleteEntity methods

* Refactor search client architecture with entity management abstraction

* Updated log level to ERROR form WARN

* deleteEntity method impl removed from ES client

* migrated deleteEntityByFields method to new es/os client

* migrated deleteEntityByFQNPrefix method to use new ES/OS client

* deleteEntityByFQNPrefix method removed

* Migrated deleteByScript method to new ES/OS client

* Removed deleteByScript method from SearchClient

* Migrated softDeleteOrRestoreEntity method to new ES/OS client

* Reverted BulkResponse static import

* Migrated softDeleteOrRestoreChildren method to new ES/OS client

* Migrated updateEntity method to new ES/OS client

* Migrated updateChildren method to new ES/OS client

* Removed unused methods

* Migrated getDocByID method to new ES/OS client

* Added info logging

* Fixed parsing issue while passing string doc

* Migrated updateEntityRelationship method to new ES/OS client

* Migrated reindexWithEntityIds method to new ES/OS client

* Added log to show error message for updateEntityRelationship

* Refactored entity manager methods

* Added async client, fixed null obj parsing issue

* Refactoring

* Refactored duplicate methods createEntity and createTimeSeriesEntity

* Added integration tests for ES/OS entity managers

* Updated client availability checks in both index manager classes to use the consistent !isClientAvailable pattern

* Removed comment

* chore: Fix line formatting in Playwright test files

- Adjusted line length in Domains.spec.ts for better readability
- Fixed indentation and formatting in Lineage.spec.ts
- Added missing newline at end of files

* Added new line

* Migrated other index related methods and add tests

* MIgrated updateByFqnPrefix and updateLineage and deleteByRangeQuery methods

* migrated deleteByRangeAndTerm method

* Added integration tests

* Migrated updateColumnsInUpstreamLineage and deleteColumnsInUpstreamLineage methods

* Migrated updateGlossaryTermByFqnPrefix method

* Removed unused method and updated error handling

* Migrated reindexEntities method to new ES/OS client

* Added integration tests for reindexEntities method

* Added interface and impl for generic methods migration

* added default impl

* Migrated deleteDataStream method

* Migrated deleteILMPolicy method

* Migrated deleteIndexTemplate method

* Migrated deleteComponentTemplate emthod

* migrated dettachIlmPolicyFromIndexes method

* migrated removeILMFromComponentTemplate method

* Migrated cluster metric methods

* Removed from ESClient/OSClient to GenericManager

* Migrated getSearchHealthStatus

* Add integration tests

* Migrated buildDIChart and fetchDIChartFields methods

* Migrated listDataInsightChartResult emthod

* Migrated getQueryCostRecords

* Migrated getSchemaEntityRelationship

* Added data insight aggregator tests

* Fixed failing tests

* Migrated aggregate method

* Migrated genericAggregation and aggregate methods

* Fixed failing data insight query

* Updated ref for OS aggregation manager

* Fixed issues with aggregations migration

* Fixed tests

* Migrated lineage methods

* fix aggregate api response

* Fixed aggregation response structure

* Migrated searchBySourceUrl and searchByField methods

* Migrated getEntityTypeCounts, listWithOffset and listWithDeepPagination methods

* Fix failing tests

* Fix tests

* Migrated searchWithDirectQuery method

* Migrated search and previewSearch methods

* Fix tests

* Migrated searchWithNLQ method

* Migrated searchEntityRelationship, searchDataQualityLineage and searchSchemaEntityRelationship methods

* Removed unused fields and methods

* Migrated bulk ES/OS methods

* Migrated DataInsightProcess ES/OS

* Migrated ES/OS bulk and index sinks

* Remove unused search methods

* Removed deprecated HLRC for ES/OS

* Removed maven HLRC ES/OS client

* Fixed collate compile issues

* Fixed collate compile issues

* Fix nested should

* Fix for failing playwright tests

* Fix python failing tests

* fix py format

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2025-11-14 00:08:56 +05:30
Mayur Singal
96224bf9c0
Fix #23325: Deduplicate dbt tags (#24327) 2025-11-13 22:02:12 +05:30
Teddy
dec8a0e9bc
MINOR - bulk create table APIs (#24238)
* Bulk APIs for Data Assets

* Update generated TypeScript types

* Bulk apis enablement on ingestion

* feat: fixed sample data ingestion + removed thread worker

* fix: ran java linting

* chore: align alation tests

* fix: test case ordering

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-11-13 13:53:12 +01:00
IceS2
604bc03f9a
Fix Total Count (#24317) 2025-11-13 13:32:05 +01:00
Mayur Singal
c13ae7269d
Fix #24318: Dbt Nonetype has no attr len error (#24320) 2025-11-13 17:52:05 +05:30
sonika-shah
e53a98f6c0
Fix socket timeout connection issue in Mysql AUT 2 (#24313)
* Fix socket timeout connection issue in Mysql AUT 2

* update connect time
2025-11-13 17:28:04 +05:30
IceS2
a3238edb66
Fix Validators (#24310) 2025-11-13 11:18:15 +01:00
Ashish Gupta
274db6dd7c
fix the tab still visible, even if it has been removed from persona in Charts (#24315) 2025-11-13 14:49:50 +05:30
Bhanu Agrawal
2fb34a9762
Migrate/es os maven hlrc removal (#24283)
* Setup new client for ES/OS

* Migrated createIndex and addIndexAlias methods

* Migrated createAliases method to new ES/OS client

* Migrated updateIndex and deleteIndex methods

* Updated indexExists methods

* refactor: extract index management operations into dedicated manager classes

* Add tests for ElasticSearch and OpenSearch index managers

* chore: fix code style issues"

* Added integeration tests for ES/OS index manager

* Fix log level

* Fixed priority key mapping for test_case_result_index

* fix test to use field name in the query as domains.id not domain.id

* Migrated createEntity and createEntities method

* Fixed failing tests

* Set headers so 8.x client can work with 7.17x or higher server for ES

* Added OpenSearch compatible stemmer configuration

* Fix java code style

* Fixed java checkstyle issue

* Added support for 7.17.x backward compatibility

* Fixed failing tests

* Migrated createTimeSeriesEntity and deleteEntity methods

* Refactor search client architecture with entity management abstraction

* Updated log level to ERROR form WARN

* deleteEntity method impl removed from ES client

* migrated deleteEntityByFields method to new es/os client

* migrated deleteEntityByFQNPrefix method to use new ES/OS client

* deleteEntityByFQNPrefix method removed

* Migrated deleteByScript method to new ES/OS client

* Removed deleteByScript method from SearchClient

* Migrated softDeleteOrRestoreEntity method to new ES/OS client

* Reverted BulkResponse static import

* Migrated softDeleteOrRestoreChildren method to new ES/OS client

* Migrated updateEntity method to new ES/OS client

* Migrated updateChildren method to new ES/OS client

* Removed unused methods

* Migrated getDocByID method to new ES/OS client

* Added info logging

* Fixed parsing issue while passing string doc

* Migrated updateEntityRelationship method to new ES/OS client

* Migrated reindexWithEntityIds method to new ES/OS client

* Added log to show error message for updateEntityRelationship

* Refactored entity manager methods

* Added async client, fixed null obj parsing issue

* Refactoring

* Refactored duplicate methods createEntity and createTimeSeriesEntity

* Added integration tests for ES/OS entity managers

* Updated client availability checks in both index manager classes to use the consistent !isClientAvailable pattern

* Removed comment

* chore: Fix line formatting in Playwright test files

- Adjusted line length in Domains.spec.ts for better readability
- Fixed indentation and formatting in Lineage.spec.ts
- Added missing newline at end of files

* Added new line

* Migrated other index related methods and add tests

* MIgrated updateByFqnPrefix and updateLineage and deleteByRangeQuery methods

* migrated deleteByRangeAndTerm method

* Added integration tests

* Migrated updateColumnsInUpstreamLineage and deleteColumnsInUpstreamLineage methods

* Migrated updateGlossaryTermByFqnPrefix method

* Removed unused method and updated error handling

* Migrated reindexEntities method to new ES/OS client

* Added integration tests for reindexEntities method

* Added interface and impl for generic methods migration

* added default impl

* Migrated deleteDataStream method

* Migrated deleteILMPolicy method

* Migrated deleteIndexTemplate method

* Migrated deleteComponentTemplate emthod

* migrated dettachIlmPolicyFromIndexes method

* migrated removeILMFromComponentTemplate method

* Migrated cluster metric methods

* Removed from ESClient/OSClient to GenericManager

* Migrated getSearchHealthStatus

* Add integration tests

* Migrated buildDIChart and fetchDIChartFields methods

* Migrated listDataInsightChartResult emthod

* Migrated getQueryCostRecords

* Migrated getSchemaEntityRelationship

* Added data insight aggregator tests

* Fixed failing tests

* Migrated aggregate method

* Migrated genericAggregation and aggregate methods

* Fixed failing data insight query

* Updated ref for OS aggregation manager

* Fixed issues with aggregations migration

* Fixed tests

* Migrated lineage methods

* fix aggregate api response

* Fixed aggregation response structure

* Migrated searchBySourceUrl and searchByField methods

* Migrated getEntityTypeCounts, listWithOffset and listWithDeepPagination methods

* Fix failing tests

* Fix tests

* Migrated searchWithDirectQuery method

* Migrated search and previewSearch methods

* Fix tests

* Migrated searchWithNLQ method

* Migrated searchEntityRelationship, searchDataQualityLineage and searchSchemaEntityRelationship methods

* Removed unused fields and methods

* Migrated bulk ES/OS methods

* Migrated DataInsightProcess ES/OS

* Migrated ES/OS bulk and index sinks

* Remove unused search methods

* Removed deprecated HLRC for ES/OS

* Removed maven HLRC ES/OS client

* Fixed collate compile issues

* Fixed collate compile issues

* Fix nested should

* Fix for failing playwright tests

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2025-11-13 14:21:35 +05:30