13249 Commits

Author SHA1 Message Date
Sweta Agarwalla
a9961302df
Updated the count in tabs for containers (#20441)
* updated the count tabs for containers

* minor updates

* add test to check count

* updated as per comments
2025-03-26 21:06:33 +05:30
Dhruv Parmar
899e525532
fixed header wrapping (#20443) 2025-03-26 20:55:28 +05:30
IceS2
690f862177
Fix DI Data Stream delete on OpenSearch (#20447) 2025-03-26 16:09:03 +01:00
Suman Maharana
b4ec3f29a5
Fix - DBT Better Logs and Improved Error Handling (#20433)
* Fix - DBT Better Logs and Improved Error Handling

* lint
2025-03-26 15:38:54 +01:00
Suman Maharana
f9cad9e379
Fix : Test Suite 'NoneType' object has no attribute 'id' Handling (#20444) 2025-03-26 15:38:43 +01:00
Mohit Yadav
cb68a207b7
Fix Migration for postgres (#20440) 2025-03-26 14:27:26 +05:30
Shailesh Parmar
14b6bd3e6f
Add unit-test coverage and loading state improvement (#20423) 2025-03-26 13:44:10 +05:30
IceS2
8c7774e6ae
Fix LongRunning Steps Retriggering Endlessly (#20421) 2025-03-26 08:39:08 +01:00
Rounak Dhillon
482b79f484
Docs: Connectors Troubleshooting Section Unification (#20434)
Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
Co-authored-by: Prajwal214 <167504578+Prajwal214@users.noreply.github.com>
2025-03-26 07:06:52 +00:00
Rounak Dhillon
229355b7bd
Docs: Generate Sample Data Content Correction (#20431)
* Docs: Generate Sample Data Content Correction

* Docs: Generate Sample Data Content Correction

---------

Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
2025-03-26 07:06:10 +00:00
sonika-shah
d696a0c6e7
Minor: Add column count validation when creating table-type custom property (#20418)
* Minor: Add column count validation when creating table-type custom property

* Minor: Add column count validation when creating table-type custom property
2025-03-26 11:47:44 +05:30
Sweta Agarwalla
a2804a8933
Update field changes (#20425)
* update field changes

* fix sonar issue
2025-03-26 10:22:06 +05:30
Mohit Yadav
7e731648ab
Add Data Product Lineage (#20400)
* Add Data Product Lineage

* Migrate Data Products

* add null check for entity

* add data product layer

* Add index specific conditions

* add tests

* update nlp icon

* Add missing conditions

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
2025-03-26 10:02:00 +05:30
Chirag Madlani
3245b2d98e
Entity page redeisgn (#20384)
* initial commit

* support new layout for Table details page

* update tab design for detail pages

* support search withing table

* modify table desing as per mocks

* fix filtering layout

* improvement around some tables as per design

* modify more table as per new design

* minor fix around table padding in top

* modify the pagination for the number antd pagination one

* have changes the description rendere to richTextEditorPreviewNew one for handling 2 line description in table

* updated owner render in tables

* fix unit test failure

* finish layout update for entity page

* fix unit tests

* make the table header fix

* fix container tests

* update styles

* fix the unit test and table css issue

* minor revert

* update

* address comments

* fix unit test

* fix some playwright test

* fix playwright tests

* fix container spec

* fix table spec

* fix playwright tests

* fix playwright tests

* fix playwright tests

* minor clean up around tables

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-03-26 01:49:12 +05:30
Aniket Katkar
f3e90827d6
Improvements: Implement the workflow status based placeholders for platform insight widget (#20369)
* Implement the workflow status based placeholders for PlatformInsights widget

* Add the workflow status banner

* feat(ui): enhance ingestion components with collate agent functionality

- Added support for loading and displaying collate agents in the Ingestion component.
- Updated Ingestion interface to include new props for collate agent management.
- Implemented loading state for ingestion runs and collate agents in IngestionListTable and IngestionRecentRuns components.
- Integrated pagination handling for collate agents in ServiceDetailsPage.
- Improved rendering of pipeline types in IngestionListTable utility functions.

* feat(ui): enhance Ingestion component with agent counts display

- Added agentCounts prop to Ingestion component to display counts for collate agents and metadata.
- Updated Ingestion interface to include agentCounts.
- Modified ServiceDetailsPage to compute and pass agentCounts to Ingestion.
- Improved rendering of sub-tab options in Ingestion component to utilize agent counts.

* feat(ui): update SVG assets and enhance error handling in Service Insights

- Modified existing SVG assets for improved rendering and added new icons: key-hand, no-tier, and security-safe.
- Enhanced AssetsTabs component to utilize updated placeholder icons for no data scenarios.
- Updated TotalDataAssetsWidget to display a custom error placeholder when no records are found.
- Introduced a new utility function to determine the appropriate placeholder icon based on chart type.
- Added a new CSS class for text color styling.

* feat(ui): add refresh functionality to Ingestion component

- Introduced a refresh button in the Ingestion component to reload the agents list.
- Updated Ingestion interface to include a new prop for the refresh function.
- Enhanced ServiceDetailsPage to implement the refresh logic for different agent types.
- Added styling for the refresh button icon in the associated LESS file.

* localization changes

* feat(ui): refactor ingestion components for improved status handling and styling

- Moved query parameter definitions into relevant functions for better encapsulation in IngestionListTable and IngestionRecentRuns components.
- Updated styling for ingestion status badges to enhance visual clarity and consistency.
- Introduced new color variables for improved theming in LESS files.
- Added a delay mechanism for fetching pipeline status in MetadataAgentsWidget to account for service response times.

* Fix the duplicate variables

* Fix the less variable

* Add large table icon and update placeholder icons in Service Insights

- Introduced a new SVG icon for large tables.
- Updated placeholder icons in AssetsTabs and TotalDataAssetsWidget components to use the new icon.
- Changed the color class for placeholder icons from 'text-grey-8' to 'text-grey-14'.
- Refactored the getServiceInsightsWidgetPlaceholderIcon function to accept additional parameters for customization.

* Implement the placeholder messages for service insight widgets

* add the remaining widget descriptions for service insights

* Fix the unit tests

* localization changes

* Fix the app status not showing properly on the external apps logs page

* add createdBy field in test case results

* Fixed sonar issues

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2025-03-26 00:12:27 +05:30
Sweta Agarwalla
fd3f2a2a78
fix alert issues (#20416) 2025-03-26 00:11:59 +05:30
sonika-shah
246c8d4721
Revert feed pagination improvement (#20413) 2025-03-25 22:49:30 +05:30
Shrushti Polekar
1c9268d913
Minor: Links for entity card on Activity Feed (#20415)
* added entity links for entity cards

* address pr comments
2025-03-25 21:39:45 +05:30
Shrushti Polekar
6773e015fa
Fix : Delete user playwright test (#20399)
* fix delete user playwright test

* alert fix for failing test
2025-03-25 21:33:31 +05:30
Shrushti Polekar
801788072e
Minor: Update TagsViewer to show read more and less tags in Summary Cards (#20401)
* update TagsViewer to show read more

* fix failing unit test
2025-03-25 18:49:33 +05:30
Shrushti Polekar
b69f1ace24
fix approve reject btn styles (#20409) 2025-03-25 18:48:48 +05:30
Imri Paran
1db72c2c1e
MINOR: fix: close client after query (#19711)
* fix: close client after query

use context clients in SQL sampler to close the connection once the query is complete

* use self.context_client in all sql sampler implementations

* use sqlalchemy's built-in session management

* format

* format

* use get_client directly
2025-03-25 13:48:18 +01:00
Pranita Fulsundar
007580318b
fix: added service icon (#20414) 2025-03-25 17:36:28 +05:30
Sriharsha Chintalapani
ccd5f12daa
Fix #20200: Import/Export recursively at Database Service, Database, … (#20201)
* Fix #20200: Import/Export recursively at Database Service, Database, Schema

* Recursive import/ export - Fixed errors

* Recursive import - Column not getting updated fixed

* Bug fix - Error message when data length not given for varchar

* Recursive import: Flag added, parsing errors solved

* recursive import: fixed bugs and tests

* added recursive flag for bulk/edit import/export and created entity type filter for easy access

* dismiss the loading state if, dry run fails and websocket doesn't have error in it

* Recursive false for bulk edit

* Recursive import: db service level

* Recursive import: fixed tests

* Recursive import: bug fix

* Recursive import: fixed bug

* support recursive validation and skip bulkImport playright spec and fix some bulk edit playwright test

* Recursive import: bugs

* Recursive import: nested entity bug

* Data type made mandatory for column

---------

Co-authored-by: aji-aju <ajithprasad770@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2025-03-25 17:21:43 +05:30
Aniket Katkar
4b3ce930aa
I worked on fixing the faulty JSON schema and missing UI types (#20407) 2025-03-25 14:46:07 +05:30
Ayush Shah
1434b5dba2
Enhance SQL column processing for BigQuery ingestion (#20408)
- Refactored the handling of nested columns in `sql_column_handler.py` to prioritize source-provided children, ensuring they override any derived children.
- Removed the overridden `_process_col_type` method in `bigquery/metadata.py` to streamline column type handling, enforcing the use of the standard path for BigQuery.

This update improves the accuracy of column metadata processing and simplifies the codebase.
2025-03-25 14:40:46 +05:30
Rounak Dhillon
d344caa8c7
Docs: Troubleshooting Doc Addition in Connectors 1.6 (#20411)
Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
Co-authored-by: Prajwal214 <167504578+Prajwal214@users.noreply.github.com>
2025-03-25 09:01:13 +00:00
IceS2
254420bbb1
MINOR: Update Default Configurations (#20398)
* Update Default Configurations

* Update Airflow Default Scheduling

* Update Airflow Default Scheduling
2025-03-25 09:53:01 +01:00
Rounak Dhillon
662741e496
Docs: Troubleshooting Doc Addition in Connectors (#20410)
* Docs: Troubleshooting Doc Addition in Connectors

* Docs: Troubleshooting Doc Addition in Connectors

---------

Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
2025-03-25 08:35:00 +00:00
Pranita Fulsundar
fc2133dd82
fix(ui): new settings card layout and new icons (#20393)
* fix: setting card layout and new icons

* style: icon container height and width

* remove unnecessary icons

* fix height of card
2025-03-25 12:10:16 +05:30
dependabot[bot]
c0f819058d
Chore(deps): Bump @babel/runtime-corejs3 (#20196)
Bumps [@babel/runtime-corejs3](https://github.com/babel/babel/tree/HEAD/packages/babel-runtime-corejs3) from 7.15.3 to 7.26.10.
- [Release notes](https://github.com/babel/babel/releases)
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md)
- [Commits](https://github.com/babel/babel/commits/v7.26.10/packages/babel-runtime-corejs3)

---
updated-dependencies:
- dependency-name: "@babel/runtime-corejs3"
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2025-03-25 10:19:36 +05:30
Sriharsha Chintalapani
8947c7efbf
fix: openmetadata-service/pom.xml to reduce vulnerabilities (#19603)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-ORGECLIPSEJETTY-8186141

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2025-03-24 15:31:44 -07:00
Ayush Shah
a9d6b5760d
Fixes #18346: feat(BigQuery) Enhance project ID discovery and ADC support (#20085)
* Refactor SQL column processing and enhance BigQuery project ID handling

* Introduced a new `process_column` function in `sql_column_handler.py` to streamline column processing logic.
* Updated `BigquerySource` to improve project ID retrieval from service connections, ensuring compatibility with various credential types.
* Added handling for nested columns in BigQuery schema processing.
* Enhanced error handling and logging for better debugging during project ID setup.

* Add support for GCP Application Default Credentials in BigQuery ingestion

* Enhanced `BigquerySource` to include handling for GCP Application Default Credentials (ADC).
* Updated JSON schema for GCP credentials to define `gcpADC` and its properties.
* Improved logging for credential setup in `set_google_credentials` function.
* Added comments and TODOs for future enhancements related to project ID fetching from the resource manager.

* Update .gitignore to include cursor rules files

* Added .cursorrules and .cursor/ to the .gitignore to prevent tracking of cursor rule files in the repository.
* This change helps maintain a cleaner repository by excluding unnecessary files from version control.

* refactor: Bigquery Credentials to allow multiple project ids

* fix: Handle unknown array data types in SQL column processing
2025-03-24 15:31:10 -07:00
Matias Puerta
5d20d7915c
CHORE - Trivy schedule (#20016)
* CHORE - Change trivy to scheduled execution

* Fix filename
2025-03-24 17:54:28 +01:00
Sriharsha Chintalapani
a510ac0a27
Update en-us.json (#20203) 2025-03-24 22:10:19 +05:30
Shailesh Parmar
81f0f08be2
Feat: assets upload supporting PR (#20385)
* Feat: added support for assets upload

* updated ui for file attachment

* combine extention

* refactore fileNodeView

* fixed failing unit test
2025-03-24 22:01:34 +05:30
Rounak Dhillon
a209ada9e2
Docs: Reverse Metadata Doc Updation (#20392)
Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
2025-03-24 09:26:40 -07:00
IceS2
deca9f9574
Fix Workflow State not Updating on Exception (#20395) 2025-03-24 09:26:05 -07:00
Chirag Madlani
e91f8a262f
fix: openmetadata-ui/src/main/resources/ui/package.json & openmetadata-ui/src/main/resources/ui/yarn.lock to reduce vulnerabilities (#20391)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JS-AXIOS-9403194

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2025-03-24 09:25:05 -07:00
Imri Paran
640eedde35
MINOR: docs: migration process (#20371)
added javadoc for org.openmetadata.service.migration.api.Migration
2025-03-24 16:42:38 +01:00
Dhruv Parmar
52e07cf2c3
Redesign : certificates and source URL (#20368)
* Redesigned certificate || source url

* updated extrainfolabel

* updated extrainfolabel

* fixed metricHeaderInfo|| gave consistent spacing

* fixed test cases

* fixed metric text overflow

* fixed tests

* fixed comments

* fixed sonar cases
2025-03-24 20:52:39 +05:30
Karan Hotchandani
30239bdf05
fix - application configurations for reverse metadata (#20320)
* fix query builder state for custom properties

* update the app run records for external apps

* fix styling of status

* add zoom and pan for minimap

* fix sonar error
2025-03-24 18:44:41 +05:30
Gyowanny P. Queiroz
6e3c276478
Ingestion agent updates (#20240)
* updated for ingestion agent

* update playwright

* update for connection tab

* update test

* Update mappers with ingestionAgent

* Docs: 1.6.6 Release Updation (#20234)

* Docs: 1.6.6 Release Updation

* Docs: Docker and python version Updation

---------

Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>

* DOCS - Airflow Base URL for lineage backend (#20255)

* FIX #20250 - Suggestions to handle mutually exclusive tags (#20251)

* UI improvements

* Update ingestionAgent relationship on service entity updater

* Sweta - UI improvements

* updated UI elements

* updated as per comments

* minor fixes

* update test

* remove agent from create pipeline

* remove agent from create pipeline

* remove agent from create pipeline

---------

Co-authored-by: Sweta Agarwalla <swetaagarwalla13@gmail.com>
Co-authored-by: Rounak Dhillon <162090200+RounakDhillon@users.noreply.github.com>
Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2025-03-24 14:11:52 +01:00
Sriharsha Chintalapani
03e3e6d651
Add private key configs to ObjectStorage (#20386) 2025-03-24 18:34:45 +05:30
Shrushti Polekar
1c467d0ee1
error placeholder bg fix (#20394) 2025-03-24 18:32:15 +05:30
Rounak Dhillon
ed87bf892a
Docs: Docker Example Removal from Docker (#20396)
Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
2025-03-24 18:14:53 +05:30
Teddy
5a3fc2cfa4
feat: add created by for test case entities (#20304)
* feat: add created by for test case entities

* migration: createdBy sql migration
2025-03-24 13:13:01 +01:00
sonika-shah
3684a2661b
Add Announcement Count to Feed Count API (#19600)
* Add Announcement Count to Feed Count API

* add inactive and active announcement count
2025-03-24 17:37:44 +05:30
Chirag Madlani
d0f4080e37
fix(test): customize navigation playwright (#20388) 2025-03-24 14:54:43 +05:30
Imri Paran
451506cb08
MINOR: chore: added missing timestamp indexes for time series tables (#20373)
* chore: added index for apps_extension_time_series.timestamp

* - added timestamp index for workflow_instance_time_series
- added timestamp index for workflow_instance_state_time_series
2025-03-24 07:43:07 +01:00