* Refactor SQL column processing and enhance BigQuery project ID handling
* Introduced a new `process_column` function in `sql_column_handler.py` to streamline column processing logic.
* Updated `BigquerySource` to improve project ID retrieval from service connections, ensuring compatibility with various credential types.
* Added handling for nested columns in BigQuery schema processing.
* Enhanced error handling and logging for better debugging during project ID setup.
* Add support for GCP Application Default Credentials in BigQuery ingestion
* Enhanced `BigquerySource` to include handling for GCP Application Default Credentials (ADC).
* Updated JSON schema for GCP credentials to define `gcpADC` and its properties.
* Improved logging for credential setup in `set_google_credentials` function.
* Added comments and TODOs for future enhancements related to project ID fetching from the resource manager.
* Update .gitignore to include cursor rules files
* Added .cursorrules and .cursor/ to the .gitignore to prevent tracking of cursor rule files in the repository.
* This change helps maintain a cleaner repository by excluding unnecessary files from version control.
* refactor: Bigquery Credentials to allow multiple project ids
* fix: Handle unknown array data types in SQL column processing
* fix query builder state for custom properties
* update the app run records for external apps
* fix styling of status
* add zoom and pan for minimap
* fix sonar error
* feat: approve and reject action for glossaryTerm
* fix: getTaskData logic and status icons
* fix: tests
* fix: approve/reject button style and test cases
* update position of status filter
* style: update color variables
* remove extra fetchTasks call
* fix: logic of update glossaryTerm after status change
* fix: updateGlossaryTermStatus function
* fix: update glossary term status
* feat(ui): allow export as pdf from element directly
* supported lineage export as pdf
* added header in pdf file of entity name
* move the export logic in the export provider and have export type selector in modal
* added playwright test for export pdf
* added png export, and convet lineage pdf export to png
* minor optimization
* remove pdf lib and modify code to extend further and added loading state
* fix sonar issue
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
- include the junit for UI in the test report in github actions
- this will solve the false positive for "publish test report" and reduce feedback loop for getting failing tests
* feat: allow user to turn off rnadomized sample
* style: ran python linting
* fix: models default value for randomizedSample
* style: ran linting
* doc: move config to advanced
- Updated ServiceInsightsTab to fetch data for the last 6 days instead of 7.
- Enhanced PlatformInsightsWidget to conditionally display icons based on percentage change.
- Passed serviceDetails prop to CollateAIAgentsWidget in Ingestion component.
- Replaced uiSchema in FiltersConfigForm with SERVICE_CONNECTION_UI_SCHEMA for better structure.
- Introduced SERVICE_CONNECTION_UI_SCHEMA in constants for consistent UI form ordering.
- Added utility function to generate an array of timestamps for the last 7 days.
These changes improve data accuracy and UI consistency across components.
* Fix#19667: OpenSearch Connector
* Fix#19667: OpenSearch Connector
* do not ingest any system level indexes
* fix pyformat
* Add AWS auth
* Use common schema and fix ssl config in client
* Add openseach connector docs and update schema
* Remove api key auth type and complete docs checklist
* Remove unnecessary httpx dependency and pyformat
* Add compatible version of httpx for elasticsearch
* Fix pylint fails and py-tests validation error
---------
Co-authored-by: Mohit Tilala <tilalamohit123@gmail.com>
Co-authored-by: Mohit Tilala <63147650+mohittilala@users.noreply.github.com>
* update the service data insights charts to give percentage data
* fix the quey_cost_record_search_index not found error
* Fix java checkstyle
* Refactor Service Insights Tab and Widgets for Enhanced Data Visualization
- Updated ServiceInsightsTab to utilize new date utility functions for fetching chart data.
- Modified chart data structure to include percentage changes and current percentages.
- Enhanced PlatformInsightsWidget to display percentage values and improved tooltip functionality.
- Added new styles for better layout and responsiveness in TotalDataAssetsWidget.
- Removed unused SERVICE_INSIGHTS_CHART constant and integrated its logic directly into the fetching process.
- Introduced new enums for additional chart types in DataInsight.
This refactor aims to improve the clarity and usability of the service insights data presentation.
* Localization changes
* Fix the platform insights chart colors
* Make the insights tab as default for service details page
* Enhance Ingestion Component and List Table
- Added logic to conditionally render search dropdowns based on the selected sub-tab in the Ingestion component.
- Increased column widths for 'type', 'count', and 'action' in the IngestionListTable for better visibility.
- Adjusted scroll width in the IngestionListTable to accommodate new column sizes.
- Updated styles in metadata-agents-widget.less to ensure vertical alignment of table cells.
- Modified applicationAPI to include agentType in the application list parameters.
- Enhanced router utility to support sub-tab routing.
These changes aim to improve the user experience and layout of the Ingestion settings.
* Fix the playwright failures
* Fix the playwright tests
* fix notification alert flakiness
* Fix metadata agents table header styling
* localization changes
* Fix the flaky test
* supported async delete functionality for service entities
* remove the multiple delete showing logic in ui and handle the recursive false case in case entity is not empty
* added unit test for the provider
* change playwright as per async delete
* change the toast notification to success for the delete process being started
* fix playwright test
* fix playwright failure
* fix the service ingestion playwright failure
* update test connection modal to include error messages
* update test
* update as per comments
* updated component
* fix the error message and improve the code with unit test
* minor improvement
* fix the unit test and some improvement
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
