4809 Commits

Author SHA1 Message Date
Karan Hotchandani
887ffab7e1
MINOR - fix config for workflows (#18628)
* fix config for workflows

* minor spacing

* Fix certification spec

* Fix checkstyle

---------

Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2024-11-13 18:23:12 +01:00
Mohit Yadav
ee327ce0c4
Fix Unlimited Queue Size Issue (#18633) 2024-11-13 22:02:34 +05:30
Shailesh Parmar
7caec17162
fix: logs loading issue and reduce repetitive api call (#18630) 2024-11-13 20:01:12 +05:30
Sachin Chaurasiya
21becf6bff
fix: checkmark saves wrong custom property field (#18623) 2024-11-13 17:52:23 +05:30
Pranita Fulsundar
00357f68fc
Fix 16805: pagination dropdown issue (#18611)
* fix: pagination dropdown

* fix: remove handlePageChange dependency

* fix: api url for database schema response

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-11-13 17:47:19 +05:30
Karan Hotchandani
7cba2a7f51
fix lineage tests (#18618) 2024-11-13 13:29:51 +05:30
Mohit Yadav
30ee4fafb5
Update Default Values (#18621) 2024-11-13 13:24:16 +05:30
IceS2
e5441a9969
Initial implementation for Governance Workflows (#17837)
* Initial implementation for Governance Workflows

* Remove file that shouldn't be there

* Initial implementation for the governance workflows

* Refactor JSONSchema for Governance Workflows

* Refactored WorkflowInstanceStatus to keep the stage properly

* Fix Migration for MySQL for the WorkflowInstanceStates change

* Update WorkflowInstanceState to track all the states propertly

* Initial code tro migrate Request type of Tasks

* copy governance schemas into json directory

* update schema names

* Extract WorkflowInstance into its own resource

* Update WorkflowInstance to have the workflow starting variables

* Changed relatedEntity to be an EntityLink

* First iteration on checkEntityAttributes JSONLogic

* Refactored Workflow into TriggerWorkfow and MainWorkflow

* Fixed WorkflowInstance and WorkflowInstanceState

* Fixed WorkflowInstance and WorkflowInstanceState when process is terminated

* Fix UserTask

* json logic changes to workflow

* Fix Patch request. Add exclude filter to EventBasedEntityTrigger

* update query builder

* Fix Migrations

* Fix Migrations

* modify query builder widget

* fix search props

* update widget and use default fields

* Update edges to always have a true/false condition

* made changes to provider for fields

* push progress

* fix grouping of array field

* Remove duplicated migrations

* Fix migrations

* Fix migrations

* Fix Tests and commit some improvements suggestion

* fix unit tests

* Add Certification Workflow

* Fix Tests

* Fix Tests

* fix tests

* add delay in pagination tests

* fix minor config changes for json logic

* change the glossary page

* Remove Table Certification Workflow seed Data

* add json logic and elastic search parsers

* minor schedule interval changes for workflows

* minor fixes to the parsing logic

* certification ui

* fix tests

* fix tests

* add other search fields

* Add Certification to search index

* Update index mapping for certification

* add quick filter for certification

* update locales

* Fix index mapping

* Add missing index updates

* Fix checkstyle

* increase timeout

* skipping user test

* skipping team test

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2024-11-13 08:47:44 +01:00
Kenil Shah
ac3728fe67
update icon for suggestions description (#18617) 2024-11-13 13:05:15 +05:30
Mohit Yadav
9a5dc61ca7
Fix Search Index Contention (#18605)
* Fix Search Index Contention

* Update searchIndexingAppConfig.json

* Missing Error Logs and Stats

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-11-12 20:36:23 +05:30
IceS2
14e645368d
MINOR: Fix the Daily Active Users Summary Card (#18589)
* Changes Sum to Mean, in order to display Avg Active Users during the last n days

* Fix Typescript issue

* Update Summary Label
2024-11-12 12:15:22 +00:00
Sriharsha Chintalapani
1c05415321
Improve parallelization in SearchIndexApp (#18556)
* Improve parallelization in SearchIndexApp

* Improve parallelization in SearchIndexApp

* add countdown latch

* typo

* Status issues - #1

* Improve search indexing

* cleanup the code

* schemas

* Status Updates Via WebSockets

* Add Updates

* Make Logs Debug

* show duration in table

* add searchIndexApp params to openmetadata-ops.sh reindex commandline

* Fix code style

* Add Parallel Entity Level Reader

* Make code more readable

* Send App Run Record instead of jobData

* Sned App Run Record WebSocket for Data Insights

* ui: use socket connection to get the real-time updates of application data

* Send WebSocket Final Update

* ui: remove failedRecords, totalRecords and successRecordsfrom entities record

* Invert Condition

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-11-11 08:13:54 -08:00
Teddy
45d27a377d
GEN 1184 - Added Workflow Classification and Metric LevelConfig (#18572) 2024-11-11 15:59:42 +01:00
Shailesh Parmar
b59983d652
Minor: fixed sample data overlapping issue (#18581) 2024-11-11 17:48:30 +05:30
Shailesh Parmar
eeb27c3cbf
Minor: lineage data quality failure tracing support (#18580) 2024-11-11 15:21:29 +05:30
Sriharsha Chintalapani
7f5f47f618
Add export apis back and new apis should be called exportAsync (#18552)
* Add export apis back and new apis should be called exportAsync

* ui: use exportAsync endpoint for export assets

* fix build failure

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-11-08 10:35:02 -08:00
Suman Maharana
da039b197f
Add: Azure Data factory Connector (#18543)
* Added Azure Data factory Connector

* Added Lineage data factory

* removed not required files

* removed not required files

* Removed datafactory ui changes from oss

* resolve merge conflicts

* resolve merge conflicts

* added python requirements
2024-11-08 07:38:45 +01:00
Shrushti Polekar
f60e113026
name column sorting functionality for tables (#18547) 2024-11-07 22:55:51 +05:30
Pranita Fulsundar
76c017a8aa
GEN-2009: update the delete message for ingestion pipeline (#18546) 2024-11-07 21:40:27 +05:30
Shailesh Parmar
198dd35cc1
Minor: style improvement for DataStatisticWidget (#18544) 2024-11-07 19:09:03 +05:30
sonika-shah
e0d175f78e
GEN-1894 GEN-771 : Improvement - Add displayName field for custom property (#18496) 2024-11-07 16:59:59 +05:30
Chirag Madlani
6fa76ecde9
feat: persona based customisation for glossary and glossary term pages (#18454)
* Initial schemas for UI customization

* feat(ui): persona customization for entity details page
including Glossary & Glossary term detail page too

* WIP glossary term details page customization

* update persona epic

* WIP persona customization

* fix errors and warning

* WIP personalize glossary term

* WIP persona

* WIP glossary terms customization

* WIP glossary terms customization

* update glossary term customization

* remove glossary term reference from generic component

* use more of generic provider

* improved UI UX for customization

* complete glossary term customization
start with glossary customization

* apply backend linting

* support customization for glossary page

* fix issue with new tab on glossary page

* address feedbacks

* fix tests

* fix playwright for customize landing page

* fix tests

* fix tests

* address sonar errors

* fix glossary test

* fix glossary test

* fix failing playwright tests

* address comments

* fix conflict error

* add more widgets

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2024-11-07 11:13:12 +05:30
Sachin Chaurasiya
305f02c62a
Minor: improve the entity export modal feedback message (#18526) 2024-11-06 21:05:07 +05:30
Shailesh Parmar
b43c2ba4bd
Refactor: Improvement for data quality overview tab (#18523)
* Refactor: Improvement for data quality overview tab

* unite test for new component

* minor improvement

* remove unused icon
2024-11-06 20:59:54 +05:30
Aniket Katkar
5863c7e235
Add what's new content for 1.5.11 (#18534) 2024-11-06 18:53:14 +05:30
Karan Hotchandani
3dcbeb469f
App settings - lineage config (#18522)
* add lineage settings page

* update locales

* minor ui changes

* add playwright for lineage config
2024-11-06 10:26:18 +00:00
Mayur Singal
5660a751e3
GEN-2000: Add Support for PowerBI Report Server (#18513) 2024-11-06 14:55:05 +05:30
Ashish Gupta
76a257f3aa
MINOR: fix create admin page data and fields (#18458)
* fix create admin page data and fields

* fix issue around adminFlag undefined

* remove roles filed in case of admin listing game

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-11-05 20:11:27 +05:30
Sriharsha Chintalapani
b763091bf6
Make Export CSV Async API, websocket to push data back (#18497)
* Make Export CSV Async API, websocket to push data back

* ui: implement csvExportChannel to receive entity export data via websocket

* ui: update flow to show dialogbox with the response message and once data is available download the file

* minor: remove unused code

* ui: show inline banner for success and error message

* remove unused locale keys

* ui: clear existing csvexport job on sucess

* fix failing UserResourceTest

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-11-05 18:32:41 +05:30
Shailesh Parmar
7f2240698b
Feat: supporting pr for incident manager widget feature (#18488) 2024-11-04 10:41:39 +05:30
Sriharsha Chintalapani
680ace99b3
Fix #18332: Do not consolidate changes for Parent Entities (#18333)
* Fix #18332: Do not consolidate changes for Parent Entities

* Fix #18332: Do not consolidate changes for Parent Entities

* added no data placeholder when no tags there

* fix failing playwright test due to consolidation changes and added the missing services

* Fix tests

* Fix parent entity change notifications

* Fix tests

* Fix parent entity change notifications

* Fix tests

* Fix tests

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-10-31 11:48:46 -07:00
Nicola Coretti
7ebc62dca7
feat: Add support for exasol datasource (#17166)
* Add flake.nix

* Add lockfile for flake

* Update nix environment and document usage

* Add schema for exasol connector

* Add Exasol definitions to databaseService

* Fix error in exasol connector schema

* Add additional connection options/settings to exasol connector

* Add exasol-connector to ui

* Add depdencies for exasol-connector

* Update notes

* Update ingestion code

* Add Basic Documentation for Exasol Connector

* Update flake file

* Add developer notes

* Add python script which can be used as entry point for debugging in ide

* Add config file which can be used for debugging (manual execution)

* Update debug script

* Update developer notes

* Remove old developer notes

* Add .venv to gitignore

* Update dev notes

* Update development notes

* Update ExasolSource

* Establish basic connection to Exasol DB from connector

* Update exasol connector connection settings

* Add service_spec for exasol plugin

* Remove development files

* Remove unused module

* Applied code formatter

* Update exasol dependency constraint(s)

* Add unit test for exasol connection url(s)

* Fixed test expectations for exasol connection url test(s)

* Adjust the test query for the Exasol connection test
2024-10-31 08:11:30 +01:00
Karan Hotchandani
a9ec754eca
Update whatsNewData.ts (#18483) 2024-10-31 09:29:27 +05:30
Karan Hotchandani
b27603837c
update 1.5.10 whats new data (#18482) 2024-10-31 09:02:08 +05:30
Karan Hotchandani
5c3ad966a1
fix empty rule state in query builder (#18481) 2024-10-31 09:01:26 +05:30
Karan Hotchandani
854c3d6cca
fix query encoding for team asset query (#18480) 2024-10-31 08:58:12 +05:30
Ashish Gupta
23f13818a1
improvement around table constraint modal and added beta tag in tabs (#18469)
* improvement around table constraint modal and added beta tag in tabs

* supported localization key

* improvement around beta tag css

* supported dataType icons for columns

* remove beta flag from the tested tabs

* change icon make as per file standard
2024-10-30 22:17:02 +05:30
Karan Hotchandani
a47235581f
add debounce on autocomplete in advanced search (#18471) 2024-10-30 20:44:37 +05:30
sonika-shah
b42e211d29
Feat - Add import/export custom property for other entities - databaseService, database, databaseSchema level (#18385)
* Feat - Add import/export custom property for other entities - databaseService, database, databaseSchema level

* Fix java checkstyle

* Fix dbService level export

* minor: improve the inline edit of custom property

* minor: improve the number type icon

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-30 16:32:47 +05:30
Chirag Madlani
289404748a
fix(ui): block multiple paging req while fetching data (#18436) 2024-10-30 14:05:44 +05:30
Shailesh Parmar
ec3f9cf9e8
Feat: Data quality dashboard (#18146)
* Feat: Move data quality page tabs to left panel

* fixed failing unit test

* fixed failing playwright

* added support for dashboard in enum

* Add support for dashboard in enum and custom pie chart component

- Added support for dashboard in enum by creating a new file `Chart.constants.ts` and exporting the constant `CHART_BASE_SIZE`.
- Updated the `Chart.interface.ts` file to include the `CustomPieChartData` type and the `CustomPieChartProps` interface.
- Added a new file `CustomPieChart.component.tsx` to implement the custom pie chart component.
- Updated the `Color.constants.ts` file to import the `DEFAULT_THEME` from `Appearance.constants`.
- Added a new file `TestCaseStatusPieChartWidget.component.tsx` to implement the test case status pie chart widget.

Related to #18155

* Refactor: Update language translations and add initial entity health matrix

This commit refactors the language translations in the UI, specifically adding translations for "Healthy" and "Unhealthy" in multiple languages. Additionally, it introduces the initial entity health matrix, which includes the counts for healthy and unhealthy entities.

* Refactor: Update language translations and add initial entity health matrix

- Refactor the code to update language translations and add an initial entity health matrix.
- Add a new constant `INITIAL_ENTITY_HEALTH_MATRIX` to represent the initial health matrix of entities.
- Add a new constant `INITIAL_DATA_ASSETS_COVERAGE_STATES` to represent the initial coverage states of data assets.

* Refactor: Update language translations and add initial entity health matrix, added open incident area chart widget

* Refactor: Update language translations and add initial entity health matrix, added open incident area chart widget

* Refactor: Add IncidentTimeChartWidget component and related interfaces

This commit adds a new component called IncidentTimeChartWidget, which is responsible for displaying a chart widget related to incident time metrics. It also includes the necessary interfaces for the component.

The IncidentTimeChartWidget component fetches incident time metrics data from the server and displays it in a custom area chart. It calculates the average time value based on the fetched data and renders it along with the chart.

The commit also includes updates to the Chart.interface.ts and CustomAreaChart.component.tsx files to support the new component.

Issue #TBD

* Refactor: Update language translations and add resolution time and response time translations

* Refactor: Add TestCaseStatusAreaChartWidget component and related interfaces

* implemented filters for dq dashboard

* added description for menu

* move default tab view logic to classBase

* Refactor color variables and add unit test for utility function

* added unit test for dataQualityDashboardAPI

* added unit test for new component

* added unit test for new component

* updated the UI

* added unit test

* Refactor date formatting in KPIChart and DataQualityUtils

This commit refactors the date formatting in the KPIChart component and the DataQualityUtils module. In the KPIChart component, the tickFormatter function in the XAxis component is updated to use an arrow function instead of the formatDate function. In the DataQualityUtils module, the formatDate function is modified to support UTC formatting when necessary. Additionally, two functions, getStartOfDayInMillis and getEndOfDayInMillis, are added to the DateTimeUtils module to calculate the start and end of a day in milliseconds.

* fixed failing unit test

* updated icon

* refactore the code

* update the icon for sql

* fixed java checkstyle

* fixed playwright test

* minor
2024-10-29 18:18:49 +05:30
Ashish Gupta
843f8439ce
revamp table constraint modal design (#18455) 2024-10-29 18:16:30 +05:30
Kenil Shah
aea12fe4d5
fix: update settings tab with proper values (#18453)
* fix: update settings tab with proper values

* address feedback comment
2024-10-29 10:45:05 +01:00
Teddy
4f8bf05ded
GEN 1931 - Fix entity link accepted chars (#18391)
* fix: entity link accepted characters

* fix: match all characters but `<>|::`

* fix: remove unnecessary negative lookbehind

* fix the test case search not working on add ingestion page in case of special characters in FQN

* style: ran java linting

* fix: failing testst

* style: ran java linting

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-10-29 10:33:08 +01:00
Suman Maharana
67a9e63439
Minor: Fixed dbtcloud test connection and improved docs (#18408) 2024-10-29 14:39:52 +05:30
Chirag Madlani
e197c3f3d0
fix(ui): unable to add more enum values for enum cp (#18440)
* fix(ui): unable to add more enum values for enum cp

* replicate height constraint and dropdown visibility to add cp
2024-10-29 12:18:51 +05:30
Chirag Madlani
5c550e7bc3
fix(ui): navigate to listing for deleting a service (#18428) 2024-10-29 10:56:17 +05:30
Karan Hotchandani
d82d41e00a
add whats new (#18439) 2024-10-29 04:59:06 +00:00
Aniket Katkar
14760021c0
Add Galego language to the localization dropdown (#18429) 2024-10-28 21:03:29 +05:30
Ashish Gupta
5b49db2c07
MINOR: fix organization data show in teams data after search (#18427)
* fix organization data show in teams data after search

* remove unsued code
2024-10-28 21:01:56 +05:30