4531 Commits

Author SHA1 Message Date
Aniket Katkar
c30d851dad
Add the missing pipeline type in the localization key mapping object (#18692) 2024-11-19 18:24:35 +05:30
Ashish Gupta
484a9584ea
MINOR: added domain data in lineage side panel and fix relatedColumn search TableConstraints modal (#18671)
* added domain data in lineage side panel

* localization keys

* fix unit test

* fix unit test and minor improvement around permission in schema page utils

* added the encoding and escape method to query
2024-11-19 17:03:15 +05:30
Kenil Shah
ff261fb373
fix: overflowing metrics for kpi charts (#18683) 2024-11-19 13:26:00 +05:30
Sachin Chaurasiya
eb99a8563e
Minor: Add SanitizedInput Component for Input Sanitization (#18674)
* Minor: Add SanitizedInput Component for Input Sanitization

* use SanitizedInput in formutils for text type input
2024-11-19 12:45:14 +05:30
sonika-shah
853bbc205b
GEN-1983 : Feat - Support Tag Asset Page (#18505)
* Feat :  Support bulk adding tag to Assets

* remove warnings

* add apis for assets remove

* Fix: Add tag page (#18461)

* add tag page which all Assets

* update as per feedbacks

* update as per feedbacks

* add divider in header badge

* remove styling

* update permission and refactoring code

* updated as per comments

* fix sonar cloud issues

* add delete asset functionality

* refactor entityTypeString

* made the top bar fixed to top

* add tests for tag page

* fix check failures issue

* fix tag page check failure

* fix flaky test issue

* add tag page tests

* update the add asset test

* update playwright tests

* add right panel in tag page

* updated as per feedbacks

* remove usage test

* updated as per feedbacks

---------

* Backend: make bulkAssets api async

* Backend: limit bulkAssets api async to Tag Assets page

* Update Tag page Assets API (#18622)

* create branch

* update add asset API

* add websocket on tags page

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>

* add failed case for socket operation

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sweta Agarwalla <105535990+sweta1308@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2024-11-18 17:57:07 +05:30
Mohit Yadav
66c253ef10
Add Queue Size in Configuration (#18672) 2024-11-18 12:45:08 +01:00
Shrushti Polekar
5cdde2054b
fix glossary column dropdown drag drop integration test (#18662) 2024-11-17 21:44:33 +05:30
Sriharsha Chintalapani
88c8fb48f3
Add Edit glossary terms, Edit Tier , Edit Tags as separate permissions (#18331)
* Add EditGlossaryTerms Permission

* Fix #18330: Add EDIT_GLOSSARY_TERM permission and enforce EDIT_TIER permisson

* add edit glossary term permission check in UI

* revert EDIT_GLOSSARY_TERMS operation

* Add EDIT_GLOSSARY_TERMS to common operations

* Add EDIT_TIER to common operations

* add default empty array for tags field, as patch calls can run into issues

* Fix tests

* Fix tests

* added glossary terms

* fix conflicts

* fix permission check for data model

* Add EditGlossaryTerms to DataConsumerPolicy

* Add EditGlossaryTerms,EditTier to DataConsumerPolicy

* fix tests

* Fix migrations for EditTier,EditGlossaryTerms

* add edit tier permission to data consumer

* Fix tests

* fix pytests

* missing test_dbt.py

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-11-15 10:50:15 -08:00
Sriharsha Chintalapani
783ec62e4c
Add Lineage Export Async endpoint using Search (#18553)
* Add Lineage Export Async endpoint using Search

* Lineage api (#18593)

* add api for lineage export

* update API call for lineage async

* use JsonUtils instead of objectMapper to read lineage search response

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sweta Agarwalla <105535990+sweta1308@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-11-15 09:11:23 -08:00
Sachin Chaurasiya
be642e7253
GEN-1694: metric entity improvement (#18659)
* GEN-1694: metric entity improvement

* add clike and python language mode in schemaeditor

* add getMetricExpressionLanguageName util to extract the language from metric expression
2024-11-15 20:30:42 +05:30
Aniket Katkar
9850810b04
Update the jsonpath-plus resolution version to fix the vulnerability (#18652) 2024-11-15 16:38:16 +05:30
Shrushti Polekar
393c7a4098
Fix: update glossary terms table stylings and temporarily bypass failing drag-and-drop test (#18653)
* update terms table styling and skip failing drag-drop test

* address pr comments related to styles
2024-11-15 16:05:15 +05:30
Aniket Katkar
5afd8c31b6
Updated the what's new content for the 1.5.11 release (#18540)
* Updated the what's new content for the 1.5.11 release

* Update the what's new content for 1.5.11

* Update the what's new release date for 1.5.11

* update what's new

* Update the release date

* Update the what's new data

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-11-15 09:06:24 +01:00
Sweta Agarwalla
6cdfff35b5
MINOR: fix owner link and update search index in navbar (#18609)
* fix minor issues

* update owner column component

* update list sequence

* add database search indexes

* add navbar search dropdown tests

* updated owner snapshot

* update wait for response API

* fix tests

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
2024-11-15 13:22:59 +05:30
Mayur Singal
8104c3aec6
MINOR: Fix Redshift DBT Playwright (#18642) 2024-11-14 18:15:23 +05:30
Aniket Katkar
bfc12feafc
GEN-1976: Alert improvements (#18448)
* Move the alert components to the components folder from pages folder

* auto generated localization changes and linting changes

* Fix the imports

* Add AlertConfigDetails component

* Add tabs in the alert details page

* auto generated localization changes

* Apply permissions for alert and add owner update functionality

* Change the alert update method from PUT to PATCH

* Add the recent events tab

* auto generated localization changes

* Improve the recent events tab for alert details

* Implement the logic to show alert events count on the alert details page

* auto generated localization and linting changes

* Add the pagination logic for the recent events list on the alert details page

* Fix the logic for previous click in NextPreviousWithOffset

* Localization change for other languages

* Hide pagination when not required

* Add pagination and permissions to API; create eventsRecord API; fix timestamp issue

* fix: remove publisherName from test destination message.

* Fix the recent events tab error

* fix: Resolve PATCH API issue with destination ID creation.

* fix: Delete associated alert metrics upon alert deletion

* feat: Detailed HTTP Response for Destination Test

* Add the test external destination functionality

* localization changes for other languages

* fix: Set status for testing destination in EmailPublisher

* Add advanced configurations for webhook destinations

* Localization changes for other languages

* Fix the destination status not showing
Add the POST and PUT radio button in advanced configs

* Localization changes for other languages

* Update the API to fetch the alert event diagnostic counts

* Improve the AlertConfigDetails component to show the alert configs properly

* fix: Display displayName instead of fullyQualifiedName in templates

* Fix the test destination result not showing the first time

* Fix the test destination status not showing for already existing destinations

* Fix the unit tests

* Fix playwright tests

* refactor: Separate sendMessage and sendTestMessage methods; add statusContext

* fix: Use name or fqn as fallback if displayName is null or empty

* fix: upsert successfulChangeEvent

* eliminate the GovernanceWorkflowChangeEvent option from external destination

* Add code comments

* Fix the unit test

---------

Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
Co-authored-by: Siddhanttimeline <siddhanttimeline9@gmail.com>
2024-11-14 17:39:45 +05:30
Kenil Shah
eadd93eb9b
Fix: support string value for xAxis apart from date (#18590)
* Fix: show string value for xAxis apart from date

* update key name
2024-11-14 15:38:33 +05:30
Sachin Chaurasiya
00fbf07387
Minor: show metric granularity in smallest to largest order unit (#18638) 2024-11-14 14:46:29 +05:30
codeasai
a3026811db
Add Thai languages (#18595)
* Add Thai languages

* add language support in language dropdown

---------

Co-authored-by: Ben <surask.suksathit@lotuss.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-11-14 11:22:34 +05:30
Jorge Gustavo Rocha
7aae60b760
Add support for another language: Português (Portugal) (#18587) 2024-11-14 10:34:49 +05:30
Mayur Singal
f345c4914f
GEN-1933: Separate Description Status for table and column (#18614) 2024-11-14 10:31:44 +05:30
Shrushti Polekar
626464f443
Feat: Support for column customization for Glossary term table (#18584) 2024-11-14 10:22:11 +05:30
Karan Hotchandani
887ffab7e1
MINOR - fix config for workflows (#18628)
* fix config for workflows

* minor spacing

* Fix certification spec

* Fix checkstyle

---------

Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2024-11-13 18:23:12 +01:00
Mohit Yadav
ee327ce0c4
Fix Unlimited Queue Size Issue (#18633) 2024-11-13 22:02:34 +05:30
Shailesh Parmar
7caec17162
fix: logs loading issue and reduce repetitive api call (#18630) 2024-11-13 20:01:12 +05:30
Sachin Chaurasiya
21becf6bff
fix: checkmark saves wrong custom property field (#18623) 2024-11-13 17:52:23 +05:30
Pranita Fulsundar
00357f68fc
Fix 16805: pagination dropdown issue (#18611)
* fix: pagination dropdown

* fix: remove handlePageChange dependency

* fix: api url for database schema response

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-11-13 17:47:19 +05:30
Karan Hotchandani
7cba2a7f51
fix lineage tests (#18618) 2024-11-13 13:29:51 +05:30
Mohit Yadav
30ee4fafb5
Update Default Values (#18621) 2024-11-13 13:24:16 +05:30
IceS2
e5441a9969
Initial implementation for Governance Workflows (#17837)
* Initial implementation for Governance Workflows

* Remove file that shouldn't be there

* Initial implementation for the governance workflows

* Refactor JSONSchema for Governance Workflows

* Refactored WorkflowInstanceStatus to keep the stage properly

* Fix Migration for MySQL for the WorkflowInstanceStates change

* Update WorkflowInstanceState to track all the states propertly

* Initial code tro migrate Request type of Tasks

* copy governance schemas into json directory

* update schema names

* Extract WorkflowInstance into its own resource

* Update WorkflowInstance to have the workflow starting variables

* Changed relatedEntity to be an EntityLink

* First iteration on checkEntityAttributes JSONLogic

* Refactored Workflow into TriggerWorkfow and MainWorkflow

* Fixed WorkflowInstance and WorkflowInstanceState

* Fixed WorkflowInstance and WorkflowInstanceState when process is terminated

* Fix UserTask

* json logic changes to workflow

* Fix Patch request. Add exclude filter to EventBasedEntityTrigger

* update query builder

* Fix Migrations

* Fix Migrations

* modify query builder widget

* fix search props

* update widget and use default fields

* Update edges to always have a true/false condition

* made changes to provider for fields

* push progress

* fix grouping of array field

* Remove duplicated migrations

* Fix migrations

* Fix migrations

* Fix Tests and commit some improvements suggestion

* fix unit tests

* Add Certification Workflow

* Fix Tests

* Fix Tests

* fix tests

* add delay in pagination tests

* fix minor config changes for json logic

* change the glossary page

* Remove Table Certification Workflow seed Data

* add json logic and elastic search parsers

* minor schedule interval changes for workflows

* minor fixes to the parsing logic

* certification ui

* fix tests

* fix tests

* add other search fields

* Add Certification to search index

* Update index mapping for certification

* add quick filter for certification

* update locales

* Fix index mapping

* Add missing index updates

* Fix checkstyle

* increase timeout

* skipping user test

* skipping team test

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2024-11-13 08:47:44 +01:00
Kenil Shah
ac3728fe67
update icon for suggestions description (#18617) 2024-11-13 13:05:15 +05:30
Mohit Yadav
9a5dc61ca7
Fix Search Index Contention (#18605)
* Fix Search Index Contention

* Update searchIndexingAppConfig.json

* Missing Error Logs and Stats

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-11-12 20:36:23 +05:30
IceS2
14e645368d
MINOR: Fix the Daily Active Users Summary Card (#18589)
* Changes Sum to Mean, in order to display Avg Active Users during the last n days

* Fix Typescript issue

* Update Summary Label
2024-11-12 12:15:22 +00:00
Sriharsha Chintalapani
1c05415321
Improve parallelization in SearchIndexApp (#18556)
* Improve parallelization in SearchIndexApp

* Improve parallelization in SearchIndexApp

* add countdown latch

* typo

* Status issues - #1

* Improve search indexing

* cleanup the code

* schemas

* Status Updates Via WebSockets

* Add Updates

* Make Logs Debug

* show duration in table

* add searchIndexApp params to openmetadata-ops.sh reindex commandline

* Fix code style

* Add Parallel Entity Level Reader

* Make code more readable

* Send App Run Record instead of jobData

* Sned App Run Record WebSocket for Data Insights

* ui: use socket connection to get the real-time updates of application data

* Send WebSocket Final Update

* ui: remove failedRecords, totalRecords and successRecordsfrom entities record

* Invert Condition

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-11-11 08:13:54 -08:00
Teddy
45d27a377d
GEN 1184 - Added Workflow Classification and Metric LevelConfig (#18572) 2024-11-11 15:59:42 +01:00
Shailesh Parmar
b59983d652
Minor: fixed sample data overlapping issue (#18581) 2024-11-11 17:48:30 +05:30
Shailesh Parmar
eeb27c3cbf
Minor: lineage data quality failure tracing support (#18580) 2024-11-11 15:21:29 +05:30
Sriharsha Chintalapani
7f5f47f618
Add export apis back and new apis should be called exportAsync (#18552)
* Add export apis back and new apis should be called exportAsync

* ui: use exportAsync endpoint for export assets

* fix build failure

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-11-08 10:35:02 -08:00
Suman Maharana
da039b197f
Add: Azure Data factory Connector (#18543)
* Added Azure Data factory Connector

* Added Lineage data factory

* removed not required files

* removed not required files

* Removed datafactory ui changes from oss

* resolve merge conflicts

* resolve merge conflicts

* added python requirements
2024-11-08 07:38:45 +01:00
Shrushti Polekar
f60e113026
name column sorting functionality for tables (#18547) 2024-11-07 22:55:51 +05:30
Pranita Fulsundar
76c017a8aa
GEN-2009: update the delete message for ingestion pipeline (#18546) 2024-11-07 21:40:27 +05:30
Shailesh Parmar
198dd35cc1
Minor: style improvement for DataStatisticWidget (#18544) 2024-11-07 19:09:03 +05:30
sonika-shah
e0d175f78e
GEN-1894 GEN-771 : Improvement - Add displayName field for custom property (#18496) 2024-11-07 16:59:59 +05:30
Chirag Madlani
6fa76ecde9
feat: persona based customisation for glossary and glossary term pages (#18454)
* Initial schemas for UI customization

* feat(ui): persona customization for entity details page
including Glossary & Glossary term detail page too

* WIP glossary term details page customization

* update persona epic

* WIP persona customization

* fix errors and warning

* WIP personalize glossary term

* WIP persona

* WIP glossary terms customization

* WIP glossary terms customization

* update glossary term customization

* remove glossary term reference from generic component

* use more of generic provider

* improved UI UX for customization

* complete glossary term customization
start with glossary customization

* apply backend linting

* support customization for glossary page

* fix issue with new tab on glossary page

* address feedbacks

* fix tests

* fix playwright for customize landing page

* fix tests

* fix tests

* address sonar errors

* fix glossary test

* fix glossary test

* fix failing playwright tests

* address comments

* fix conflict error

* add more widgets

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2024-11-07 11:13:12 +05:30
Sachin Chaurasiya
305f02c62a
Minor: improve the entity export modal feedback message (#18526) 2024-11-06 21:05:07 +05:30
Shailesh Parmar
b43c2ba4bd
Refactor: Improvement for data quality overview tab (#18523)
* Refactor: Improvement for data quality overview tab

* unite test for new component

* minor improvement

* remove unused icon
2024-11-06 20:59:54 +05:30
Aniket Katkar
5863c7e235
Add what's new content for 1.5.11 (#18534) 2024-11-06 18:53:14 +05:30
Karan Hotchandani
3dcbeb469f
App settings - lineage config (#18522)
* add lineage settings page

* update locales

* minor ui changes

* add playwright for lineage config
2024-11-06 10:26:18 +00:00
Mayur Singal
5660a751e3
GEN-2000: Add Support for PowerBI Report Server (#18513) 2024-11-06 14:55:05 +05:30
Ashish Gupta
76a257f3aa
MINOR: fix create admin page data and fields (#18458)
* fix create admin page data and fields

* fix issue around adminFlag undefined

* remove roles filed in case of admin listing game

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-11-05 20:11:27 +05:30