1040 Commits

Author SHA1 Message Date
Aniket Katkar
b8713690f0
Minor: Fix ingestion pipeline name search not working for observability alert (#15057)
* add ingestion pipeline search index in UI

* Add Ingestion Pipeline index

* Update date in case of running

* Use timestamp

* add cypress test case for ingestion pipeline trigger

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-02-06 19:30:32 +05:30
Aniket Katkar
8e860a56bf
Minor: Fix the wrong test result options on add observability alert page (#15043)
* fix test result options

* fix observability alert cypress test

* Fix Filtering

* fix naming order

* ignore $

* Fix Alerts

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-02-05 22:54:29 +05:30
Mohit Yadav
327319d2ff
Catch ALl Exception for App Resoruce Init (#15028) 2024-02-05 18:31:50 +05:30
mohitdeuex
9733ea2f01 Get Or Null App 2024-02-05 16:43:47 +05:30
Sriharsha Chintalapani
3055b8bf46
Add boosting function for tiers (#15019) 2024-02-04 22:12:50 -08:00
Mohit Yadav
1ecc461b62
App Bot is null during init (#15020) 2024-02-05 10:34:07 +05:30
Sriharsha Chintalapani
87efc57394
taskCount should show total count for openedBy or assignedTo a user (#15016) 2024-02-04 17:24:12 +05:30
Sriharsha Chintalapani
f817921750
Add boosting function and cleanup highlight builder code (#15017)
* add boost function based on votes

* Add boosting function and cleanup highlight builder code
2024-02-04 17:23:51 +05:30
Sriharsha Chintalapani
5839efcbf1
Fix migration for postgres db (#15009)
* Fix openmetadata-ops.sh migrate for Postgres

* Fix openmetadata-ops.sh migrate for Postgres
2024-02-02 09:54:51 -08:00
Mohit Yadav
d18a8ed82c
[Fix] Revert optionals error (#15008)
* Fix Optional Error in Object Mapper

* Revert Optional Changes

* return null instead of 0.0
2024-02-02 20:18:51 +05:30
Mohit Yadav
7dc0bc5829
[Data Observability] Add Pipeline Observability and status (#14958)
* Add Pipeline Observability and status

* Compare equalsIgnoreCase

* Fix Naming Args

* Fix Thread Type Filtering

* CheckStyle Fix

* Add Pipeline List Options

* References not being used, handle USERS and TEAMS category separately

* Fix Test Case Failures

* Update Code for PipelineRepository

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-02-02 16:21:04 +05:30
Sriharsha Chintalapani
968f146fe7
Add localhost:8585 as jwks urls; Fix entity consolidations when there is no previous chnage available (#15001) 2024-02-02 14:29:06 +05:30
Mohit Yadav
d551b86001
[Fix] Fixed Search Index Repository Tests Exceptions (#15000)
* Init Organization Errors Fix

* Fix Index Factory Tests

* CheckStyle Fix

* remove unnecessary code

* Use default in case of failures

* Use default in case of failures
2024-02-02 12:29:48 +05:30
Mohit Yadav
a840385693
[Fix]Init Organization Errors Fix (#14998)
* Init Organization Errors Fix

* Initialize the Role & Policy repositories

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2024-02-01 17:24:22 -08:00
07Himank
795bd5c867
Added Index Factory Config (#14950)
* Index Factory Config

* CI handling

* checkstyle
2024-02-01 19:50:51 +05:30
Mohit Yadav
8f94e54395
Wait for Indexing Job to complete (#14986) 2024-02-01 17:04:21 +05:30
Sriharsha Chintalapani
306adfcca0
Fix TestSuiteSearch case for OpenSearch (#14979)
* Fix TestSuiteSearch case for OpenSearch

* Fix checkstyle
2024-02-01 16:07:23 +05:30
07Himank
5a11d601b7
Remove unwanted fields (#14975)
* remove unwanted fields from current entity

* remove unwanted fields from current ent

* revert overriding the existing entity

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
2024-02-01 12:00:51 +05:30
Karan Hotchandani
5bd4de89fc
remove wildcards from search (#14938)
* remove wildcards from search

* remove wildcard from global search

* Fix aggregations and pagination

* fix explore issues

* increase maxlength

* move search options to base class

* Add Ngrams and highlighter for partial matches

* use search query instead of suggestions

* fix tags redirection issue

* add ngram to other indexes

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2024-02-01 09:21:48 +05:30
Sriharsha Chintalapani
2e95fcb98d
Fix #14786: Suggestions API (#14821)
* Fix #14786: Suggestions API

* Handle suggestions in ometa

* Minor: Optimise Databricks Client (#14776)

* MINOR - Fix SP topology context & Looker usage context (#14816)

* MINOR - Fix SP topology context & Looker usage context

* MINOR - Fix SP topology context & Looker usage context

* Fix tests

* Fixes #14598: Fix Tags / Labels ingestion on includeTags as False (#14782)

* fix(ui): password error message for char limits (#14808)

* fix(ui): password error message for char limits

* fix java side code

* Fixes #13556: Support for Salesforce table description ingestion (#14733)

* ISSUE-13556: Add suport for Salesforce table description ingestion

* ISSUE-13556: Remove unnecessary blank line

* ISSUE-13556: Fix to get description for each table

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>

* MINOR - Better handling of Ingestion Pipeline Status (#14792)

* MINOR - Better handling of Ingestion Pipeline Status

* format

* format

* MINOR: Added table validation for cost analysis data (#14793)

* Added validation for cost analysis source

* centralized life cycle logic

* CYPRESS: simplify side navigation click in cypress (#14818)

* simplify side navigation click in cypress

* make sidbar item uses common enum

* fix cypress failure of outside import

* fix(#14326): tier dropdown is not working in advance search (#14780)

* improvement in advance search based on custom property

* fix a reading undefined property issue

* wip: advance search based on tier

* some code cleanup and improvement

* some fixes

* fix: ui flicker when advanceSearched is apply and refresh the page

* some cleanup

* no need to call customproperty api call, if entity not suppport customProperties

* minor change

* fix: autocomplete not working in tier search option in advance search modal

* added unit test for advance search provider component

* some cleanup

* added testcase for open modal

* added testcase for resetAllFilters method

* removed unwanted code

* added e2e test for testing tier advance search

* fix: e2e search flow for single field

* fix: string field not working after giving listValues in TierSearch

* fix: group query e2e test fix

* used asyncFetch way to get the tierOptions synchronously

* some cleanup

* remove unwanted lines

* some cleanup

* fix: selected option show option value instead of option title

* fix(minor): update skip icon for executions (#14809)

* Fixes #14803: ignore capitalization when confirming deletes  (#14804)

* ignore case when confirming deletes

* Test confirmation of deletes works when case differs 

Added test case for 'delete' as the confirmation text.

* minor(config): update openmetadata-ui code reviewers (#14823)

* Add Tests

* Add list/accept/reject apis

* initial ui changes

* localisation

* show suggestion for empty description

* ui feedbacks

* Fix permission check for entities without owner

* Fix entityLink and add tests

* Add update suggestion WIP

* Fix test

* Fix PUT and Pagination

* Fix styling

* update test

* Update status

* add OM server connection in apps

* add permissions check

* Fix CI

* Remove TODO

* Fix feedResourceTest

* fix unit tests

* add private configs for apps

* add private configs for apps

* fix update application icons

* minor center align icon

* add private configs for apps

* Format

* Fix pydantic gen

* Remove token

* Update name

* Rework private conf

* Fix apps

* Fix apps

* Format

* Format

* show metapilot only if its installed

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: kwgdaig <18678754+kwgdaig@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Abhishek Porwal <80886271+Abhishek332@users.noreply.github.com>
Co-authored-by: Carlo Q <carlo@machina.bio>
Co-authored-by: karanh37 <karanh37@gmail.com>
2024-01-31 18:51:09 -08:00
07Himank
a715bdef3e
missing followers in data assets (#14968) 2024-01-31 07:09:30 -08:00
Ashish Gupta
c3f8a9da9d
MINOR: supported test email modal (#14947)
* suported test email modal

* localization keys and senderMail check

* added unit test for the test modal

* Fix Test Email request

* change the payload to test email

* test-email

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-01-31 13:24:11 +05:30
Chirag Madlani
727cd48ed5
fix(server): generate report for T-7days (#14932)
* fix(server): generate report for T-7days

* support weekly option for DataInsightReport application

* fix checkstyle

* add cypress to validate schedule

* skip ml flow schedule spec
2024-01-30 17:00:08 +05:30
Mohit Yadav
5940d7219a
[Fix] Add Regex for Entity FQn matches (#14946)
* Add Regex for Entity FQn matches

* Removing Reports app to be installed by default
2024-01-30 15:15:55 +05:30
Mohit Yadav
410f351528
Data Insight Report Link Fix (#14939) 2024-01-30 12:07:50 +05:30
Mohit Yadav
6d015980b7
[Fixes] Patch Minor Fixes (#14919)
* Patch Minor Fixes

* Patch Minor Fixes

* java checkstyke

* Fix Failing Test

* fix filtering
2024-01-30 00:41:45 +05:30
07Himank
38726971cf
Knowledge Center SearchFIx (#14935) 2024-01-29 09:01:19 -08:00
Pere Miquel Brull
1492f79da1
MINOR - Revert #14468 - DuckDB Connector (#14934) 2024-01-29 16:10:53 +01:00
07Himank
b90d0e6365
Sort by popularity (#14873)
* sort by popularity

* sort by popularity

* checkstyle

* Fix totalVotes & indexing

* rebase with main

* add popularity and name sort field

* update unit tests

* Fix feedResourceTest

* Remove Ngram from team search indexing

* Change boost for ngram to 1.0

* py test

* 0 totalvotes if votes is null

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2024-01-29 20:40:18 +05:30
07Himank
db7db07812
make isBot (#14914) 2024-01-29 16:08:03 +05:30
Aniket Katkar
8afde3ee82
Minor[WIP]: Fix observability cypress tests (#14907)
* modify error alert to get more details on API failure in cypress

* Fix Alert Type for postgres

* revert changes to show API errors in error toast

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-01-29 12:31:52 +05:30
Mohit Yadav
c9fab1e4e6
Sort Tier Data (#14916) 2024-01-29 12:23:22 +05:30
Sachin Chaurasiya
8d1780c5b4
Fix(#14262): Format the ES reindexing logs for better readability (#14849)
* Fix(#14262): Format the ES reindexing logs for better readability

* Refactor formatJsonString function in StringsUtils.ts

* Fix Error Schema for Search Indexing Application

* Refactor AppLogsViewer.component.tsx

* Fix import order and add default value for formatJsonString

* Refactor formatJsonString function in StringsUtils.ts

* Fix Schema

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-01-29 12:16:20 +05:30
IceS2
373cafcda2
Fixes #5448: Implement initial Iceberg Connector using PyIceberg (#14825)
* Create the iceberg connection schema

* Link the IcebergConnection configuration with the forms on the UI

* Add the pyiceberg dependency on the ingestion package

* Create the get_connection and test_connection functions

* First iteration on the iceberg ingestion logic

* Add A more comprehensive implementation of the Iceberg Source

* Add UnitTests

* Update icebergConnection definition

* Update the iceberg souce code based on new schema

* Updated icebergConnecgtion schema for simplicity and to be able to configure Converters

* Updated setup dependencies to be more flexible

* Updated get_owner_ref logic

* Fix formatting

* Changed the icebergConnection json schema structure to enable the ClassConverters

* Add the IcebergCatalog and IcebergFileSystem ClassConverters

* Refactor the code to take into account the new jsonSchema structure

* Fix formatting

* Add Documentation for the Iceberg Connector

* Fix Menu order for Iceberg

* ui: add Iceberg service icon and constant

* Fix DynamoDb Catalog issue due to how PyIceberg instantes it

* Changed uri title to URI

* Fix ClassConverter for Iceberg

* Fix GetSecretValue for password types

* Fix formatting

* Fix formatting

* Add Iceberg Connector Images for the docs

* Add pylint disable for Hacky super() call

* Add Iceberg.md for the UI docs

* Fix pylint complaint

* Fix pylint complaint

* Fix UnitTests

* fix type error and unit tests

* update pipeline type checks

* Fix Sonar Cloud complaints

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-01-29 06:32:58 +01:00
Pere Miquel Brull
90dc3e830c
MINOR - Bump testcontainers & mysql test version (#14906)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-01-28 16:45:34 -08:00
Sriharsha Chintalapani
5bdd057440
Fix feedResourceTest (#14904) 2024-01-28 15:45:57 -08:00
Teddy
bd09755fc0
ISSUE #14851: Error when deleting last test case result (#14868)
* fix: error if no testCaseResult exists when updating testSuite summary state

* fix: add return statement on resultsummaries update
2024-01-28 10:27:05 -08:00
Pere Miquel Brull
0332fb9fe5
#13934 - Update test connection no response error msg (#14897)
* #13934 - Update test connection no response error msg

* #13934 - Update test connection no response error msg
2024-01-28 10:25:15 -08:00
Sriharsha Chintalapani
f6ecc41d7c
Fix #11944: API support to fetch count for mentions (#14858)
* Fix #11944: API support to fetch count for mentions

* Add closedTask/totalTask

* fetch count as per new count api for entity and user page and remove mentions tab from entity

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-01-28 16:19:29 +05:30
Mohit Yadav
3f234df7cc
Fix Display Name for Alerts (#14889)
* Fix Display Names

* FIx Names and condition
2024-01-26 14:50:18 +05:30
Chirag Madlani
c936ac3a41
feat(ui): Data observability and notifications (#14806)
* Centralize EventType

* Format better for readability

* Add Handling for Thread as Change Event

* Centralize logic for Entity Message Creation

* Add Thread Parent Message in Conversation

* Add Task and Announcement

* Support for multiple destinations

* Delete existing alerts for schema changes

* Delete all tables data for event subscription

* Centralize logic for Notifications and Observability

* Change Field Name and Remove redundant code

* Test Fixes

* Add more fields

* Add Optional

* Logical Test Case Addition Chnage Event

* Add Filter By Owner

* Fix Email Entity Url

* added data observability settings page

* localization changes for other languages

* Handler closure and cleanup

* complete create observibility flow

* fix searchIndex method error

* Update Entity Observability Filter

* working observability creation

* complete creation flow

* fix action with arguments

* fix form validations and form data formulation for 'input.actions' field

* localization changes for other languages

* Fixes for Data Observability

* Add mention notification

* Only apply mention to conversation

* fIx test case

* bug fixes for add and edit observability alert form functionality

* fIx test case

* add feature to config internal along with external destinations in alert form

* Fix Formatting

* Fix Url Redirection

* add observability alert details page

* localization changes for other languages

* remove filter and action required rules

* fix styling for observability form and details page

* add delete functionality on observability alert details page

* update view page layout and add a warning in configure destination step

* localization changes for other languages

* feat(ui): support notification with new flow (#14848)

* feat(ui): support notification with new flow

* fix notifications

* add missing filters for notification alert creation flow

* update the filter hiding logic to show at the start always

* add check to show Actions section in observability alert creation form

* code cleanup and improvements

* add breadcrumb to add notification alert page

* Add General Metadata Filters

* remove code smells and code cleanup for add notification page

* add new filter

* show activity feed alert on notifications list page

* add notification alert details page changes

* localization changes for other languages

* show icons for all the entities in the alert trigger dropdown

* redirect to notification alert listing page after deleting the notification alert

* change add alert page title

* add logic to disable all destinations except email for users

* fixed unit tests

* code improvements and cleanups

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-01-26 13:56:18 +05:30
07Himank
5e653d25ff
fix a part of es exception while migrate (#14876) 2024-01-25 20:03:15 +05:30
Mayur Singal
dcd3d47c2b
Fix #14590: Improve airflow plugin error message (#14839)
* Fix #14590: Improve airflow plugin error message

* Update openmetadata-service/src/main/java/org/openmetadata/service/clients/pipeline/airflow/AirflowRESTClient.java

Co-authored-by: Ayush Shah <ayush@getcollate.io>

* Update msg

* Update RichTextEditorPreviewer in AirflowMessageBanner component

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-01-25 15:26:49 +01:00
07Himank
cb7db8ce8f
Minor fix Search service (#14837)
* Minor fix

* fix minor search fixes
2024-01-25 18:17:48 +05:30
Onkar Ravgan
70316b9680
MINOR: Fixed life cycle index mapping for raw_cost_analysis_report_index (#14832)
* fix life cycle index mapping

* fixes

* removed epoch
2024-01-24 09:23:09 -08:00
07Himank
115870b2af
Pipeline and stored procedure lineage fix (#14681)
* pipeline and stored procedure lineage

* pipeline and stored procedure lineage

* fix: pass entity type in url

* fix unit tests

* pipeline and stored procedure lineage

* pipeline lineage

* pipeline and stored procedure lineage

* fix performance issues

* pipeline and stored procedure lineage done

* optimisation

* fix fullscreen mode

* add pipeline animation

* update loading state

* fix styling

* update pipeline status

* fixes

* fix cypress

* fix cypress

* fix cypress

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-01-24 11:04:31 +05:30
Chirag Madlani
45efaefc46
fix(ui): password error message for char limits (#14808)
* fix(ui): password error message for char limits

* fix java side code
2024-01-23 12:38:35 +05:30
Sriharsha Chintalapani
a9e193608c
Fix #14318: Add support of JSON logs (#14783) 2024-01-21 22:59:28 -08:00
Sriharsha Chintalapani
15eb094cff
Improve error response to the client. Added errorType so that client can handle better than depending on the http status codes (#14784) 2024-01-21 08:07:07 -08:00
Mohit Yadav
0851b87442
Use 8.2.0 version of mysql (#14801) 2024-01-19 23:41:46 +05:30