1531 Commits

Author SHA1 Message Date
Ayush Shah
2c84442e39
fix: Empty Connection Overwrite Logic (#18122) 2024-10-07 10:10:57 +05:30
Mohit Yadav
7ae81272b3
Fix performance issue (#18128)
* Fix ClassGraph

* Use CONST
2024-10-05 10:38:22 -07:00
Mayur Singal
52ea9ea81e
Fix #18005: Mask Greenplum Password (#18047) 2024-10-04 16:57:15 +05:30
Mohit Yadav
a352a7ca6c
[GEN-1704] Remove Table Details from Import/Export at table level (#18093)
* [GEN-1704] Remove Table Details from Import/Export at table level, and only add columns

* Fix Tests
2024-10-04 10:07:45 +05:30
Mayur Singal
eada21326a
GEN-1215: DI Populate Test Case Result & Resolution Status (#17969) 2024-10-03 16:30:53 +05:30
Pere Miquel Brull
e81efc8d98
MINOR - Add App Limits (#18072)
* App Limits - Prep extension

* App Limits - Prep extension

* App Limits - Allow to search by name

* App Limits - Allow to search by name

* fix postgres sql

* comments
2024-10-03 11:00:29 +02:00
Teddy
751bceb763
MINOR - Reduce lineage response size (#18080)
* fix import issue

* fix: reduce response size and exclude fields in search client source

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-10-03 08:13:13 +02:00
Teddy
4cfce98090
GEN 1409 - Logic to fetch DQ Lineage (#18069)
* fix import issue

* feat: added dq lineage tracing logic

* fix: move dq lineage to its own endpoint

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-10-02 15:15:26 +02:00
Siddhant
ed052596e8
[IMPROVEMENT] Add Support for Custom Headers and HTTP Methods in Generic Webhook (#18056)
* Add support for custom headers and JSON payload configuration in webhook

* move testDestination API to EventSubscriptionResource

* support POST and PUT http ops for generic webhook requests

* fix name

* remove json field from webhook

* fix getEmailTestMessage

* refactor exception handling
2024-10-02 09:20:12 +05:30
sonika-shah
f6b6874f57
Minor : fix term references validation msg on glossary import (#18058) 2024-10-01 14:21:33 +05:30
Mohit Yadav
66cd00c568
Fix Ops (#18061) 2024-10-01 12:31:08 +05:30
sonika-shah
683193fb41
Fix #18035 - getting type null exception on feed (#18052) 2024-10-01 10:49:48 +05:30
sonika-shah
1d727d57bd
Feature: Allow Custom Property Update in Glossary Bulk Import/export (#17919)
* fix import issue

* Feat : Allow Custom Property Update in Bulk Upload

* Feat : Allow Custom Property Update in Bulk Upload

* supported editable imports in glossary page

* added remaning localizaion keys

* update logic of fieldToExtensionStrings to use csvparser

* update json and partialStatus condition

* fix tests for partialSuccess status change

* supported customProperty editable field

* fix error in custom property edit modal on new line empty custom property

* added entity type from root to support other bulk import entity as well

* fix the quote removing due to the regex in the string type

* Add backend tests , and error msg improvements

* GlossaryStatus header change

* fix unit test and dry run in case of synonyms having quotes in it

* Remove extension column in CSVs for all entities except glossaryTerm

* added editor for reviewers

* unit test around csv utils

* added escape for string too, in case of semicolon comes

* added playwright test without extension and supported relatedTerm as editable

* added unit test around csv util logic

* resolve conflicts

* Backend - add support for enumWithDescriptions in bulk import

* add tests and other error handling improvements related to enumWithDescriptions

* fix the custom property modal header and render the layout as per right panel in entities

* parese enumWithDescription for the customProperty modal while editable

* fix description data in enumWithDescription one

* fix: Handle NullPointerException when adding custom properties to ensure loop continues for other schemas of the same type for addToRegistry

* added extension playwrigth test and fix enumWithDescription object failure

* descrease the size of extension  modal

* remove additional comments

* fix the escape in parent key

* improve custom property layout

* improve ui for inline properties

* fix description, glossary and relatedTerm escape char issue

* fix some customProperty ui changes

* fix sonar issue

* minor layout changes

* minor label improvements for entity ref and list

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-01 00:12:43 +05:30
Mohit Yadav
f1ae49e1a0
[Search] Indexing Fixes (#18048)
* Fix OpenSearch Content Size Issue
entity content is too long [204857600] for the configured buffer limit [104857600]

* Change Type to Long

* Add Payload Size to take dynamic entry

* Migrations for 1.5.6

* Mark Stale Entries Stopped

* Format checkstyle

* Fix failure

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-09-30 23:39:27 +05:30
Teddy
9552886f29
GEN 1211 - Added TTResp and TTReso metrics (#18033)
* fix import issue

* feat: compute test case resolution metrics

* feat: added  index key to resolution index

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-09-30 14:21:41 +02:00
Mayur Singal
fd75db3e9b
MINOR: DI Missing Dashboard Description Status (#18018) 2024-09-30 11:06:22 +05:30
sonika-shah
35bef5669a
Fix #18035 - getting type null exception on user feed (#18036) 2024-09-29 10:24:46 +05:30
sonika-shah
baa2781619
Fix #18039 - Fix incorrect @Schema implementations in Swagger annotations (#18040) 2024-09-29 10:23:30 +05:30
sonika-shah
1b029d2bf2
Add new ‘enumWithDescriptions’ Custom Property to allow adding Enum Keys with Description (#17777)
* Add new ‘metaEnum’ Custom Property to allow adding Enum Keys with Description

* replace JsonNodeFactory method with JsonUtils

* rename property from metaEnum to enumWithDescriptions, and other method optimizations

* ui: add support for creating enumWithDescription property

* minor locale changes

* ui: add edit support for created enumWithDescription property

* Refactor enum description field layout in AddCustomProperty and EditCustomPropertyModal

* add support for adding values to enumWithDescription custom property type

* Refactor custom property input IDs in AddCustomProperty and EditCustomPropertyModal components

* Refactor custom property table rendering logic and UI components

* Refactor custom property table rendering logic and UI components

* Refactor custom property table rendering logic and UI components

* add basic card layout

* Refactor CustomPropertyTable component to improve UI and functionality

* update playwright test part 1

* Refactor PropertyValue component to conditionally render right panel styles

* fix: entity reference property update

* Refactor CustomPropertyTable component to conditionally render right panel styles

* fix: flaky test

* Refactor CustomPropertyTable test to use updated test IDs and remove unnecessary code

* fix flaky test

* improve the playwright test

* add more test

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-09-29 00:45:46 +05:30
Teddy
e3f1d528d8
MINOR - Improve Data Quality Results List from Search (#18014)
* fix import issue

* feat: added dimension and type field in DQ Result search listing

* fix: remove code duplication

* style: ran java linting

* chore: clean up unecessary elements

* style: ran java linting

* fix: remove unecessary conditional statement + remove comments

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-09-26 22:47:03 +02:00
Pere Miquel Brull
d26449576a
GEN-1234 - Clean up suggestions when a user is deleted (#17988)
* GEN-1234 - Clean up suggestions when a user is deleted

* add method

* add method

* fix postgres query
2024-09-26 16:22:36 +02:00
Suman Maharana
37b6dc8290
Add Sigma Dashboard Connector (#17855)
* Add Sigma Dashboard Connector

* changed to id instead of name in dashboard entity

* Address Comments

* addressed comments

* Added Docs

* yaml file changes

* fix ui changes
2024-09-26 16:29:35 +05:30
sonika-shah
a99fbb2eb6
GEN-1434: Improvement : Show displayName for database and databaseSchema in explore tree (#17876)
* Show displayName for database and databaseSchema in explore tree

* use display name for database and schema

* add playwright

* resync indexes of dataAssets based on database & databaseSchema displayName change

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2024-09-26 16:18:16 +05:30
Mohit Yadav
afd7887bd8
[Issue-16487] Add Stop for Search Indexing (#17914)
* Add Stop for Search Indexing

* Update Logs and Message

* test for stop func

* remove test
2024-09-25 18:29:32 +05:30
Shailesh Parmar
8dd6a84d1f
GEN-1160 Minor: migrate to test case search api from test case api (#17902)
* Minor: migrate to test case search api from test case api

* fix: testSuites search client update

* style: ran java linting

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2024-09-25 10:10:30 +05:30
Imri Paran
350e0f70cb
MNIOR: feat(apps): support config file (#17872)
* feat(apps): support config file

- added support for app config files
- removed AppPrivateConfig from the OpenMetadata server configuration

* use dorpwizard utility classes for resolving environment variables in the config

* moved fields to class level

* format
2024-09-24 14:21:50 +00:00
Mohit Yadav
d77cf36b38
Adds Saml Redirect Uri (#17936)
* Add Saml Redirect Uri In Session

* Refactor

* add redirect url in saml

* add Saml logout servlet

* fix for message

* added logout method for saml

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
2024-09-23 22:20:48 +05:30
Siddhant
bc3d4c778e
[FIX] ldap User Name (#17764)
* migrate userName to emailPrefix for ldap auth provider.

* migrate userName to emailPrefix for ldap auth provider.

* fix migration.

* move migration to v155

* Update Ldap Authenticator

* Fix Ldap Issue on Login

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-09-23 11:35:20 +05:30
sonika-shah
c1e20873b1
Fix #17929 : Fix team export issue (#17930) 2024-09-22 21:30:35 +05:30
Teddy
e008a0d6f7
MINOR - Add list testCase for Logical Test Suite ID (#17941)
* fix import issue

* fix: listing test cases linked to logical test suite

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-09-21 11:04:42 +02:00
Mohit Yadav
512946df15
FIx Failing Test Case (#17945)
* FIx Failing Test Case

* Update dependency version
2024-09-20 22:21:18 +05:30
Mohit Yadav
f85246583e
Fix Automator Search (#17942)
* Fix Automator Search

* Add only to index
2024-09-20 19:37:16 +05:30
Mohit Yadav
f4ce0e8693
fix owners patch issue (#17900)
* fix owners patch issue

* rename func

* typo

* Add Test Case

* Fix Tests

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-09-20 11:41:13 +05:30
Mohit Yadav
043b18ee0f
Add Redirect Uri Handling for Confidential Flow (#17913)
* Add Redirect Uri Handling for Confidential Flow

* add redirect uri in login

* fix session redirect uri

* update redirect path to absolute path

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2024-09-20 10:31:54 +05:30
Teddy
4331570f7f
GEN 1507 - Implement Origin Entity FQN parms to incident manager listing (#17890)
* fix import issue

* feat: added originEntityFQN param in incident listing

* chore: removed table alias

* fix: added alias to derived table

* style: ran java linting

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-09-18 22:45:36 +02:00
Pere Miquel Brull
0f5746499a
GEN-1492 - Decrypt JWT internally for system health check (#17864) 2024-09-18 19:24:10 +02:00
Teddy
33c50efdbf
GEN-1192 - Move Test Case to its Own Resource (#17862)
* feat: indexed test case results

* feat: added indexation logic for test case results

* style: ran java linting

* fix: IDE warnigns

* chore: added test case results migration

* style: ran java linting

* fix: postgres migration column json ref

* empty commit to trigger queued

* chore: extracted test case results to its own resource

* chore: fix failing tests

* chore: move testCaseResult state from testSuite and testCase to dynamic field fetched from test case results search index

* chore: clean up test case repository

* style: ran java linting

* chore: removed testCaseResultSummary and testCaseResult state from db

* fix: test failures

* chore: fix index mapping type for result value

* chore: fix test failure
2024-09-18 11:58:59 +02:00
Sriharsha Chintalapani
4d0339b270
Improve Search Relevancy for tier, usage and use stemming to generate ngrams (#17871)
* Improve Search Relevancy for tier, usage and use stemming to generate ngrams

* Improve Search Relevancy for tier, usage and use stemming to generate ngrams

* add displayName using name

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-09-17 19:41:35 -07:00
Pere Miquel Brull
6a1cd0ef8b
GEN-1493 - Fix paginate_es in opensearch (#17858)
* GEN-1493 - Fix opensearch pagination

* GEN-1494 - Add CI for py-tests with Postgres and Opensearch

* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
2024-09-17 14:21:10 +02:00
Sriharsha Chintalapani
6c307e69a6
Fix Search RBAC to consider only ViewAll, ViewBasic (#17865)
* Fix Search RBAC to consider only ViewAll, ViewBasic

* Fix Search RBAC to consider only ViewAll, ViewBasic

* Fix Search RBAC to consider only ViewAll, ViewBasic
2024-09-16 21:31:38 -07:00
Sriharsha Chintalapani
2da9c34730
Issue-13782: RBAC for search (#17757)
* Issue-13782: RBAC for search

* Handle complex conditions

* Add optimizations to condition parsing

* Add SPeL based condition evaluation

* Fix styling

* use SpeL Parser AST to compose ES conditions

* Fix test message

* improvements in query generation

* improvements in query generation

* Add more complex queries

* Add more complex codndition tests and evaluation

* Add OpenSearch query builders and tests

* Add handling resources in a rule, add tests for multiple rules and inherited roles
2024-09-16 11:45:21 +05:30
Sriharsha Chintalapani
c34545adeb
Issue-15768: Support Metric Entity (#17680)
* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Fix tests

* Fix tests

* Fix tests

* Minor: Fix tests

* ui: add metricsAPI rest utils

* ui: metric list page part 1

* feat: Add metric translations for multiple languages

* chore: Add "metric" field to SearchIndexingApplication schema

* ui: add create metric page

* ui: metric details page patch 1

* ui: add custom property and lineage support for metric entity

* ui: add expression component

* ui: add metric summary component

* chore: Update tab labels in MetricDetails and MetricVersion components

* ui: show other info like metric type, granularity, etc

* feat: Add support for metric entity in search dropdown

* feat: Rename custom property to Metric in MetricEntity.md

* feat: Add OwnerLabel component to MetricListPage

* Fix expression field in Metric

* chore: update expression to metricExpression

* ui: add metric header component with edit option

* Add metric to SearchIndexApp

* chore: Update expression to metricExpression

* ui: allow metric expression edit

* ui: update metric icon

* minor improvements

* Fix lineage indexing for Metric

* Update GlobalSettingsClassBase.ts to use MetricIcon for metrics in the global settings menu

* Fix error handling in MetricListPage component

* add related metrics

* minor improvements

* Fix relatedTerms patch

* Fix relatedTerms validation

* Add Boolean for deleted

* filter active entity from related metric list

* playwrite e2e part 1

* Refactor MetricSummary component to include RelatedMetrics in the summary panel

* test: add playwright test for metric special cases

* Add 'Metrics' to Explore Tree

* test: add e2e for add metric page

* test: add test for metric listing page content

* Add Boolean for deleted, remove deleted from suggests

* Refactor LineageProvider to handle deleted flag properly

* add playwright for metric listing

* fix test

* Add colored metric icon and update its usage in GlobalSettingsClassBase

* Fixed py_test test_ometa_endpoint for metric

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: SumanMaharana <sumanmaharana786@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-09-16 11:36:31 +05:30
Sriharsha Chintalapani
ea62f34385
fix: openmetadata-service/pom.xml to reduce vulnerabilities (#17841)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-ORGJETBRAINSKOTLIN-2393744

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-09-15 16:31:34 -07:00
IceS2
4ae318f11b
Refactor code to improve di pipeline performance (#17820) 2024-09-13 11:18:22 +02:00
IceS2
9d2784e8e8
Hotfix to the Term Aggregation size on Data Insights (#17822) 2024-09-12 16:50:04 +02:00
Teddy
e066929ff7
GEN-1380 - Implement support for date_histogram aggregation (#17738)
* feat: indexed test case results

* feat: added indexation logic for test case results

* style: ran java linting

* fix: IDE warnigns

* chore: added test case results migration

* style: ran java linting

* fix: postgres migration column json ref

* empty commit to trigger queued

* feat: add support for date_histogram aggregation

* style: ran java linting

* chore: clean up conflicts

* fix: move from iterator loop to forEach

* fix: removed comment

* style: ran java linting
2024-09-12 13:11:30 +02:00
Pere Miquel Brull
f4d0eba08f
MINOR - Add ES pagination with error handling (#17776)
* MINOR - Add ES pagination with error handling

* format

* format

* add nullable

* prepare API

* fix pagination

* format
2024-09-12 07:14:56 +02:00
Ayush Shah
1c6695b8d7
Fixes #17804: Classification API returns Table class for restore (#17806)
* fix: Classification API returns Table class for restore
2024-09-12 10:11:41 +05:30
Siddhant
052430ee07
fix announcement notification. (#17793) 2024-09-11 22:08:54 +05:30
Imri Paran
0e43aeb1b6
Fix 17304: make ingestion pipeline APIs public (#17700)
* chore(api): make ingestion pipeline APIs public

* format

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-09-11 19:58:20 +05:30