743 Commits

Author SHA1 Message Date
Mayur Singal
a4d62f6d85
MINOR: Add location path to table entity (#18307) 2024-10-21 10:31:27 +05:30
Sriharsha Chintalapani
9bffbfc3d5
Fix #18007: Disabled Classifications or Tags shouldn't be visible in UI (#18242)
* Fix #18007: Disabled Classifications or Tags shouldn't be visible in UI

* added playwright test for disabled tags should not be visible while search

* replace testing tag to new generated one to avoid flakyness

* added test for checking tags are re-enabling it from disabled state

* fix the playwright test for the wrong column selector

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-10-19 14:18:38 -07:00
Sriharsha Chintalapani
85cc240e1e
Minor: Add an API to list all the properties and custom properties for a given entity type (#18248)
* Minor: Add an API to list all the properties and custom properties for a given entity type

* Minor: Add an API to list all the properties and custom properties for a given entity type

* add custom properties

* add custom properties

* add custom properties
2024-10-19 14:14:34 -07:00
Katarzyna Kałek
141caf4b80
fixed dashboardDataModel enum in spec (#18304)
Co-authored-by: Katarzyna Kałek <kkalek@olx.pl>
2024-10-18 08:28:31 +02:00
Teddy
fc51d7feb4
FIX #17042 - Allow non admin/bot to read profiler global config (#18292)
* fix: custom properties folder name in generation to match expected package name (i.e. customProperties -- uppercase P)

* fix: allow non admin/bot to read profiler global config with the right permission

* style: ran java linting

* fix: custom properties import casing
2024-10-17 09:27:34 +02:00
sonika-shah
2093c24f83
Minor : remove EnumWithDescriptions in basic.json (#18305) 2024-10-17 12:08:39 +05:30
Pere Miquel Brull
7012e73d75
GEN-1166 - Improve Ingestion Workflow Error Summary (#18280)
* GEN-1166 - Improve Ingestion Workflow Error Summary

* fix test

* docs

* comments
2024-10-16 18:15:50 +02:00
Imri Paran
befa1f72e8
MINOR: add supportsDataDiff flag (#18205)
* feat: add supportsDataDiff flag

enables filtering services based on support of the data diff test case

* fixed mysql migration

* fixed conflicts
2024-10-16 11:07:31 +02:00
sonika-shah
62ee79e71e
GEN-1836: Table custom property enhancements (#18239)
* Table custom property enhancements

* ui: remove rowCount

* minor style file changes

* minor change to show no data placeholder

* add icons for property types

* fix type filtering issue

* fix spacing issue for custom properties card

* refactor comments and style file

* fix test

* add playwright test for adding/updating table type property in entity

* add sesrchField for search property by name

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-15 12:41:10 +05:30
sonika-shah
7d736f6195
Feature : Add table-type custom property (#18135)
* Feature : Add table-type custom property

* ui: remove enumWithDescriptions property

* Refactor AddCustomProperty component to remove unused EnumWithDescriptionsConfig import and property

* ui: add support for creating and updating the table-type property

* add rowCount field in table-type and clean logic of validation

* add rowCount field in table-type and clean logic of validation

* add rowCount field in table-type and clean logic of validation

* ui: add support for adding value to table type property part 1

* use rowCount from the API

* default row-count to 10

* fix tooltip to read rowcount from the props

* fix empty row issue

* rename from table-type to table-cp

* Refactor table type property to handle empty rows and display in modal

* Refactor table type property to handle empty rows and display in modal

* use locales for messages

* minor change

* ui: use column layout for rendering custom properties cards

* remove enumwithDescriptions from backend and add tests for table type

* fix unit test

* fix playwright

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-10 13:43:19 -07:00
Sachin Chaurasiya
457f3d919a
GEN-1322: API Entity - Remove Beta (#17967)
* GEN-1322: API Entity - Remove Beta

* minor: add doc for the metadata pipeline

* api service refactor

* api service refactor backend changes

* add apiconnection in test service connection

* pytest fix

* fix java file formatting

* Fix casing of REST in ApiServiceRest.spec.ts

* Refactor REST to Rest in API classes

* minor change

* minor change

* minor change

* fix cashing for API to Api

* add playwright test for api service ingestion

* fix: playwright test

---------

Co-authored-by: harshsoni2024 <harshsoni2024@gmail.com>
2024-10-08 14:39:55 +05:30
Ayush Shah
221327afd4
Refactor ServiceConnectionDetails component to handle specific Airflow connection type (#18100) 2024-10-08 11:29:45 +05:30
Pere Miquel Brull
e81efc8d98
MINOR - Add App Limits (#18072)
* App Limits - Prep extension

* App Limits - Prep extension

* App Limits - Allow to search by name

* App Limits - Allow to search by name

* fix postgres sql

* comments
2024-10-03 11:00:29 +02:00
Siddhant
ed052596e8
[IMPROVEMENT] Add Support for Custom Headers and HTTP Methods in Generic Webhook (#18056)
* Add support for custom headers and JSON payload configuration in webhook

* move testDestination API to EventSubscriptionResource

* support POST and PUT http ops for generic webhook requests

* fix name

* remove json field from webhook

* fix getEmailTestMessage

* refactor exception handling
2024-10-02 09:20:12 +05:30
Mohit Yadav
f1ae49e1a0
[Search] Indexing Fixes (#18048)
* Fix OpenSearch Content Size Issue
entity content is too long [204857600] for the configured buffer limit [104857600]

* Change Type to Long

* Add Payload Size to take dynamic entry

* Migrations for 1.5.6

* Mark Stale Entries Stopped

* Format checkstyle

* Fix failure

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-09-30 23:39:27 +05:30
Teddy
9552886f29
GEN 1211 - Added TTResp and TTReso metrics (#18033)
* fix import issue

* feat: compute test case resolution metrics

* feat: added  index key to resolution index

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-09-30 14:21:41 +02:00
sonika-shah
1b029d2bf2
Add new ‘enumWithDescriptions’ Custom Property to allow adding Enum Keys with Description (#17777)
* Add new ‘metaEnum’ Custom Property to allow adding Enum Keys with Description

* replace JsonNodeFactory method with JsonUtils

* rename property from metaEnum to enumWithDescriptions, and other method optimizations

* ui: add support for creating enumWithDescription property

* minor locale changes

* ui: add edit support for created enumWithDescription property

* Refactor enum description field layout in AddCustomProperty and EditCustomPropertyModal

* add support for adding values to enumWithDescription custom property type

* Refactor custom property input IDs in AddCustomProperty and EditCustomPropertyModal components

* Refactor custom property table rendering logic and UI components

* Refactor custom property table rendering logic and UI components

* Refactor custom property table rendering logic and UI components

* add basic card layout

* Refactor CustomPropertyTable component to improve UI and functionality

* update playwright test part 1

* Refactor PropertyValue component to conditionally render right panel styles

* fix: entity reference property update

* Refactor CustomPropertyTable component to conditionally render right panel styles

* fix: flaky test

* Refactor CustomPropertyTable test to use updated test IDs and remove unnecessary code

* fix flaky test

* improve the playwright test

* add more test

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-09-29 00:45:46 +05:30
Pere Miquel Brull
d26449576a
GEN-1234 - Clean up suggestions when a user is deleted (#17988)
* GEN-1234 - Clean up suggestions when a user is deleted

* add method

* add method

* fix postgres query
2024-09-26 16:22:36 +02:00
Suman Maharana
37b6dc8290
Add Sigma Dashboard Connector (#17855)
* Add Sigma Dashboard Connector

* changed to id instead of name in dashboard entity

* Address Comments

* addressed comments

* Added Docs

* yaml file changes

* fix ui changes
2024-09-26 16:29:35 +05:30
Mohit Yadav
afd7887bd8
[Issue-16487] Add Stop for Search Indexing (#17914)
* Add Stop for Search Indexing

* Update Logs and Message

* test for stop func

* remove test
2024-09-25 18:29:32 +05:30
Sachin Chaurasiya
62ced79671
minor: add supportsMetadataExtraction property in restConnection.json (#17974) 2024-09-24 20:05:57 +05:30
Ayush Shah
8087f48e15
Fixes #17869: Add validations in place for salesforce connection (#17870) 2024-09-20 16:33:38 +05:30
Teddy
33c50efdbf
GEN-1192 - Move Test Case to its Own Resource (#17862)
* feat: indexed test case results

* feat: added indexation logic for test case results

* style: ran java linting

* fix: IDE warnigns

* chore: added test case results migration

* style: ran java linting

* fix: postgres migration column json ref

* empty commit to trigger queued

* chore: extracted test case results to its own resource

* chore: fix failing tests

* chore: move testCaseResult state from testSuite and testCase to dynamic field fetched from test case results search index

* chore: clean up test case repository

* style: ran java linting

* chore: removed testCaseResultSummary and testCaseResult state from db

* fix: test failures

* chore: fix index mapping type for result value

* chore: fix test failure
2024-09-18 11:58:59 +02:00
Sriharsha Chintalapani
c34545adeb
Issue-15768: Support Metric Entity (#17680)
* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Fix tests

* Fix tests

* Fix tests

* Minor: Fix tests

* ui: add metricsAPI rest utils

* ui: metric list page part 1

* feat: Add metric translations for multiple languages

* chore: Add "metric" field to SearchIndexingApplication schema

* ui: add create metric page

* ui: metric details page patch 1

* ui: add custom property and lineage support for metric entity

* ui: add expression component

* ui: add metric summary component

* chore: Update tab labels in MetricDetails and MetricVersion components

* ui: show other info like metric type, granularity, etc

* feat: Add support for metric entity in search dropdown

* feat: Rename custom property to Metric in MetricEntity.md

* feat: Add OwnerLabel component to MetricListPage

* Fix expression field in Metric

* chore: update expression to metricExpression

* ui: add metric header component with edit option

* Add metric to SearchIndexApp

* chore: Update expression to metricExpression

* ui: allow metric expression edit

* ui: update metric icon

* minor improvements

* Fix lineage indexing for Metric

* Update GlobalSettingsClassBase.ts to use MetricIcon for metrics in the global settings menu

* Fix error handling in MetricListPage component

* add related metrics

* minor improvements

* Fix relatedTerms patch

* Fix relatedTerms validation

* Add Boolean for deleted

* filter active entity from related metric list

* playwrite e2e part 1

* Refactor MetricSummary component to include RelatedMetrics in the summary panel

* test: add playwright test for metric special cases

* Add 'Metrics' to Explore Tree

* test: add e2e for add metric page

* test: add test for metric listing page content

* Add Boolean for deleted, remove deleted from suggests

* Refactor LineageProvider to handle deleted flag properly

* add playwright for metric listing

* fix test

* Add colored metric icon and update its usage in GlobalSettingsClassBase

* Fixed py_test test_ometa_endpoint for metric

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: SumanMaharana <sumanmaharana786@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-09-16 11:36:31 +05:30
k.nakagaki
3d8e30142c
Fixes 8428: make it possible to choice a sampling method type when we create profile ingestion for the Snowflake (#17831)
* Add test for existing code

* Add sampling method at ingestion.

* add samplingMethodType into UI

* modify init method to use new parameter.

* create descriptions

* execute isort

* fix an unintended change.

* apply py_format

* close section

* specify  init arguments

* fix bug

* apply py_format

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2024-09-15 21:51:17 +02:00
harshsoni2024
4f57f4e710
GEN-309: OpenAPI Connector (#17754)
* configure api service metadata

* add rest api service

* fix test con. pyformat changes

* add models, fix test con.

* improve test con.

* add docs, side doc

* fix model data parse, url error fix

* add tests

* fix pytest errors

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-09-11 13:36:53 +05:30
Ayush Shah
8decd2338f
MINOR: Add Matillion test connection and schema for pipeline service (#17710) 2024-09-10 17:41:28 +05:30
Teddy
1f72a3c9c6
GEN-1213 - logic to index test case result (#17706)
* feat: indexed test case results

* feat: added indexation logic for test case results

* style: ran java linting

* fix: IDE warnigns

* chore: added test case results migration

* style: ran java linting

* fix: postgres migration column json ref

* empty commit to trigger queued

* fix: added index on id field
2024-09-09 07:58:45 +02:00
harshsoni2024
a89c4e9f40
Connector: VertexAI UI, docs (#17649) 2024-09-04 11:45:33 +05:30
sonika-shah
6d32257765
Fixes #16498 : Support custom properties for Data Products (#17641) 2024-08-31 09:43:37 +05:30
Pere Miquel Brull
bcb29b46da
MINOR - Implement SAP Hana Lineage (#17615)
* MINOR - SAP Hana Lineage

* skeleton

* parser

* lineage

* manage formulas

* add cvs

* add cvs

* better typing

* enum

* handle cvs

* saphana docs
2024-08-30 07:42:43 +02:00
harshsoni2024
5393e5ede2
Connector: vertexai schema (#17632)
* vertexai connection schema

* remove yaml file
2024-08-29 21:19:07 -07:00
Mohit Yadav
e4073c7824
Bump Openmetadata Snapshot version (#17621) 2024-08-29 01:51:20 +05:30
Imri Paran
b48c6a0485
feat(postgres): add money profile (#17558)
add support for profiling of money type
2024-08-22 14:53:34 -07:00
IceS2
9e7ddfbd69
MINOR: Fix unparsable date and remove legacy code (#17549)
* Fix unparsable date and remove legacy code

* fix ui for data insight cleanup

* fix unit tests

* fix tour playwright

* fix percentage value and label for charts

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
2024-08-22 14:52:47 -07:00
gpby
61e835093f
add support for Teradata LDAP logon mechanism (#17454) 2024-08-22 08:34:52 +02:00
IceS2
f80388bcb9
MINOR: Add unique count to data insights (#17505)
* Adding Unique Count via ElasticSearch Cardinality

* Add Unique Count to Data Insights functions
2024-08-21 09:30:38 +02:00
Chirag Madlani
8ad3db243b
fix(ui): misc issues (#17447)
* fix(ui): redirect unauthorised users to error page

* remove left panel for import action in glossary import

* fix data insight app stuck in aut

* fix path issue

* fix setup issue

* add tree field to build advance filter on ui side

* update unauthorised page level to root instead authenticated

* minor fixes

* update error message
2024-08-20 13:01:00 +05:30
IceS2
14a473e5d3
Make Transient table ingestion On by default (#17484) 2024-08-19 14:18:45 +02:00
Mayur Singal
8acf6d3e94
MINOR: Make Include ddl disabled by default (#17450)
* MINOR: Make Include ddl disabled by default

* make schema def failure warning debug

* Add missing condition

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2024-08-19 14:07:18 +02:00
Shailesh Parmar
75f62a7872
#17437 Data Quality UI can't display type as column name (#17442) 2024-08-14 18:01:41 +00:00
Pere Miquel Brull
4a0c251e5c
MINOR - Config to PATCH metapilot descriptions if empty (#17424)
* MINOR - Config to PATCH metapilot descriptions if empty

* MINOR - Config to PATCH metapilot descriptions if empty
2024-08-14 15:41:53 +02:00
Mohit Yadav
7033870e71
Update SAML to have signed Messages (#17415)
* Update SAML to have signed Messages

* test fix
2024-08-13 11:38:37 +05:30
Mohit Yadav
05baba9dc1
Change Default Value of Batch Size and Payload Size (#17391) 2024-08-12 11:18:18 +05:30
Mohit Yadav
aa5b9b2985
Increase Payload Size to 100MIB for ELastic Search (#17366)
* Increase Payload Size to 100MIB

* Increase batchSize Size to 500

* Add Payload Size as option in openmetadataoperations , and searchIndexJob

* //Comment

* Add Logs
2024-08-11 21:58:55 +05:30
Mohit Yadav
6e80d3f405
Fix Migrations (#17364) 2024-08-09 20:58:44 +05:30
IceS2
717c9709e0
Update Data Insights Pipeline to change CustomProperty key based on entityType (#17350) 2024-08-08 18:50:27 -07:00
Mayur Singal
d3baf4fcf2
MINOR: DI Improve system charts (#17310) 2024-08-07 14:15:09 +05:30
sam-mccarty-mavenclinic
11a49ef08d
Fixes 16010: Support Looker ingestion from Gitlab (#17284)
* introduce gitlab option to lookml ingestion

* fix reader and disable test

* fix copy paste in test case

* fix file read and keyset pagination for tree

* fix credentials to include gitlab credentials

* uncomment arguments for unused credentials to fix validation error

* fix credentials test

* fix credentials test

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-08-07 06:53:44 +02:00
Siddhant
40009dbeae
Slack App Integration. (#17286)
* slack bolt app integration.

* checkstyle fix.

* fix slack uninstall.

* removed callback and events from JwtFilter.
2024-08-06 10:23:05 +05:30