2333 Commits

Author SHA1 Message Date
IceS2
dc46072df0
Handle empty aggregations on data insights (#17523) 2024-08-21 11:32:45 +02:00
IceS2
f80388bcb9
MINOR: Add unique count to data insights (#17505)
* Adding Unique Count via ElasticSearch Cardinality

* Add Unique Count to Data Insights functions
2024-08-21 09:30:38 +02:00
Siddhant
841f880370
fixed: avoid escaping html content in dataInsightReport template. Ensure that all newlines and special characters are preserved in the output. (#17514)
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-08-21 12:57:05 +05:30
Mohit Yadav
91f69ba34b
Add Page Type filter (#17519) 2024-08-21 11:02:50 +05:30
Mohit Yadav
6d22deb618
Init Search Repo in Entity (#17518) 2024-08-21 10:16:51 +05:30
Mohit Yadav
dc10b38b77
Keep Status ActiveError on Job Run (#17515) 2024-08-21 08:51:13 +05:30
Sriharsha Chintalapani
d59473f718
Fix #15529: Glossary CSV import with empty synonyms (#17497)
* Fix #15529: Glossary CSV import with empty synonyms

* Fix #15529: Glossary CSV import with empty synonyms

* Minor: fix test
2024-08-20 15:38:22 -07:00
Mohit Yadav
8e2aa16279
Add Missing Fields to mapping files (#17512)
* Add Missig Fields to mapping files

* Add null_value
2024-08-21 02:40:22 +05:30
Mohit Yadav
09afdfdb74
Add UnmappedType Sort Field (#17511) 2024-08-21 00:32:27 +05:30
Mohit Yadav
d906bcfcac
Adds Ability to Extend search functionalty (#17508)
* Adds Ability to Extend search functionalty

* Fix Tests
2024-08-20 23:14:30 +05:30
Ayush Shah
9880f06b2c
Fixes #17489: Allow non numeric numbers to be sent via Json, Replace NaN value… (#17490)
* fix: Allow non numeric numbers to be sent via Json, Replace NaN values with None in SQAProfilerInterface

Replace NaN values with None in the SQAProfilerInterface class to maintain database parity. NaN values will be cast to null in OpenMetadata. This change ensures that data handling processes account for this conversion.

* fix: histogram overflow error

* test: Add Unit Test for Null and Null Ratio Metric

* chore: Address comments

* chore: Address comments

* fix: checkstyle and message

* fix: failing tests as null count works as expected
2024-08-20 16:33:55 +05:30
Pere Miquel Brull
f7aab6b7a4
MINOR - Fix list API for automation workflows (#17494)
* MINOR - Fix list API for automation workflows

* MINOR - Fix list API for automation workflows
2024-08-20 12:40:21 +02:00
sonika-shah
d587e6f8e6
Fix : Propagation of owners and domain in search (#17400)
* Fix : Propagation of owners and domain in search

* update test

* optimize ADD_REMOVE_OWNERS_SCRIPT update query es script

* optimizes script by checking for inherited fields(inherited = true) before updating or adding them.

* fix glossary verion spec by adding slow since, test contains steps

* added slow in test as they are exceding in the runs

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-08-19 23:11:56 +05:30
Imri Paran
7508848376
fix(dq): data types for unique columns (#17431)
1. remove json and array from supported data types of unique column test.
2. migrations.
3. tests.
2024-08-19 14:28:42 +02:00
Mohit Yadav
2b915a53e9
Fix Ldap Username issue (#17485) 2024-08-19 16:02:54 +05:30
sonika-shah
82d3a87556
Updated the listBefore and listAfter methods to use both name and id as cursors for pagination. (#17477) 2024-08-18 10:15:57 +05:30
Sriharsha Chintalapani
29745b2abe
Minor: Fix ranking in search results (#17460)
* Minor: fix patch tags on apiendpoint

* Fix #17118: Elasticsearch HTTP Method Error in update_by_query; Fix: patch tags for request/response schema in ApiEndpoint

* Minor: Fix Search ranking based on exact match, tiering, usage, total votes
2024-08-16 16:12:51 -07:00
Sriharsha Chintalapani
39203a1156
Fix #17352: Remove TablesInitializer and bootstrap_storage.sh (#17353)
* Fix #17352: Remove TablesInitializer and bootstrap_storage.sh

* Fix #17352: Remove TablesInitializer and bootstrap_storage.sh

* Fix #17352: Remove TablesInitializer and bootstrap_storage.sh
2024-08-15 19:27:02 -07:00
IceS2
455e66bc37
Fix Missing Users errors on Data Insights Pipeline (#17446) 2024-08-14 14:26:51 +02:00
IceS2
c221f80d15
Fix DataInsights Field Getter (#17427) 2024-08-13 16:28:33 +00:00
Mayur Singal
ac2264e9c7
MINOR: DI Simplify Formulas (#17382)
* MINOR: DI Similify Formulas

* checkstyle
2024-08-13 17:42:46 +05:30
Mohit Yadav
7033870e71
Update SAML to have signed Messages (#17415)
* Update SAML to have signed Messages

* test fix
2024-08-13 11:38:37 +05:30
Siddhant
6ba4d69127
added /refresh implementation for LDAP with a refresh token expiry time of 24 hours. (#17410) 2024-08-13 11:36:43 +05:30
Mohit Yadav
f7f30799d4
Update Max Payload Size (#17414) 2024-08-12 23:21:49 +05:30
Chirag Madlani
995320a9ec
fix(ui): data insight page feedbacks (#17390)
* fix(ui): data insight page feedbacks

* Ignore null counts

* adjust advance filter

* Revert toastNotification changes

* Fix typo

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-08-12 22:46:43 +05:30
Siddhant
330b97a669
fix fetch loginConfiguration. (#17402) 2024-08-12 19:37:58 +05:30
IceS2
7b191b891d
MINOR: Implement Multi Data Streams. Fix Duplicating issue (#17365)
* Implement Multi Data Streams. Fix Duplicating issue

* Fix Test for multi index search

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-08-12 14:09:13 +02:00
Mohit Yadav
df9753c7bb
Add Opensearch max payload Size (#17392) 2024-08-12 11:34:52 +05:30
Mohit Yadav
aa5b9b2985
Increase Payload Size to 100MIB for ELastic Search (#17366)
* Increase Payload Size to 100MIB

* Increase batchSize Size to 500

* Add Payload Size as option in openmetadataoperations , and searchIndexJob

* //Comment

* Add Logs
2024-08-11 21:58:55 +05:30
Sriharsha Chintalapani
a6ebac9b68
Fix #17118: Elasticsearch HTTP Method Error in update_by_query; Fix: patch tags for request/response schema in ApiEndpoint (#17383)
* Minor: fix patch tags on apiendpoint

* Fix #17118: Elasticsearch HTTP Method Error in update_by_query; Fix: patch tags for request/response schema in ApiEndpoint
2024-08-10 17:56:52 -07:00
sonika-shah
8d5661a7bf
Support Elasticsearch cluster alias for E2E tests  (#17117)
* Fix backend tests and have index alias with clusterAlias appended

* Fix backend tests and have index alias with clusterAlias appended

* Fix failing tests

* alias setup

* fix suggestion not working due to alias

* fix getIndexOrAliasName method for multiple indexes

* update openmetadata.yaml

* update childAliases with clusterAlias

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-08-10 11:26:49 -07:00
IceS2
e5a7cff5a5
Updated oracle to use DBA_ tables (#17274)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2024-08-10 16:58:21 +05:30
Mayur Singal
52a7f99145
MINOR: Data insight feedbacks (#17337)
* MINOR: DI add no tier in system chart

* checkstyle

* minor ui fixes

* tier filter fix

* ignore tags and glossaries from tier table

* enhance advance search provider

* Ignore data products as well

* Empty Filter Condition & Lowecase Owner Displayname

* opensearch

* sort graph data to confirm order of graph

* address comments

* handle domain in advanced filter

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-08-10 14:20:14 +05:30
Sriharsha Chintalapani
98229591b5
Minor: Validate apiEndpoint response schema fields (#17370) 2024-08-09 21:49:26 -07:00
IceS2
8cfa6915d2
Fix permission for getPipelineStatus (#17356) 2024-08-09 16:13:00 +02:00
Sriharsha Chintalapani
8d67382513
Minor: Fix Postgres missing policy rule migrations for EditOwners (#17351) 2024-08-08 23:13:57 -07:00
Onkar Ravgan
945c6b2137
Fix #16039: Fixed sourceHash patch for DatabaseSchema entity (#17309) 2024-08-09 11:10:21 +05:30
IceS2
717c9709e0
Update Data Insights Pipeline to change CustomProperty key based on entityType (#17350) 2024-08-08 18:50:27 -07:00
Sriharsha Chintalapani
9acdab0d49
Minor: inherit owner/domain for apiEndpoints (#17300)
* Minor: inherit owner/domain for apiEndpoints

* test: add playwright test for owner propagation

* Minor: inherit owner/domain for apiEndpoints

* add script to propagate owners in search index of children entity

* exclude propagateOwners for testcase

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-08-08 10:42:10 -07:00
IceS2
9fcc8794a5
Remove sampleData and changeDescription from Data Insights documents (#17332) 2024-08-08 07:50:32 +02:00
Mohit Yadav
4c43509203
Check Expiry on Callback (#17326) 2024-08-07 15:28:03 +05:30
IceS2
66d7da05c4
MINOR: Iterate on data insights failure handling (#17324)
* Provide better errors on DataInsights Pipeline

* Improve DI Error Log

* Improve DI Error Log for OpenSearch

* Improve DI Error Log

* Fix Checkstyle
2024-08-07 11:14:55 +02:00
Mayur Singal
d3baf4fcf2
MINOR: DI Improve system charts (#17310) 2024-08-07 14:15:09 +05:30
sonika-shah
b6d3a2fbb3
Explore tree improvements (#17277)
* Explore tree improvements backend changes

* supported resizeable left panel in explore page

* change the more button feel to text

* fix playwright test for multi-owner

* minor changes

* fix playwright test

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-08-07 08:40:13 +02:00
sam-mccarty-mavenclinic
11a49ef08d
Fixes 16010: Support Looker ingestion from Gitlab (#17284)
* introduce gitlab option to lookml ingestion

* fix reader and disable test

* fix copy paste in test case

* fix file read and keyset pagination for tree

* fix credentials to include gitlab credentials

* uncomment arguments for unused credentials to fix validation error

* fix credentials test

* fix credentials test

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-08-07 06:53:44 +02:00
Mayur Singal
a579431e4a
MINOR: Fix DI most view data assets aggregation (#17288) 2024-08-07 00:34:54 +05:30
IceS2
0a8ea56c99
Provide better errors on DataInsights Pipeline (#17315) 2024-08-06 17:02:03 +02:00
Mohit Yadav
9bbc6a1801
Migrate NameHash (#17317) 2024-08-06 18:41:37 +05:30
Pere Miquel Brull
744ca6227c
MINOR - Clean Automator owner migration (#17294)
* MINOR - Clean Automator owner migration

* MINOR - Clean Automator owner migration
2024-08-06 09:02:09 +02:00
IceS2
6683fd1fbb
MINOR: Data insights provide better exceptions (#17292)
* Provide better errors

* Remove break; in order to allow the pipeline to process as much as they can
2024-08-06 08:08:26 +02:00