2141 Commits

Author SHA1 Message Date
sonika-shah
266920197f
Implement full text index on taskAssigneesIds column in both mysql and postgresql (#17520)
* Implement full text index on taskAssigneesIds column in both mysql and postgresql

* move migrations in 1.5
2024-08-21 12:18:50 -07:00
Mohit Yadav
f0d0dbf766
Add Get EntityReferenceById (#17538) 2024-08-21 22:06:52 +05:30
Aniket Katkar
5f04488a75
Fix #17375: Add secret key field for webhooks config in alert creation form (#17371)
* Add secret key field for the webhook destination in alerts

* Localization changes

* Add connection timeout field in the alert form destination section.
Add tests for the changes

* Localization changes

* masking the secret key.

* masking the secret key.

* moved the logic to prepare and entitySpecificUpdate functions.

* cleaned comments

* Test Fixes

---------

Co-authored-by: Siddhanttimeline <siddhanttimeline9@gmail.com>
Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-08-21 22:00:59 +05:30
sonika-shah
4c4e5edc38
Minor : Added lowercase_normalizer in serviceType for missing entities to have consistency in aggregation results (#17530) 2024-08-21 07:12:20 -07:00
Sriharsha Chintalapani
b8ad58a8dc
fix domain propagation apiCollection -> apiEndpoint, remove default cluster alias, catch exceptions while counting the docs for reindex (#17516)
* Minor: fix domain propagation apiCollection -> apiEndpoint, remove default cluster alias, catch exceptions while counting the docs for reindex

* Minor: fix mlModelService domain propagation

* Minor: optimize owner update
2024-08-21 06:33:46 -07:00
Imri Paran
5e6e54ef18
fix: testCase index (#17524)
do not index failed rows sample
2024-08-21 06:33:09 -07:00
Mohit Yadav
9309cf94c1
Add Api Service, Collection and Endpoint in reindex app (#17527)
* Add Api Service and Collection and Endpoint

* Keep SearchIndexing as false by Default
2024-08-21 16:26:50 +05:30
sonika-shah
d31215e72c
Fix: Remove unintended inclusion of child aliases after cluster after changes (#17472)
* Fix: Remove unintended inclusion of child aliases after cluster alias update

* fix the dataAssetWidget and entityDoc playwright failure

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-08-21 15:03:21 +05:30
IceS2
dc46072df0
Handle empty aggregations on data insights (#17523) 2024-08-21 11:32:45 +02:00
IceS2
f80388bcb9
MINOR: Add unique count to data insights (#17505)
* Adding Unique Count via ElasticSearch Cardinality

* Add Unique Count to Data Insights functions
2024-08-21 09:30:38 +02:00
Siddhant
841f880370
fixed: avoid escaping html content in dataInsightReport template. Ensure that all newlines and special characters are preserved in the output. (#17514)
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-08-21 12:57:05 +05:30
Mohit Yadav
91f69ba34b
Add Page Type filter (#17519) 2024-08-21 11:02:50 +05:30
Mohit Yadav
6d22deb618
Init Search Repo in Entity (#17518) 2024-08-21 10:16:51 +05:30
Mohit Yadav
dc10b38b77
Keep Status ActiveError on Job Run (#17515) 2024-08-21 08:51:13 +05:30
Sriharsha Chintalapani
d59473f718
Fix #15529: Glossary CSV import with empty synonyms (#17497)
* Fix #15529: Glossary CSV import with empty synonyms

* Fix #15529: Glossary CSV import with empty synonyms

* Minor: fix test
2024-08-20 15:38:22 -07:00
Mohit Yadav
8e2aa16279
Add Missing Fields to mapping files (#17512)
* Add Missig Fields to mapping files

* Add null_value
2024-08-21 02:40:22 +05:30
Mohit Yadav
09afdfdb74
Add UnmappedType Sort Field (#17511) 2024-08-21 00:32:27 +05:30
Mohit Yadav
d906bcfcac
Adds Ability to Extend search functionalty (#17508)
* Adds Ability to Extend search functionalty

* Fix Tests
2024-08-20 23:14:30 +05:30
Ayush Shah
9880f06b2c
Fixes #17489: Allow non numeric numbers to be sent via Json, Replace NaN value… (#17490)
* fix: Allow non numeric numbers to be sent via Json, Replace NaN values with None in SQAProfilerInterface

Replace NaN values with None in the SQAProfilerInterface class to maintain database parity. NaN values will be cast to null in OpenMetadata. This change ensures that data handling processes account for this conversion.

* fix: histogram overflow error

* test: Add Unit Test for Null and Null Ratio Metric

* chore: Address comments

* chore: Address comments

* fix: checkstyle and message

* fix: failing tests as null count works as expected
2024-08-20 16:33:55 +05:30
Pere Miquel Brull
f7aab6b7a4
MINOR - Fix list API for automation workflows (#17494)
* MINOR - Fix list API for automation workflows

* MINOR - Fix list API for automation workflows
2024-08-20 12:40:21 +02:00
sonika-shah
d587e6f8e6
Fix : Propagation of owners and domain in search (#17400)
* Fix : Propagation of owners and domain in search

* update test

* optimize ADD_REMOVE_OWNERS_SCRIPT update query es script

* optimizes script by checking for inherited fields(inherited = true) before updating or adding them.

* fix glossary verion spec by adding slow since, test contains steps

* added slow in test as they are exceding in the runs

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-08-19 23:11:56 +05:30
Imri Paran
7508848376
fix(dq): data types for unique columns (#17431)
1. remove json and array from supported data types of unique column test.
2. migrations.
3. tests.
2024-08-19 14:28:42 +02:00
Mohit Yadav
2b915a53e9
Fix Ldap Username issue (#17485) 2024-08-19 16:02:54 +05:30
sonika-shah
82d3a87556
Updated the listBefore and listAfter methods to use both name and id as cursors for pagination. (#17477) 2024-08-18 10:15:57 +05:30
Sriharsha Chintalapani
29745b2abe
Minor: Fix ranking in search results (#17460)
* Minor: fix patch tags on apiendpoint

* Fix #17118: Elasticsearch HTTP Method Error in update_by_query; Fix: patch tags for request/response schema in ApiEndpoint

* Minor: Fix Search ranking based on exact match, tiering, usage, total votes
2024-08-16 16:12:51 -07:00
Sriharsha Chintalapani
39203a1156
Fix #17352: Remove TablesInitializer and bootstrap_storage.sh (#17353)
* Fix #17352: Remove TablesInitializer and bootstrap_storage.sh

* Fix #17352: Remove TablesInitializer and bootstrap_storage.sh

* Fix #17352: Remove TablesInitializer and bootstrap_storage.sh
2024-08-15 19:27:02 -07:00
IceS2
455e66bc37
Fix Missing Users errors on Data Insights Pipeline (#17446) 2024-08-14 14:26:51 +02:00
IceS2
c221f80d15
Fix DataInsights Field Getter (#17427) 2024-08-13 16:28:33 +00:00
Mayur Singal
ac2264e9c7
MINOR: DI Simplify Formulas (#17382)
* MINOR: DI Similify Formulas

* checkstyle
2024-08-13 17:42:46 +05:30
Mohit Yadav
7033870e71
Update SAML to have signed Messages (#17415)
* Update SAML to have signed Messages

* test fix
2024-08-13 11:38:37 +05:30
Siddhant
6ba4d69127
added /refresh implementation for LDAP with a refresh token expiry time of 24 hours. (#17410) 2024-08-13 11:36:43 +05:30
Mohit Yadav
f7f30799d4
Update Max Payload Size (#17414) 2024-08-12 23:21:49 +05:30
Chirag Madlani
995320a9ec
fix(ui): data insight page feedbacks (#17390)
* fix(ui): data insight page feedbacks

* Ignore null counts

* adjust advance filter

* Revert toastNotification changes

* Fix typo

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-08-12 22:46:43 +05:30
Siddhant
330b97a669
fix fetch loginConfiguration. (#17402) 2024-08-12 19:37:58 +05:30
IceS2
7b191b891d
MINOR: Implement Multi Data Streams. Fix Duplicating issue (#17365)
* Implement Multi Data Streams. Fix Duplicating issue

* Fix Test for multi index search

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-08-12 14:09:13 +02:00
Mohit Yadav
df9753c7bb
Add Opensearch max payload Size (#17392) 2024-08-12 11:34:52 +05:30
Mohit Yadav
aa5b9b2985
Increase Payload Size to 100MIB for ELastic Search (#17366)
* Increase Payload Size to 100MIB

* Increase batchSize Size to 500

* Add Payload Size as option in openmetadataoperations , and searchIndexJob

* //Comment

* Add Logs
2024-08-11 21:58:55 +05:30
Sriharsha Chintalapani
a6ebac9b68
Fix #17118: Elasticsearch HTTP Method Error in update_by_query; Fix: patch tags for request/response schema in ApiEndpoint (#17383)
* Minor: fix patch tags on apiendpoint

* Fix #17118: Elasticsearch HTTP Method Error in update_by_query; Fix: patch tags for request/response schema in ApiEndpoint
2024-08-10 17:56:52 -07:00
sonika-shah
8d5661a7bf
Support Elasticsearch cluster alias for E2E tests  (#17117)
* Fix backend tests and have index alias with clusterAlias appended

* Fix backend tests and have index alias with clusterAlias appended

* Fix failing tests

* alias setup

* fix suggestion not working due to alias

* fix getIndexOrAliasName method for multiple indexes

* update openmetadata.yaml

* update childAliases with clusterAlias

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-08-10 11:26:49 -07:00
IceS2
e5a7cff5a5
Updated oracle to use DBA_ tables (#17274)
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2024-08-10 16:58:21 +05:30
Mayur Singal
52a7f99145
MINOR: Data insight feedbacks (#17337)
* MINOR: DI add no tier in system chart

* checkstyle

* minor ui fixes

* tier filter fix

* ignore tags and glossaries from tier table

* enhance advance search provider

* Ignore data products as well

* Empty Filter Condition & Lowecase Owner Displayname

* opensearch

* sort graph data to confirm order of graph

* address comments

* handle domain in advanced filter

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-08-10 14:20:14 +05:30
Sriharsha Chintalapani
98229591b5
Minor: Validate apiEndpoint response schema fields (#17370) 2024-08-09 21:49:26 -07:00
IceS2
8cfa6915d2
Fix permission for getPipelineStatus (#17356) 2024-08-09 16:13:00 +02:00
Sriharsha Chintalapani
8d67382513
Minor: Fix Postgres missing policy rule migrations for EditOwners (#17351) 2024-08-08 23:13:57 -07:00
Onkar Ravgan
945c6b2137
Fix #16039: Fixed sourceHash patch for DatabaseSchema entity (#17309) 2024-08-09 11:10:21 +05:30
IceS2
717c9709e0
Update Data Insights Pipeline to change CustomProperty key based on entityType (#17350) 2024-08-08 18:50:27 -07:00
Sriharsha Chintalapani
9acdab0d49
Minor: inherit owner/domain for apiEndpoints (#17300)
* Minor: inherit owner/domain for apiEndpoints

* test: add playwright test for owner propagation

* Minor: inherit owner/domain for apiEndpoints

* add script to propagate owners in search index of children entity

* exclude propagateOwners for testcase

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-08-08 10:42:10 -07:00
IceS2
9fcc8794a5
Remove sampleData and changeDescription from Data Insights documents (#17332) 2024-08-08 07:50:32 +02:00
Mohit Yadav
4c43509203
Check Expiry on Callback (#17326) 2024-08-07 15:28:03 +05:30
IceS2
66d7da05c4
MINOR: Iterate on data insights failure handling (#17324)
* Provide better errors on DataInsights Pipeline

* Improve DI Error Log

* Improve DI Error Log for OpenSearch

* Improve DI Error Log

* Fix Checkstyle
2024-08-07 11:14:55 +02:00