11033 Commits

Author SHA1 Message Date
Sachin Chaurasiya
00fbf07387
Minor: show metric granularity in smallest to largest order unit (#18638) 2024-11-14 14:46:29 +05:30
RounakDhillon
cefbdf0fc3
Docs: Admin Guides Images Updation (#18631) 2024-11-14 14:30:31 +05:30
codeasai
a3026811db
Add Thai languages (#18595)
* Add Thai languages

* add language support in language dropdown

---------

Co-authored-by: Ben <surask.suksathit@lotuss.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-11-14 11:22:34 +05:30
Mayur Singal
14643f2b50
DI - Add metric name in group name (#18634) 2024-11-14 10:50:19 +05:30
Jorge Gustavo Rocha
7aae60b760
Add support for another language: Português (Portugal) (#18587) 2024-11-14 10:34:49 +05:30
Mayur Singal
f345c4914f
GEN-1933: Separate Description Status for table and column (#18614) 2024-11-14 10:31:44 +05:30
Shrushti Polekar
626464f443
Feat: Support for column customization for Glossary term table (#18584) 2024-11-14 10:22:11 +05:30
Chirag Madlani
fda0e542e4
fix: openmetadata-service/pom.xml to reduce vulnerabilities (#18620)
The following vulnerabilities are fixed with an upgrade:
- https://snyk.io/vuln/SNYK-JAVA-IONETTY-8367012

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-11-13 20:48:07 -08:00
Mohit Yadav
e7e4cbf77a
Fix Producer Sync Issue (#18635) 2024-11-14 02:20:49 +05:30
Karan Hotchandani
887ffab7e1
MINOR - fix config for workflows (#18628)
* fix config for workflows

* minor spacing

* Fix certification spec

* Fix checkstyle

---------

Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2024-11-13 18:23:12 +01:00
Mohit Yadav
ee327ce0c4
Fix Unlimited Queue Size Issue (#18633) 2024-11-13 22:02:34 +05:30
IceS2
128e4e8e93
MINOR: Fix App Insights not processing URL Encoded Data Assets (#18613)
* use 'URLDecoder.decode' before getting entity by name

* Update Test to cover this scenario
2024-11-13 16:14:56 +01:00
Shailesh Parmar
7caec17162
fix: logs loading issue and reduce repetitive api call (#18630) 2024-11-13 20:01:12 +05:30
Sachin Chaurasiya
21becf6bff
fix: checkmark saves wrong custom property field (#18623) 2024-11-13 17:52:23 +05:30
Pranita Fulsundar
00357f68fc
Fix 16805: pagination dropdown issue (#18611)
* fix: pagination dropdown

* fix: remove handlePageChange dependency

* fix: api url for database schema response

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-11-13 17:47:19 +05:30
Imri Paran
089fa785a8
build(setup-py): update pydantic version (#18541)
Update pydantic version to ">=2.7.0" in order to include `IncEx` that was introduced in 3d1355f168
2024-11-13 10:14:06 +01:00
RounakDhillon
f4634db1bc
Docs: Invite Users Image Updation in Admin Guide (#18608) 2024-11-13 13:47:57 +05:30
Karan Hotchandani
7cba2a7f51
fix lineage tests (#18618) 2024-11-13 13:29:51 +05:30
Mohit Yadav
30ee4fafb5
Update Default Values (#18621) 2024-11-13 13:24:16 +05:30
IceS2
e5441a9969
Initial implementation for Governance Workflows (#17837)
* Initial implementation for Governance Workflows

* Remove file that shouldn't be there

* Initial implementation for the governance workflows

* Refactor JSONSchema for Governance Workflows

* Refactored WorkflowInstanceStatus to keep the stage properly

* Fix Migration for MySQL for the WorkflowInstanceStates change

* Update WorkflowInstanceState to track all the states propertly

* Initial code tro migrate Request type of Tasks

* copy governance schemas into json directory

* update schema names

* Extract WorkflowInstance into its own resource

* Update WorkflowInstance to have the workflow starting variables

* Changed relatedEntity to be an EntityLink

* First iteration on checkEntityAttributes JSONLogic

* Refactored Workflow into TriggerWorkfow and MainWorkflow

* Fixed WorkflowInstance and WorkflowInstanceState

* Fixed WorkflowInstance and WorkflowInstanceState when process is terminated

* Fix UserTask

* json logic changes to workflow

* Fix Patch request. Add exclude filter to EventBasedEntityTrigger

* update query builder

* Fix Migrations

* Fix Migrations

* modify query builder widget

* fix search props

* update widget and use default fields

* Update edges to always have a true/false condition

* made changes to provider for fields

* push progress

* fix grouping of array field

* Remove duplicated migrations

* Fix migrations

* Fix migrations

* Fix Tests and commit some improvements suggestion

* fix unit tests

* Add Certification Workflow

* Fix Tests

* Fix Tests

* fix tests

* add delay in pagination tests

* fix minor config changes for json logic

* change the glossary page

* Remove Table Certification Workflow seed Data

* add json logic and elastic search parsers

* minor schedule interval changes for workflows

* minor fixes to the parsing logic

* certification ui

* fix tests

* fix tests

* add other search fields

* Add Certification to search index

* Update index mapping for certification

* add quick filter for certification

* update locales

* Fix index mapping

* Add missing index updates

* Fix checkstyle

* increase timeout

* skipping user test

* skipping team test

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2024-11-13 08:47:44 +01:00
Kenil Shah
ac3728fe67
update icon for suggestions description (#18617) 2024-11-13 13:05:15 +05:30
Mayur Singal
8e1c839473
MINOR: Summary Card changes require for UI (#18585) 2024-11-13 12:22:47 +05:30
Mohit Yadav
b6812f732f
Fix Version Comparison Logic (#18616) 2024-11-13 02:52:41 +05:30
Ayush Shah
6fa03ee66a
Fixes GEN-1994: Remove View Lineage from Metadata Ingestion flow (#18558) 2024-11-13 00:08:55 +05:30
Sriharsha Chintalapani
b5f4aee676
Search RBAC improvements (#18591)
* Search RBAC: process multiple view policies that can grant access to different set of entities

* Search RBAC: process multiple view policies that can grant access to different set of entities

* Search RBAC: process multiple view policies that can grant access to different set of entities
2024-11-12 09:11:02 -08:00
Mohit Yadav
9a5dc61ca7
Fix Search Index Contention (#18605)
* Fix Search Index Contention

* Update searchIndexingAppConfig.json

* Missing Error Logs and Stats

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-11-12 20:36:23 +05:30
Mayur Singal
f4fdafeb8a
MINOR: Athena & Tableau E2E fix (#18596) 2024-11-12 19:14:45 +05:30
Imri Paran
70c7880dfa
fixed bigquery system metrics e2e test (#18601) 2024-11-12 14:06:54 +01:00
Mayur Singal
76984c97a2
MINOR: mising menu for powerbi rs in 1.5 (#18607) 2024-11-12 17:58:38 +05:30
IceS2
14e645368d
MINOR: Fix the Daily Active Users Summary Card (#18589)
* Changes Sum to Mean, in order to display Avg Active Users during the last n days

* Fix Typescript issue

* Update Summary Label
2024-11-12 12:15:22 +00:00
RounakDhillon
75ccb1adc3
Docs: Anomaly Detection Updation (#18484) 2024-11-12 17:23:01 +05:30
Mayur Singal
ce43c975af
MINOR: mising menu for powerbi rs (#18603) 2024-11-12 15:57:48 +05:30
RounakDhillon
ea56a86c48
Docs: JWT Troubleshooting Doc Updation (#18602) 2024-11-12 15:47:23 +05:30
RounakDhillon
8526687506
Docs: One Line Updation in Security Section (#18597)
Co-authored-by: Rounak Dhillon <rounakdhillon@Rounaks-MacBook-Air.local>
2024-11-12 13:36:47 +05:30
Sriharsha Chintalapani
5589b9665e
Minor: add searchPageHierarchy method (#18578) 2024-11-11 09:04:50 -08:00
Sriharsha Chintalapani
1c05415321
Improve parallelization in SearchIndexApp (#18556)
* Improve parallelization in SearchIndexApp

* Improve parallelization in SearchIndexApp

* add countdown latch

* typo

* Status issues - #1

* Improve search indexing

* cleanup the code

* schemas

* Status Updates Via WebSockets

* Add Updates

* Make Logs Debug

* show duration in table

* add searchIndexApp params to openmetadata-ops.sh reindex commandline

* Fix code style

* Add Parallel Entity Level Reader

* Make code more readable

* Send App Run Record instead of jobData

* Sned App Run Record WebSocket for Data Insights

* ui: use socket connection to get the real-time updates of application data

* Send WebSocket Final Update

* ui: remove failedRecords, totalRecords and successRecordsfrom entities record

* Invert Condition

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-11-11 08:13:54 -08:00
Teddy
45d27a377d
GEN 1184 - Added Workflow Classification and Metric LevelConfig (#18572) 2024-11-11 15:59:42 +01:00
Shailesh Parmar
b59983d652
Minor: fixed sample data overlapping issue (#18581) 2024-11-11 17:48:30 +05:30
Shailesh Parmar
eeb27c3cbf
Minor: lineage data quality failure tracing support (#18580) 2024-11-11 15:21:29 +05:30
Imri Paran
a6d97b67a8
MINOR: fix system profile return types (#18470)
* fix(redshift-system): redshift return type

* fixed bigquery profiler

* fixed snowflake profiler

* job id action does not support matrix. using plain action summary.

* reverted gha change
2024-11-11 10:49:42 +01:00
Suman Maharana
fc79d60d83
Fixes: Added Sigma Column Level Lineage and Datamodels (#18571) 2024-11-11 14:42:57 +05:30
Imri Paran
cdaa5c10af
[GEN-1996] feat(data-quality): use sampling config in data diff (#18532)
* feat(data-quality): use sampling config in data diff

- get the table profiling config
- use hashing to sample deterministically the same ids from each table
- use dirty-equals to assert results of stochastic processes

* - reverted missing md5
- added missing database service type

* - use a custom substr sql function

* fixed nounce

* added failure for mssql with sampling because it requires a larger change in the data-diff library

* fixed unit tests

* updated range for sampling
2024-11-11 10:07:23 +01:00
Mayur Singal
943b4efb4d
GEN-1418: DI - Allow Support for Multiple Metrics (#18476) 2024-11-11 14:27:46 +05:30
Mayur Singal
efed932d97
Mask SQL Queries in Usage & Lineage Workflow (#18565) 2024-11-11 11:44:47 +05:30
Siddhant
2437d0124e
Improve Templates for Slack, GChat, and MSTeams (#17771)
* Add default ChangeEvent template and Slack destination test template

* fix conflict

* initial ingestionPipeline template

* clean templates.

* clean templates

* teams templates.

* add getBoldWithSpace()

* java check style

* google chat templates.

* Refactor GChat and Teams message for template data handling for DQ_Template_Section and General_Template_Section.

* Refactor slack message for template data handling for DQ_Template_Section and General_Template_Section.

* improvements gChatMessageDecorator

* improvements slackMessageDecorator

* improve MSTeamsMessageDecorator

* Call templates per entityType and refactor code

* Update EmailPublisher to use testEmail template instead of changeEvent template for sendTestMessage

* add parameterValues sections for teams

* Update SlackMessage to have attachments.
Update DQ template.

* Update dq templates, build dq template data in MessageDecorator

* remove IngestionPipeline template

* move SlackCallbackResource into a separate class. Fix tests.

* simulate timeout and 300.

* remove the validation of messages.

* fix teams MessageDecorator

* remove unused code from SlackMessageDecorator

* fix owners and tags data population in the template

* fix: changes in test case and test case result should load different templates.
2024-11-11 11:05:31 +05:30
sonika-shah
687b564ef6
GEN-1501 : Fix aggregation results on displayName.keyword for dataAssets (#18569) 2024-11-11 10:55:56 +05:30
Siddhant
2d6e21d7a9
fix postgres query. (#18568) 2024-11-10 09:22:23 +00:00
Pere Miquel Brull
92b9f44bf8
MINOR - Update Lineage Propagation Config (#18486)
* MINOR - Update Automator Lineage Propagation configs

* naming

* options

* Ignore metastore_db

* toggle parent propagation

* toggle parent propagation
2024-11-10 09:38:39 +01:00
Sriharsha Chintalapani
e7632428f4
Minor: Return pagination of for listHierarchy (#18567) 2024-11-09 23:28:52 -08:00
Sriharsha Chintalapani
a03d915909
Minor: JsonUtils.extractValue shouldn't throw exceptions (#18554) 2024-11-09 09:45:07 -08:00