2200 Commits

Author SHA1 Message Date
Onkar Ravgan
d08c928801
Added project property to dashboards (#11986)
* Added projects to dashboards

* Added powerbi proj

* merge conflicts after source url

* fixed mongo pytest
2023-06-15 21:23:43 +05:30
Mayur Singal
82a0222257
SourceUrl changes for dashboard, pipeline & chart entities (#11991) 2023-06-15 14:44:48 +05:30
Pere Miquel Brull
75aa3d5fa1
Fix #10542 & #10830 - Mask Sample Data PII & Column Metrics (#11919)
* Mask Sample Data PII

* Mask column profile

* Mask Test Cases

* Mask Test Cases

* Add tests

* Format

* Format

* Fix test case resource masking
2023-06-15 09:13:57 +02:00
Mayur Singal
7fa963eec3
Fix #1076: Add mongodb support (#11943) 2023-06-15 11:14:22 +05:30
Onkar Ravgan
d409c339e4
Fix 10689: Added Powerbi Datamodels (#11923) 2023-06-14 17:00:08 +05:30
Suresh Srinivas
7d9b27b996
Fixes #11976 Add support for suggest and request description and tags for all the entities (#11977) 2023-06-13 12:45:07 -07:00
Pere Miquel Brull
ba5f929f77
Add fine-grained health validation for Airflow APIs (#11792)
* Add fine-grained health validation for Airflow APIs

* Add ingestion version response

* Improve messaging

* Format

* Format

* Update response

* ui:update the hook to return airflow status based on response status field

* ui: add unit test for useAirflowStatus hook

* chore: only show test connection if airflow is available

* feat: add airflow message banner

* chore: update icon and background color

* chore: update typography to text

* test: add unit test

* address comments

* chore: show banner on service detail page

* fix: update test suite api workflow to reflect new implementation

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2023-06-12 07:47:45 +02:00
07Himank
62af9bb633
fixed issue for lineage description (#11500)
* fixed issue for lineage description

* fixed issue while ingesting

* fixed issue while ingesting

* added test case for Lingeage with description

* addressing comments .. enhancement

* addressing comments .. enhancement

* modified py test case and removed description from addLineage as we are not using it.

* add support for topic entity and description in lineage details

* fix pylint & test

* pytest fix

* fix column lineage null issue

---------

Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-06-12 11:17:32 +05:30
Pere Miquel Brull
20ebe70d9e
Update test conn definition on initialize (#11940) 2023-06-09 14:06:55 +02:00
Pere Miquel Brull
00f07190e9
Change ES mapping maximumMessageSize & retentionSize to long (#11907) 2023-06-07 12:50:24 +02:00
Sriharsha Chintalapani
4ecbbcd43c
Fix Japanese language indexing (#11903) 2023-06-07 07:22:55 +02:00
Suresh Srinivas
f23f67dfe9
Fixes #11617 Entity get operations check for ViewAll operation instead of specific view operations based on the field requested (#11904) 2023-06-06 15:18:57 -07:00
Mohit Yadav
57e44c22be
Make Resource Packages Dynamic (#11891) 2023-06-07 01:02:52 +05:30
mohitdeuex
d43ca31891 Add UUID on Metadata Service Creation in case of CREATE 2023-06-07 00:16:00 +05:30
Sriharsha Chintalapani
809fbbf536
Fix #11348: Return a proper error message when a token with wrong keyID sent (#11509)
* Fix #11348: Return a proper error message when a token with wrong keyID is sent

* Fix #11348: Return a proper error message when a token with wrong keyID is sent

* fix formatting

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2023-06-06 11:37:39 -07:00
Pere Miquel Brull
b68d063694
Update Test Connection Definition FQN (#11861)
* Update Test Connection Definition FQN

* chore: use object literal for string concatenation

* Fix test

* Update Test Connection Definition FQN

* Update Test Connection Definition FQN

* Update Test Connection Definition FQN

* Fix test

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-06 15:22:32 +02:00
Teddy
4b9f213dbf
Fixes Issue #11863 - Add Status to DQ (#11893)
* feat: added entityReference field in testSuite to link testSuite to an entity when the testSuite is executable.

* feat: added `executableEntityReference` as an entity reference for executable test suite to their entity

* feat: add status object to test case results

* feat: ran python linting

* feat: fixed  update to
2023-06-06 10:09:16 +00:00
Teddy
721869428e
Revert "Fixe Issue #11863 - Add Status logic for test case results (#11881)" (#11892)
This reverts commit 06735fe8dbaac5b267c9a2cf744ca154f88a9247.
2023-06-06 09:56:12 +02:00
Teddy
06735fe8db
Fixe Issue #11863 - Add Status logic for test case results (#11881)
* feat: added entityReference field in testSuite to link testSuite to an entity when the testSuite is executable.

* feat: added `executableEntityReference` as an entity reference for executable test suite to their entity

* feat: add status object to test case results

* feat: ran python linting
2023-06-06 09:45:49 +02:00
Ayush Shah
65f370e4aa
Rename GCS to GCP (#11812) 2023-06-06 11:57:00 +05:30
Suresh Srinivas
ee52b631f9
Move entity registration from EntityRepository to EntityResource (#11886) 2023-06-05 20:51:08 -07:00
Mohit Yadav
26db825b71
Added Support for Slack Related Configuration , Added Slack Bot , Sla… (#11791)
* Added Support for Slack Related Configuration , Added Slack Bot , Slack Installer and Slack Application Configuration

* comment

* Checkstyle

* Update Encrypt and Decrypt to encrypt all settings for slack

* Fix typo

* Update version for client and bolt

* Update deps

* Remove dependency for Slack bolt servlet
2023-06-06 00:55:14 +05:30
Mohit Yadav
146a7b285c
Fix Update Issue in MetadataService in case the ES config is updated (#11885) 2023-06-06 00:22:33 +05:30
Suresh Srinivas
d978580703
Fixes #11872 - Add authorization for Usage APIs (#11873) 2023-06-02 15:50:29 -07:00
Suresh Srinivas
b4d56bb559
Remove unnecessary constructors in ResultList implementations (#11871) 2023-06-02 13:03:24 -07:00
Teddy
d0cffdcd66
Fixes Issue #11438 - Implement threshold and startegy for custom SQL (#11847)
* feat: Add threshold and strategy logic on the custom SQL object test

* feat: ran python linting

* feat: added safety checks for custom sql query

* feat: ran python linting
2023-06-02 09:41:31 +02:00
Teddy
c98a15ca19
Fixes #11705 - Update ingestion and backend to match new DQ flow (#11836)
* feat: refactor ingestion flow logic

* feat: ran python linting

* feat: update tests to match new workflow

* feat: ran python linting

* feat: update sample data test suite name

* feat: Added backend logic to support logical and executable test suites

* feat: clean up java and json code

* feat: added sample data for logical and executable test suites

* feat: remove executable from CreateTestSuite

* feat: ran python and java linting

* feat: added README info for data quality structure

* skipping cypress to keep main green

* fixed typescript type issue

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2023-06-01 23:19:13 -07:00
Pere Miquel Brull
fdeea71671
Fix Looker explore git link & Add BitBucket reader (#11837)
* Add looker test connection step

* Add looker test connection step

* Update Credentials

* Fix explore link and add bitbucket reader

* Format

* Fix test

* Fix spline linting

* Fix import
2023-06-02 07:19:32 +02:00
Sriharsha Chintalapani
dae4b64326
Fix #11855: Tags can get deleted if the service name matches partially with targetFQN (#11856)
* Fix #11855: Tags can get deleted if the service name matches partially with targetFQN

* Fix #11855: Tags can get deleted if the service name matches partially with targetFQN
2023-06-01 20:48:29 -07:00
Mohit Yadav
e55628fef3
Improvements for Change Event Parsing Logic (#11616)
* Improvements for Change Event Parsing Logic

* Remove plural from columns , update message and handle case for array field types

* Removed Redundant Code for Feed Threads, moved to util class

* Fix Entity Conversion Failures

* Fix Message Decorator type

* Revert to previous logic for extracting change event

* update File
2023-06-01 21:54:41 +05:30
Pere Miquel Brull
11c07ee8ab
Fix #11516 - SAP Hana Connector (#11777)
* SAP Hana skeleton

* Add SAP Hana Connector

* Fix ingestion and docs

* Prep SAP Hana Profiler

* Linting

* Update index.md

* Revert: Update index.md

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-05-31 16:00:31 +02:00
Mohit Yadav
3255c5980a
Remove Unnecessary fields from GET and DELETE (#11745) 2023-05-28 23:55:13 +05:30
Mohit Yadav
79d6d0bcd3
Add extension to OpenMetadata (#11786)
* Add extension to OpenMetadata

* Typo Comment

* remove log
2023-05-28 23:44:02 +05:30
Snyk bot
be57a6d86a
fix: upgrade software.amazon.awssdk:cloudwatch from 2.20.53 to 2.20.56 (#11698)
Snyk has created this PR to upgrade software.amazon.awssdk:cloudwatch from 2.20.53 to 2.20.56.

See this package in Maven Repository:
https://mvnrepository.com/artifact/software.amazon.awssdk/cloudwatch/

See this project in Snyk:
https://app.snyk.io/org/harshach/project/97e6fbe6-21ff-4c69-b9c5-388bb60a9097?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-05-28 10:11:13 -07:00
Louie
956ec58930
Add ES RestHighLevelClient TCP keep alive timeout (#11731)
* Add RestHighLevelClient TCP keep alive timeout

* Merge RestHighLevelClient configs
2023-05-28 10:10:46 -07:00
Suresh Srinivas
f0f64a7b21
Change combined EncryptDecrypt methods to separate Encrypt and Decrpyt methods (#11755) 2023-05-25 12:35:25 -07:00
07Himank
ef2869cd9a
Delete Tasks related to entity when the entity is deleted (#11660)
* Delete Tasks related to entity when the entity is deleted

* Delete all conversation related to entity only when its permanantly deleted

* renamed function name deleteTasksForRelatedEntity to deleteAllConversationsRelatedToEntity

---------

Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
2023-05-25 10:48:46 +05:30
Pere Miquel Brull
5bd1b09402
Add new Looker test connection steps (#11749) 2023-05-24 18:01:08 +02:00
Sriharsha Chintalapani
160984baf1
Fix #11673: Service connection details will be viewable based on view permissions and by default masked for users and unmasked for bots (#11738) 2023-05-24 08:54:14 +02:00
Pere Miquel Brull
da32d392d8
Revert ES backend version (#11703)
* Revert ES backend version

* Revert ES backend version

* Revert ES backend version

* Format
2023-05-23 10:00:06 -07:00
Suresh Srinivas
ac8498ab80
Remove unnecessary singletons (#11713) 2023-05-23 08:37:03 -07:00
Pere Miquel Brull
ea1930e6c2
fix: upgrade software.amazon.awssdk:ssm from 2.20.53 to 2.20.55 (#11697)
Snyk has created this PR to upgrade software.amazon.awssdk:ssm from 2.20.53 to 2.20.55.

See this package in Maven Repository:
https://mvnrepository.com/artifact/software.amazon.awssdk/ssm/

See this project in Snyk:
https://app.snyk.io/org/harshach/project/97e6fbe6-21ff-4c69-b9c5-388bb60a9097?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-05-22 23:04:31 -07:00
Mohit Yadav
3050363faa
Fix Random Password not generated when Email Config is present (#11701) 2023-05-22 11:29:33 +05:30
Pere Miquel Brull
1e8cef223e
fix: upgrade software.amazon.awssdk:cloudwatch from 2.20.43 to 2.20.53 (#11668)
Snyk has created this PR to upgrade software.amazon.awssdk:cloudwatch from 2.20.43 to 2.20.53.

See this package in Maven Repository:
https://mvnrepository.com/artifact/software.amazon.awssdk/cloudwatch/

See this project in Snyk:
https://app.snyk.io/org/harshach/project/97e6fbe6-21ff-4c69-b9c5-388bb60a9097?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2023-05-19 10:46:24 -07:00
Pere Miquel Brull
d52d773707
Send encrypted automation workflow (#11681) 2023-05-19 15:04:42 +02:00
Mayur Singal
e9992a52a8
Fix #1604: Add Spline Pipeline Connector (#11562)
* Fix #1604: Add Spline Connector

* Add tests & grammer validation

* Spline UI Changes & Docs

* fix pipeline workflow doc

* chore: use common field for dbService name

* chore: use const for beta services

* chore: add service icon

* Update ingestion/src/metadata/ingestion/source/pipeline/spline/metadata.py

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-05-19 14:46:32 +05:30
Pere Miquel Brull
50ad38ea0f
Fix #11548 - Secrets Managers comms with OMeta (#11602)
* Remove secretsManagerCredentials from backend

* Remove secretsManagerCredentials from backend

* Add secrets manager loader

* Load SM in the ometa client

* Fix tests

* Fix tests

* Fix Lint

* Mock AWS region

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-05-19 09:43:11 +02:00
Deepa Rao
2369c782f8
Part of fix 11492 - Add missing fields in Container update (#11493)
* Part of fix 11492 -  Provide support for updating Container entity

* Part of fix 11492 -  Provide support for updating Container entity
2023-05-18 12:52:54 -07:00
07Himank
4d2bbb3b75
Logged the Email Messages which are not sent as smpt settings is not … (#11650)
* logged the Email Messages which are not sent as smpt settings is not enabled

* modified change event message

* used a genric message for all

* logs a genric message with email id

---------

Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
2023-05-18 19:03:14 +05:30
Onkar Ravgan
e6a7ffc57c
Fixes: 11540 Added api_version and domain fields to Salesforce (#11618)
* Added sales new fields

* fixed image after merge

* Fix Failing Test

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-05-18 10:53:44 +05:30