717 Commits

Author SHA1 Message Date
harshsoni2024
6fa662bfda Connector: VertexAI UI, docs (#17649) 2024-09-04 18:07:31 +05:30
harshsoni2024
5aba41ef58 Connector: vertexai schema (#17632)
* vertexai connection schema

* remove yaml file
2024-09-04 18:06:26 +05:30
Sriharsha Chintalapani
a477e23854 create 1.5.3 release branch 2024-09-02 10:49:43 -07:00
Pere Miquel Brull
72afde0aca MINOR - Implement SAP Hana Lineage (#17615)
* MINOR - SAP Hana Lineage

* skeleton

* parser

* lineage

* manage formulas

* add cvs

* add cvs

* better typing

* enum

* handle cvs

* saphana docs
2024-09-02 07:42:35 +02:00
sonika-shah
7e4a06af34 Fixes #16498 : Support custom properties for Data Products (#17641)
(cherry picked from commit 6d322577653aea67d0f192b65b1edb6b6b4616a2)
2024-08-31 09:47:57 +05:30
github-actions
aff2adbb98 chore(release): Prepare Branch for 1.5.2 2024-08-28 20:25:55 +00:00
github-actions
8ec2cdc638 chore(release): Prepare Branch for 1.5.1 2024-08-27 23:14:48 +00:00
Imri Paran
b48c6a0485
feat(postgres): add money profile (#17558)
add support for profiling of money type
2024-08-22 14:53:34 -07:00
IceS2
9e7ddfbd69
MINOR: Fix unparsable date and remove legacy code (#17549)
* Fix unparsable date and remove legacy code

* fix ui for data insight cleanup

* fix unit tests

* fix tour playwright

* fix percentage value and label for charts

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
2024-08-22 14:52:47 -07:00
gpby
61e835093f
add support for Teradata LDAP logon mechanism (#17454) 2024-08-22 08:34:52 +02:00
IceS2
f80388bcb9
MINOR: Add unique count to data insights (#17505)
* Adding Unique Count via ElasticSearch Cardinality

* Add Unique Count to Data Insights functions
2024-08-21 09:30:38 +02:00
Chirag Madlani
8ad3db243b
fix(ui): misc issues (#17447)
* fix(ui): redirect unauthorised users to error page

* remove left panel for import action in glossary import

* fix data insight app stuck in aut

* fix path issue

* fix setup issue

* add tree field to build advance filter on ui side

* update unauthorised page level to root instead authenticated

* minor fixes

* update error message
2024-08-20 13:01:00 +05:30
IceS2
14a473e5d3
Make Transient table ingestion On by default (#17484) 2024-08-19 14:18:45 +02:00
Mayur Singal
8acf6d3e94
MINOR: Make Include ddl disabled by default (#17450)
* MINOR: Make Include ddl disabled by default

* make schema def failure warning debug

* Add missing condition

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2024-08-19 14:07:18 +02:00
Shailesh Parmar
75f62a7872
#17437 Data Quality UI can't display type as column name (#17442) 2024-08-14 18:01:41 +00:00
Pere Miquel Brull
4a0c251e5c
MINOR - Config to PATCH metapilot descriptions if empty (#17424)
* MINOR - Config to PATCH metapilot descriptions if empty

* MINOR - Config to PATCH metapilot descriptions if empty
2024-08-14 15:41:53 +02:00
Mohit Yadav
7033870e71
Update SAML to have signed Messages (#17415)
* Update SAML to have signed Messages

* test fix
2024-08-13 11:38:37 +05:30
Mohit Yadav
05baba9dc1
Change Default Value of Batch Size and Payload Size (#17391) 2024-08-12 11:18:18 +05:30
Mohit Yadav
aa5b9b2985
Increase Payload Size to 100MIB for ELastic Search (#17366)
* Increase Payload Size to 100MIB

* Increase batchSize Size to 500

* Add Payload Size as option in openmetadataoperations , and searchIndexJob

* //Comment

* Add Logs
2024-08-11 21:58:55 +05:30
Mohit Yadav
6e80d3f405
Fix Migrations (#17364) 2024-08-09 20:58:44 +05:30
IceS2
717c9709e0
Update Data Insights Pipeline to change CustomProperty key based on entityType (#17350) 2024-08-08 18:50:27 -07:00
Mayur Singal
d3baf4fcf2
MINOR: DI Improve system charts (#17310) 2024-08-07 14:15:09 +05:30
sam-mccarty-mavenclinic
11a49ef08d
Fixes 16010: Support Looker ingestion from Gitlab (#17284)
* introduce gitlab option to lookml ingestion

* fix reader and disable test

* fix copy paste in test case

* fix file read and keyset pagination for tree

* fix credentials to include gitlab credentials

* uncomment arguments for unused credentials to fix validation error

* fix credentials test

* fix credentials test

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-08-07 06:53:44 +02:00
Siddhant
40009dbeae
Slack App Integration. (#17286)
* slack bolt app integration.

* checkstyle fix.

* fix slack uninstall.

* removed callback and events from JwtFilter.
2024-08-06 10:23:05 +05:30
Mohit Yadav
f07daae196
[RbacDomain] RBAC with Domain Restrictions (#16586)
* Add domain field to entity jsons

* add subdomains ui

* Add Domain To create Request

* Typo fix

* commit progress

* allow multiple domains

* - add userDomains and teamDomains

* - Restrict Api Access

* allow multiple domain support

* - Add Listing Restricition
- Add Search Restrictions

* - Field Fix

* - Inherit Team Domains

* remove glossary filter from ui

* - Fix Tag and User Filter

* - Id -> FQN in create

* - Missing domain on indexes

* add domain in creation of team, user and glossary

* fix merge conflicts

* fix feedbacks

* fix tests

* add domain playwright

* minor changes

* fix domain fetching calls

* Add Thread Listing with EntityDomain

* Fixes for Team Join

* Mkae list mutable

* Add missing import

* Fix Team Test

* User Test case fix

* Fix User Domain

* Minor bug fixes related to sub domains

* Fix data retaining in add sub domain modal after closing

* fix NavBar unit test

* Make Domain Access Work Via Policies

* Fix errors introduced after conflict resolutions

* update render function in SubDomainsTable.component.tsx to accept an array of owners instead of a single owner

* fix java checkstyle

* fix py-test

* format

* Fix the domain e2e owner field type

* Fix Failing Tests

* fixed pytest list index

* update userDomains and teamDomains to domains

* Entity Domain to Domain for thread and changeEvent

* FIx fields

* @Ovveride

* FIx

* Remove Duplicate Field

* Remove Duplicate Field

* Fix Failing Test

* fix domain related changes

* fix subdomains loader

* Fix Owners Order

* Enforce Domain Permissions

* fix domain cleanup

* Fix Test

* rever owner change

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
2024-08-02 10:16:14 +05:30
Siddhant
0ca3d7afb8
Fetch templates for collate and oss. (#17254)
* fetch template for oss and collate.

* migrations.
2024-08-01 11:01:36 +02:00
Sachin Chaurasiya
7d519e180e
minor: add api entities in EntityLink.g4 (#17231) 2024-07-31 08:07:22 +00:00
Mayur Singal
840a102887
Fix #17195: Support automated unstructured files ingestion & tags (#17196) 2024-07-31 00:05:58 +05:30
Pere Miquel Brull
f52e93f88f
MINOR - Automator multiple ownership (#17232) 2024-07-30 15:22:49 +02:00
Mayur Singal
ef67474310
Data Insights Refactor (#16532)
---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Pablo Takara <pjt1991@gmail.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-07-30 17:38:43 +05:30
Sriharsha Chintalapani
fe107aa3cb
Issue #17012: Multi User/Team Ownership (#17013)
* Add multiple owners

* Multi Ownership

* Issue #17012: Multi User/Team Ownership

* Issue #17012: Multi User/Team Ownership

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 1

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 2

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 3

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 4

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 5

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 6

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 7

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 8

* Add Migrations for Owner Thread

* update ingestion for multi owner

* fix pytests

* fixed checkstyle

* Add Alert Name to Publishers (#17108)

* Add Alert Name to Publishers

* Fix Test

* Add Bound to Setuptools (#17105)

* Minor: fixed testSummaryGraph issue (#17115)

* feat: updated multi pipeline ui as per new mock (#17106)

* feat: updated multi pipeline ui as per new mock

* translation sync

* fixed failing unit test

* fixed playwright test

* fixed viewService click issue

* sorted pipeline based on test case length

* Added domo federated dataset support (#17061)

* fix usernames (#17122)

* Doc: Updated Doris & Redshift Docs (#17123)

Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>

* Fix #12677: Added Synapse Connector - docs and side docs (#17041)

* Fix #17098: Fixed case sensitive partition column name in Bigquery (#17104)

* Fixed case sensitive partiion col name bigquery

* update test

* #13876: change placement of comment and close button in task approval workflow (#17044)

* change placment of comment and close button in task approval workflow

* minor change

* playwright test for the close and comment function

* supported ref in activityFeedEditor

* fix playwright test

* added playwright test for data steward

* fix the test for the data streward user

* fix the close button not showing if task has no suggestions and icon fixes

* fix sonar issue

* change glossary and add suggestion button to dropdown button

* fix the glossary failure due to button change

* icon change for add tag and description

* fix glossary cypress failure due to button chnages

* changes as per comments

* MINOR: docs links fix (#17125)

* alation link fix

* dbt yaml config source link fix

* bigquery doc fix

* Explore tree feedbacks (#17078)

* fix explore design

* update switcher icon

* show menu when search query exists

* fix selection of active service

* fix type error

* fix tests

* fix tests

* fix tests

* MINOR: Databricks view TableType fix (#17124)

* Minor: fixed AUT test (#17128)

* Fix #16692: Override Lineage Support for View & Dashboard Lineage (#17064)

* #17065: fix the tags not rendering in selector after selection in edit tags task (#17107)

* fix the tags not rendering in selector after selection in edit tags taks

* added playwright test

* minor changes

* minor fix

* fix the tags not updating in edit and accept tag

* fix explore type changes for collate (#17131)

* MINOR: changed log level to debug (#17126)

* changed log level to debug

* fixed type

* changed type to optional

* Get feed and count data of soft deleted user (#17135)

* Doc: Adding OIDC Docs (#17139)

Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>

* Doc: Updating Profiler Workflow Docs URL (#17140)

Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>

* fix playwright and cypress (#17138)

* Minor: fixed edit modal issue for sql test case (#17132)

* Minor: fixed edit modal issue for sql test case

* fixed test

* Minor: Added whats new content for 1.4.6 release (#17148)

* MINOR [GEN-799]: add option to disable manual trigger using scheduleType (#17031)

* fix: raise for triggering system app

* added scheduleType ScheduledOrManual

* minor: remove "service" field from required properties in createAPIEndpoint schema (#17147)

* initial commit multi ownership

* update glossary and other entities

* update owners

* fix version pages

* fix tests

* Update entity_extension to move owner to array (#17200)

* fix tests

* fix api page errors

* fix owner label design

* locales

* fix owners in elastic search source

* fix types

* fix tests

* fix tests

* Updated CustomMetric owner to entityReferenceList. (#17211)

* Fix owners field in search mappings

* fix search aggregates

* fix inherited label

* Issue #17012: Multi User/Team Ownership - Fix Tests - Part 9

* Fix QUeries

* Fix Mysql Queries

* Typo

* fix tests

* fix tests

* fix tests

* fix advanced search constants

* fix service ingestion tests

* fix tests

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: k.nakagaki <141020064+nakaken-churadata@users.noreply.github.com>
Co-authored-by: Prajwal214 <167504578+Prajwal214@users.noreply.github.com>
Co-authored-by: Prajwal Pandit <prajwalpandit@Prajwals-MacBook-Air.local>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: harshsoni2024 <64592571+harshsoni2024@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Imri Paran <imri.paran@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
2024-07-30 08:06:39 +02:00
Siddhant
f1c1615c0d
[Initial draft] Alert Destination Test Functionality (#17207)
* Implemented TemplateProvider.

* validating templatePlaceholders.

* Refactor: Update query to fetch documents for EmailTemplate type. Implemented a function that retrieves documents from doc_store for documents of type "EmailTemplate". This function returns the email template content.

* revert commit.

* revert commit.

* revert commit.

* initial draft test alert functionality.
2024-07-29 19:55:17 +05:30
Onkar Ravgan
c03f47118b
unify tableau datamodels (#17185) 2024-07-29 10:32:29 +05:30
Onkar Ravgan
b6745d7cf1
MINOR: Implemented SAPERP Connector feedback (#17137)
* Implemented saperp feedback

* Fixed pytest
2024-07-29 10:31:20 +05:30
Mohit Yadav
c6e7938e47
[Issue-16343]Activity Feed Improvements (#17086)
* Add entityRef in Thread_entity

* fix testCase breadcrumb, profile pic and supported entity reference

* fix sonar

* fix feed reseting on deleting comments

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-07-28 05:25:06 -07:00
Teddy
ab7b1a444c
[GEN-976] Generic Aggregation Endpoint for Data Quality Reports (#17181)
* fix: added parser for aggregation queryParams

* feat: added generic aggregation logic

* feat: added DQ generic aggregation endpoint

* fix: typo in resources description
2024-07-26 15:23:10 +05:30
Siddhant
d17d13cee8
Email template provider (#17015)
* Implemented TemplateProvider.

* validating templatePlaceholders.

* Refactor: Update query to fetch documents for EmailTemplate type. Implemented a function that retrieves documents from doc_store for documents of type "EmailTemplate". This function returns the email template content.

* revert commit.

* revert commit.

* revert commit.

* implemented templateProvider.

* Added placeholders definition (emailTemplatePlaceholder.json) for each emailTemplates.

* implemented resetSeedApi.
implemented schema for emailTemplate and emailTemplatePlaceholder.
implemented util functions for fetching placeHolders.

* Fix Issue in Template Provider

* Updated EmailUtil to use getTemplate from DocStore.
Updated ResetEmailTemplate API to only reset EmailTemplate type documents.
Cleaned up the previous method for retrieving the template from the path.

* fixes.

* added email validation test.

* Remove ftls

* Update code to better structure

* Fix Failing Tests

* Add Custom Exception and error on null entity

* Update Default Template Provier

* Fix Condition

* Correct Message

* added dataInsightReport.json

* fix dataInsightReport.json

* Fix Failing Test

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-07-25 12:03:34 +05:30
Onkar Ravgan
79348a2259
MINOR: Added Alation Sink connector (#16982) 2024-07-24 12:08:07 +05:30
Sachin Chaurasiya
303c8ce49e
minor: remove "service" field from required properties in createAPIEndpoint schema (#17147) 2024-07-23 22:38:40 +05:30
Imri Paran
b41a505795
MINOR [GEN-799]: add option to disable manual trigger using scheduleType (#17031)
* fix: raise for triggering system app

* added scheduleType ScheduledOrManual
2024-07-23 17:42:26 +02:00
Mayur Singal
5140203311
Fix #16692: Override Lineage Support for View & Dashboard Lineage (#17064) 2024-07-22 20:42:38 +05:30
Suman Maharana
d3ea1ead01
Fix #12677: Added Synapse Connector - docs and side docs (#17041) 2024-07-22 12:35:46 +05:30
Pere Miquel Brull
7ef90c3628
MINOR - Update MetaPilot config (#17101) 2024-07-19 15:18:41 +02:00
Mohit Yadav
ebdd7f7fd9
Centralize OIDC Flow to handler, and refresh logic (#17082)
* Centralize OIDC Flow to handler, and refresh logic

* Remove forced condition

* Return on success
2024-07-18 23:53:50 +05:30
Imri Paran
0fee79b200
MINOR: fix sample data issue with Pydantic v2 and refactor python integration tests (#16943)
* tests: refactor

refactor tests and consolidate common functionality in integrations.conftest

this enables writing tests more concisely.
demonstrated with postgres and mssql.
will migrate more

* format

* removed helpers

* changed scope of fictures

* changed scope of fixtures

* added profiler test for mssql

* fixed import in data_quality test

* json safe serialization

* format

* set MARS_Connection

* use SerializableTableData instead of TableData

* deleted file test_postgres.py

* fixed tests

* added more test cases

* format

* changed name test_models.py

* removed the logic for serializing table data

* wip

* changed mapping in common type map

* changed mapping in common type map

* reverted TableData imports

* reverted TableData imports

* reverted TableData imports
2024-07-17 08:11:34 +02:00
Antoine Balliet
e67ba6b14c
feat: BigQuery ingestion allow specify projectId and fix primary key constrains retrieval (#16956) 2024-07-17 11:21:17 +05:30
Teddy
924a100ab4
[GEN-876] Implement DQ dimension filter for test cases (#17032)
* fix: typo in key for data dimension

* fix: table test key typo in data dimension

* fix: typo in migration util

* fix: add filter test cases by dimensions in search API

* style: ran java linting
2024-07-16 08:12:39 +02:00
Antoine Balliet
aeea2df86e
feat: allow specify schemaRegistryTopicSuffixName (#17027)
* feat: allow specify schemaRegistryTopicSuffixName

* chore: use default value

* add comment

* fix doc
2024-07-15 17:35:01 +02:00
Sachin Chaurasiya
6e99fe7bda
fix(#15683): backend validation error for the following property types (#16881)
* fix(#15683): backend validation error for the following property types

* feat: Update supported date and datetime formats

* test: add e2e playwright test for custom properties

* chore: remove cypress test for custom properties

* Add date, dateTime, time backend support for custom property

* fix: update custom property display names in AddCustomProperty component

* feat: Update supported date and time formats

* chore: Update supported date and time formats

* feat: Update date and time formats in CustomProperty

* test: add playwright test

* Update supported date and time formats

* feat: Add tests for custom properties in Playwright in 2 parts

* Update supported date and time formats

* Update supported date and time formats

* chore: Update date format to uppercase in PropertyValue component

* feat: Add support for custom date format in PropertyValue component

* Update supported date and time formats

* test: add playwright test for time, dateTime, timeInterval and date properties

* Update supported time formats

* chore: Add focus to time and date pickers before filling values

* chore: Add focus to time and date pickers before filling values

* Remove date, dateTime, time from type_entity

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-07-15 11:37:38 +05:30
Pere Miquel Brull
8d739563f2
FIX CL-548 - Encrypt JWT Token w/ Secrets Manager (#16861)
* encrypt jwt

* encrypt jwt

* Handle token revokes with SM

* #10192 - Validate both include and excludes filters

* #10192 - Validate both include and excludes filters

* docs
2024-07-11 09:16:48 +02:00