652 Commits

Author SHA1 Message Date
Ayush Shah
527b714d34
Fixes #16760: Remove maxLength for tagFQN (#16794) 2024-06-26 21:14:07 +05:30
harshsoni2024
afec7703cc
Fix #11888: Apache flink connector (#16601) 2024-06-26 19:21:16 +05:30
Teddy
38fe061227
MINOR -- Add Test Definition Dimension (#16769)
* feat: added test defintion dimension + sample data for bounds

* chore: added migration for definition dimension

* style: ran python linting

* fix: rename dimension to dataQualityDimension

* fix: test definition dimension key
2024-06-24 15:01:12 +00:00
Suman Maharana
5bd48fcc34
Fixes #14065 : Added DBT Cloud connector (#16705) 2024-06-21 17:16:47 +05:30
Matt Chamberlin
ac6ddbf6c4
MINOR: support JSONL datalake file types (#16614)
* fix: support JSONL datalake file types

* add jsonl zip file types

* update fileFormat enum in table schema

* add tests

* fix test data ref

* reformat

* fix tests

---------

Co-authored-by: Matthew Chamberlin <mchamberlin@ginkgobioworks.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2024-06-21 09:54:19 +02:00
Ayush Shah
c9a017d8db
#16720: Add Support for Salesforce SSL (#16719) 2024-06-20 12:10:41 +05:30
IceS2
f0049853ec
FIXES 14885: Initial deltalake implementation for s3 (#16665)
* Initial deltalake implementation for s3

* Fix styles

* Fix test_amundsen

* Fix UnitTests

* Fix Checkstyle

* Fix integration tests due to datalake client refactor

* Fix unit tests

* Fix tests

* Fix Integration DeltaLake Storage test

* Skip delta storage integration test for python 3.8

* DeltaLake JSONSchema changes migrations

* Update import name

* Add some comments based on sonarcloud suggestions

* Update DeltaLake documentation

* Resolve some comments
2024-06-20 12:08:21 +05:30
Imri Paran
c79c5da00a
Fixes #16632: Add testCases property to a test suite source config (#16631)
* feat: test-suites

enable specifying a subset of test cases to run in a test suite pipeline

* improved property description
2024-06-19 12:47:38 +02:00
Mohit Yadav
53407fb681
[Issue-16642] Add Claim Mapping to uniquely identifty username and email from claims (#16643)
* - Add Claim Mapping to uniquely identift username and email from claims

* - Null Check

* - Add field to yaml

* - Fix issue with token being null

* - Auth Code Flow Fix

* support jwtPrincipleClaimMapping from UI

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
2024-06-19 13:13:09 +05:30
Sachin Chaurasiya
55cd180ffa
feat(#16208): custom properties for Dashboard Data Models (#16628)
* feat(#16208): custom properties for Dashboard Data Models

* chore: update the padding for custom properties tab content

* Backend : Support extension field for dashboardDataModel

* ui: update ui to fetch the extension data

* feat: show custom properties on right panel

* chore: add custom properties in version page

* test: add playwright test for data model custom properties

* test: remove duplicate checks

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-06-13 20:51:47 +05:30
Onkar Ravgan
4f50e0b6a0
cleaned dbt code after pydantic update (#16605) 2024-06-12 11:24:29 +05:30
Matt Chamberlin
5236950b6d
Fixes 16595: Parse Iceberg REST table FQN from identifier (#16596) 2024-06-11 15:07:57 +05:30
Teddy
ff00175fb1
MINOR - Data Quality Preparatory Work (#16523)
* chore: added min/max bound and dynamic support flag

* chore: added migration for dynamic attribute

* style: fix pythin checkstyle
2024-06-10 11:03:55 +02:00
Pere Miquel Brull
f5996b2f3a
FIX - e2e test for pydantic v2 (#16572) 2024-06-07 22:52:16 +02:00
Pere Miquel Brull
cb72a22b59
Fix - e2e tests for pydantic V2 (#16551)
* Fix - e2e tests for pydantic V2

* add correct default

* add correct default

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* fix apis

* format
2024-06-06 19:36:17 -07:00
Mohit Yadav
9ec3d94e3b
[FIX] GlossaryTerm reviewers should be user or team only (#16372)
* add teams as reviewer

* Check Users to be reviewers

* Reviewers can be a team or user

* Fix check by id or name

* Review can be team or user both

* Validate Reviewers

* add multi select control

* - Fix Reviewers

* - Centralize Reviewer Relationship to EntityRepository

* - Sort

* add team as reviewer for glossary terms

* locales

* cleanup

* - Update Reviewer should remove existing reviewers

* fix selectable owner control

* fix code smells

* fix reviewer issue

* add glossary cypress

* fix patch issue on reviewers set to null

* update cypress tests

* fix cypress

* fix cypress

* fix reviewers in glossary task and supported cypress

* fix pytest

* Fix

* fix cypress

* fix code smells

* Inherited Reviewers need to be present always

* filter out inherited users

* fix cypress

* fix backend tests failure

* fix backend tests failure -checkstyle

* restrict owner to accept task in case of reviewer present

* fix pytest

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-06-06 20:23:37 +05:30
Pere Miquel Brull
d8e2187980
#15243 - Pydantic V2 & Airflow 2.9 (#16480)
* pydantic v2

* pydanticv2

* fix parser

* fix annotated

* fix model dumping

* mysql ingestion

* clean root models

* clean root models

* bump airflow

* bump airflow

* bump airflow

* optionals

* optionals

* optionals

* jdk

* airflow migrate

* fab provider

* fab provider

* fab provider

* some more fixes

* fixing tests and imports

* model_dump and model_validate

* model_dump and model_validate

* model_dump and model_validate

* union

* pylint

* pylint

* integration tests

* fix CostAnalysisReportData

* integration tests

* tests

* missing defaults

* missing defaults
2024-06-05 21:18:37 +02:00
Teddy
6b00dde902
Fix #16229 - Tag and Service filters for test cases (#16484)
* fix: added test case support for tags (inherit from table/column)]"

* feat: add tag and service filter for test cases

* feat: add tier query param

* fix: tests
2024-06-03 22:32:01 -07:00
Siddhant
4ba05c2401
Bug fix: Added DataProduct support for Glossary and GlossaryTerm. (#16473)
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-05-30 14:11:41 +05:30
gpby
d909a3141e
Teradata Connector (#16373)
* [WIP] add teradata connector

* [WIP] add teradata ingestion

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* Reformat code

* Remove unused databaseName property
2024-05-28 06:40:22 +02:00
Mohit Yadav
fe2db2d63c
Add Null Check for isAdmin (#16407)
* Remove Retry From Abstract Event Consumer

* - Add Check for null Or Empty in isAdmin

* - Fix Test
2024-05-24 15:58:41 +05:30
Akash Jain
015d71c9de
chore: Update main branch for 1.5.0 Release (#16401) 2024-05-23 18:46:15 +05:30
Aniket Katkar
798fe311c0
Improvements: Activity feed improvements (#16221)
* Pass parameterValues for the test case summary feed

* Change test case link formation to navigate to incident manager page

* Changes to show graph plot of test summary in the feeds

* Localization change for other languages

* Fix unit tests
Add missing tests

* Fix the feed cards for Team and Application

* Update localization for other languages

* Add icon for the chart asset
2024-05-11 20:04:16 +05:30
Suman Maharana
0e2736ee74
MINOR: Removed supportsDDL from json schemas (#16171) 2024-05-10 17:40:12 +05:30
Mohit Yadav
de850f1b3b
- Default value for isBot should be false (#16163)
* - Default value for isBot should be false

* - Test Fix

* - Fix Failing Test
2024-05-08 19:41:33 +05:30
sonika-shah
abdaea55e2
Fix custom property entity and entityReferenceList (#16145)
* fix validation error for entity and entityRef property types

* fix validation error for entity and entityRef property types

* fix validation error for entity and entityRef property types

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-05-08 07:01:09 +02:00
Mayur Singal
e13a1af395
MINOR: S3 Add support for single bucket ingestion (#16096) 2024-05-07 11:33:57 +05:30
Aniket Katkar
23a526db4c
Feat: Activity feed revamp (#15716)
* add new activity feed card component according to the new designs

* add new localization values

* replace the old ActivityFeedCardV1 component with the new ActivityFeedCardV2
delete unused components

* add the delete and edit options for the feeds

* move new activity feed card component to a common folder

* Refactor activity feed header to implement new backend changes

* Refactor activity feed header to implement new backend changes

* Add DescriptionFeed component to show feed info for description changes

* Improve getFeedChangeOperationLabel util function

* Fix activity feed card styling to remove text primary color on hover

* Refactor code for ActivityFeedCardV2 component and add new test case feed components

* Update localization files for test status pluralization

* Update cardStyle enum values in thread.json schema

* Update feed header messages

* Update localization files for feed headers

* Update feed body to display asset details for entityCreated, entityDeleted and entitySoftDeleted

* Update localization files for soft-deleted-lowercase translation

* Add tags feed component show feeds related to tags

* Update localization files for added and removed entity translations

* Refactor TagsFeed component to remove unnecessary code

* - Send Actual Field Name

* - Add json Type in one thread.json

* added feed support for customProperty

* localization keys

* fix sonar issue

* changes made as per comments

* localization keys

* fix feed message not coming up

* testCaseFeed fix

* localizaion keys added

* removed action button on system generated feed

* fix redirect link for alerts feed

* fix the classification and glossary feed card icon

* fix the classification icon and card ui overflow

* minor fix

* changes as per comments

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-05-06 21:42:08 +05:30
Sachin Chaurasiya
b04837cdeb
Minor: make password and username field non required in email config form (#16133)
* Minor: make password field as non required in email config form

* chore: Update email config form to make password field non-required
2024-05-06 19:25:34 +05:30
Mohit Yadav
0769d71ee7
Fixes Test Suite Reference in Table Schema (#16129)
* Fixes Test Suite Reference in Table Schema

* fix: fix test suite to interact with entity reference

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2024-05-06 19:03:23 +05:30
Suman Maharana
488078da8a
Add DDL query ingest (#15860) 2024-05-06 18:03:50 +05:30
Mayur Singal
215a274bd0
Fix #14130: Add support for azure auth in trino (#16117) 2024-05-03 15:17:25 +05:30
Onkar Ravgan
3d2e2e8214
MINOR: Updated test case name field (#16048)
* Removed entityName char limit

* updated new field

* fixed backend tests

* fixed linting
2024-05-03 10:19:44 +02:00
Yuki Shiga
f2cfbd5982
Fixes #16075: Fix Incorrect Spark Property Name in DeltaLake Connection Configuration (#16097)
* fix typo (sparks.driver.extraClassPath -> spark.driver.extraClassPath)

* Fix incorrect Spark session username configuration in DeltaLake connection
2024-05-02 18:04:31 +02:00
Onkar Ravgan
c63d441aa1
Fix #12929: Added config to include/exclude undeployed pipelines while ingestion (#15797) 2024-04-30 10:54:06 +05:30
Pere Miquel Brull
a5e062cf30
MINOR - Metadata Actions - New Actions (#16037)
* MINOR - Metadata Actions - New Actions

* Add actions

* Add actions

* Add actions

* Add actions

* Add actions

* Add actions

* Add support for chart search in UI

* Refactor searchEntity function in AlertsUtil.tsx and getSearchIndexEntityTypeMapping function in SearchClassBase.ts

* add domain

* add domain

* format

* Update setSourceAsValue parameter in searchEntity function

* add missing fields in the advanced search field selection for all the assets

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-04-30 07:11:03 +02:00
sonika-shah
432820d929
Fix #16046 : modify glossaryTerms api endpoint to support querying immediate children with childrenCount (#16056)
* Fix #16046 : modify glossaryTerms api endpoint to support querying immediate children with childrenCount

* childrenCount added in glossary_term_index_mapping.json

* childrenCount added in glossary_term_index_mapping.json
2024-04-30 00:01:07 +05:30
Mayur Singal
eac0466b4d
Fix #16064: Add support for dynamic table type (#16069) 2024-04-29 18:56:52 +05:30
Mohit Yadav
92e5b080cf
Add more information to activity feed (#16007)
* Add more information to activity feed

* Null Or Empty Descriptions

* give owner reference

* - Entity Domain Updates

* - Fix Description

* - Custom Properties Addition
2024-04-29 18:16:21 +05:30
Onkar Ravgan
ceaa9d3e8a
Fix #15611 Parse PowerBI Dax files for lineage (#15975) 2024-04-29 14:55:06 +05:30
Mohit Yadav
33f1e4f4f0
- Fix displayName (#16059) 2024-04-26 15:26:55 +02:00
Imri Paran
4ac5912d4c
MINOR: added TestCase inspection query to backend and sample data (#16003)
* added TestCase inspection query to backend and sample data

* format

* format
2024-04-26 11:49:08 +02:00
sonika-shah
a9773b9527
Add predefined color palette for UI styling assets (#15982)
* Add predefined color palette for UI styling assets - backend

* Update UI components to use customLogoConfig in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.tsx

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.test.tsx

* Remove existing customLogoConfiguration

* Refactor App.tsx and ThemeUtils.ts files

* Update AppearanceConfigSettingsPage.tsx with new imports and default theme values

* Update AppearanceConfigSettingsPage.tsx with new imports and reset functionality

* add cypress test

* Fix color update issue in CustomThemeConfig.spec.ts

* Remove existing customLogoConfiguration -migration changes

* Fix theme configuration initialization in useApplicationStore.ts

* Update CustomThemeConfig.spec.ts to check if the color is updated

* update existing customLogoConfiguration with customUiThemePreference -migration changes

* Update CustomThemeConfig.spec.ts to fix color update check

* Add unit tests for ThemeUtils

* move validateJsonSchema method to JsonUtils

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-25 22:43:02 +05:30
Ayush Shah
3621407642
Fixes #15732: Modify Reference for Tags to EntityName (#15938) 2024-04-25 11:53:46 +05:30
Onkar Ravgan
536aa62748
Added None option to db config (#15994) 2024-04-25 10:20:09 +05:30
Teddy
5606e6209d
FIX #15593 - test case exc summary endpoint improvements (#15992) 2024-04-23 20:30:05 +02:00
Teddy
449a5f2de3
FIX #11951 - ingestion logic for global profiler config (#15948)
* feat: add global metric configuration for the profiler

* style: ran java linting

* fix: renamed disable to disabled

* style: ran java linting

* feat: ometa sdk for profiler setting

* test: ingestion profiler global config tests

* fix: update metric name to use MetricType Enum

* fix: allow bot to retrieve settings

* fix: exclude GX artifacts

* feat: implement global profiler setting logic for ingestion side

* fix: exclude metrics if Metric is empty

* style: ran python linting

* style: ran python linting

* fix: skip empty metrics

* style: ran python linting

* fix: moved GET profiler config to seperate endpoint in system resource

* fix: moved compute metric filter to MetricFilter + renamed container

* fix: test failures

* fix: profiler test case
2024-04-22 22:35:37 +02:00
IceS2
08c114c340
FIXES 15626: Fix issue with not url model store (#15974)
* Changed the MLModelStore storage type to string

* fix checkstyle

* remove unused files

* Update requirements

* fix checkstyle

* Skipping MLFlow intergration on python 3.8

* Hack to allow pytest to parse the mlflow integrations test on python 3.8

* Fix checkstyle
2024-04-22 15:50:44 +02:00
Mayur Singal
c04ec3e922
Fix #14973: Qliksense - Add option to disable hostname validation (#15954) 2024-04-22 18:49:59 +05:30
Mohit Yadav
27e9012a56
Activity Feed Improvements (#15840)
* - Created Specific Schemas for different change Events
- Added Card Style for different Events
- Added Feed Info for runtime card creation and graph drawing in case test Case Result updates

* Review comment

* Test fix

* Revert unrequired changes

* Reverted changes

* Fixed Tests

* fix test

* Add Domain, Data Producs Asset addition/removal Uppdates

* Probale test case failure fix

* Followers Fields
2024-04-22 18:42:40 +05:30