743 Commits

Author SHA1 Message Date
Sachin Chaurasiya
55cd180ffa
feat(#16208): custom properties for Dashboard Data Models (#16628)
* feat(#16208): custom properties for Dashboard Data Models

* chore: update the padding for custom properties tab content

* Backend : Support extension field for dashboardDataModel

* ui: update ui to fetch the extension data

* feat: show custom properties on right panel

* chore: add custom properties in version page

* test: add playwright test for data model custom properties

* test: remove duplicate checks

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-06-13 20:51:47 +05:30
Onkar Ravgan
4f50e0b6a0
cleaned dbt code after pydantic update (#16605) 2024-06-12 11:24:29 +05:30
Matt Chamberlin
5236950b6d
Fixes 16595: Parse Iceberg REST table FQN from identifier (#16596) 2024-06-11 15:07:57 +05:30
Teddy
ff00175fb1
MINOR - Data Quality Preparatory Work (#16523)
* chore: added min/max bound and dynamic support flag

* chore: added migration for dynamic attribute

* style: fix pythin checkstyle
2024-06-10 11:03:55 +02:00
Pere Miquel Brull
f5996b2f3a
FIX - e2e test for pydantic v2 (#16572) 2024-06-07 22:52:16 +02:00
Pere Miquel Brull
cb72a22b59
Fix - e2e tests for pydantic V2 (#16551)
* Fix - e2e tests for pydantic V2

* add correct default

* add correct default

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* revert datetime aware

* fix apis

* format
2024-06-06 19:36:17 -07:00
Mohit Yadav
9ec3d94e3b
[FIX] GlossaryTerm reviewers should be user or team only (#16372)
* add teams as reviewer

* Check Users to be reviewers

* Reviewers can be a team or user

* Fix check by id or name

* Review can be team or user both

* Validate Reviewers

* add multi select control

* - Fix Reviewers

* - Centralize Reviewer Relationship to EntityRepository

* - Sort

* add team as reviewer for glossary terms

* locales

* cleanup

* - Update Reviewer should remove existing reviewers

* fix selectable owner control

* fix code smells

* fix reviewer issue

* add glossary cypress

* fix patch issue on reviewers set to null

* update cypress tests

* fix cypress

* fix cypress

* fix reviewers in glossary task and supported cypress

* fix pytest

* Fix

* fix cypress

* fix code smells

* Inherited Reviewers need to be present always

* filter out inherited users

* fix cypress

* fix backend tests failure

* fix backend tests failure -checkstyle

* restrict owner to accept task in case of reviewer present

* fix pytest

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-06-06 20:23:37 +05:30
Pere Miquel Brull
d8e2187980
#15243 - Pydantic V2 & Airflow 2.9 (#16480)
* pydantic v2

* pydanticv2

* fix parser

* fix annotated

* fix model dumping

* mysql ingestion

* clean root models

* clean root models

* bump airflow

* bump airflow

* bump airflow

* optionals

* optionals

* optionals

* jdk

* airflow migrate

* fab provider

* fab provider

* fab provider

* some more fixes

* fixing tests and imports

* model_dump and model_validate

* model_dump and model_validate

* model_dump and model_validate

* union

* pylint

* pylint

* integration tests

* fix CostAnalysisReportData

* integration tests

* tests

* missing defaults

* missing defaults
2024-06-05 21:18:37 +02:00
Teddy
6b00dde902
Fix #16229 - Tag and Service filters for test cases (#16484)
* fix: added test case support for tags (inherit from table/column)]"

* feat: add tag and service filter for test cases

* feat: add tier query param

* fix: tests
2024-06-03 22:32:01 -07:00
Siddhant
4ba05c2401
Bug fix: Added DataProduct support for Glossary and GlossaryTerm. (#16473)
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-05-30 14:11:41 +05:30
gpby
d909a3141e
Teradata Connector (#16373)
* [WIP] add teradata connector

* [WIP] add teradata ingestion

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* [WIP] add teradata connector

* Reformat code

* Remove unused databaseName property
2024-05-28 06:40:22 +02:00
Mohit Yadav
fe2db2d63c
Add Null Check for isAdmin (#16407)
* Remove Retry From Abstract Event Consumer

* - Add Check for null Or Empty in isAdmin

* - Fix Test
2024-05-24 15:58:41 +05:30
Akash Jain
015d71c9de
chore: Update main branch for 1.5.0 Release (#16401) 2024-05-23 18:46:15 +05:30
Aniket Katkar
798fe311c0
Improvements: Activity feed improvements (#16221)
* Pass parameterValues for the test case summary feed

* Change test case link formation to navigate to incident manager page

* Changes to show graph plot of test summary in the feeds

* Localization change for other languages

* Fix unit tests
Add missing tests

* Fix the feed cards for Team and Application

* Update localization for other languages

* Add icon for the chart asset
2024-05-11 20:04:16 +05:30
Suman Maharana
0e2736ee74
MINOR: Removed supportsDDL from json schemas (#16171) 2024-05-10 17:40:12 +05:30
Mohit Yadav
de850f1b3b
- Default value for isBot should be false (#16163)
* - Default value for isBot should be false

* - Test Fix

* - Fix Failing Test
2024-05-08 19:41:33 +05:30
sonika-shah
abdaea55e2
Fix custom property entity and entityReferenceList (#16145)
* fix validation error for entity and entityRef property types

* fix validation error for entity and entityRef property types

* fix validation error for entity and entityRef property types

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-05-08 07:01:09 +02:00
Mayur Singal
e13a1af395
MINOR: S3 Add support for single bucket ingestion (#16096) 2024-05-07 11:33:57 +05:30
Aniket Katkar
23a526db4c
Feat: Activity feed revamp (#15716)
* add new activity feed card component according to the new designs

* add new localization values

* replace the old ActivityFeedCardV1 component with the new ActivityFeedCardV2
delete unused components

* add the delete and edit options for the feeds

* move new activity feed card component to a common folder

* Refactor activity feed header to implement new backend changes

* Refactor activity feed header to implement new backend changes

* Add DescriptionFeed component to show feed info for description changes

* Improve getFeedChangeOperationLabel util function

* Fix activity feed card styling to remove text primary color on hover

* Refactor code for ActivityFeedCardV2 component and add new test case feed components

* Update localization files for test status pluralization

* Update cardStyle enum values in thread.json schema

* Update feed header messages

* Update localization files for feed headers

* Update feed body to display asset details for entityCreated, entityDeleted and entitySoftDeleted

* Update localization files for soft-deleted-lowercase translation

* Add tags feed component show feeds related to tags

* Update localization files for added and removed entity translations

* Refactor TagsFeed component to remove unnecessary code

* - Send Actual Field Name

* - Add json Type in one thread.json

* added feed support for customProperty

* localization keys

* fix sonar issue

* changes made as per comments

* localization keys

* fix feed message not coming up

* testCaseFeed fix

* localizaion keys added

* removed action button on system generated feed

* fix redirect link for alerts feed

* fix the classification and glossary feed card icon

* fix the classification icon and card ui overflow

* minor fix

* changes as per comments

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-05-06 21:42:08 +05:30
Sachin Chaurasiya
b04837cdeb
Minor: make password and username field non required in email config form (#16133)
* Minor: make password field as non required in email config form

* chore: Update email config form to make password field non-required
2024-05-06 19:25:34 +05:30
Mohit Yadav
0769d71ee7
Fixes Test Suite Reference in Table Schema (#16129)
* Fixes Test Suite Reference in Table Schema

* fix: fix test suite to interact with entity reference

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2024-05-06 19:03:23 +05:30
Suman Maharana
488078da8a
Add DDL query ingest (#15860) 2024-05-06 18:03:50 +05:30
Mayur Singal
215a274bd0
Fix #14130: Add support for azure auth in trino (#16117) 2024-05-03 15:17:25 +05:30
Onkar Ravgan
3d2e2e8214
MINOR: Updated test case name field (#16048)
* Removed entityName char limit

* updated new field

* fixed backend tests

* fixed linting
2024-05-03 10:19:44 +02:00
Yuki Shiga
f2cfbd5982
Fixes #16075: Fix Incorrect Spark Property Name in DeltaLake Connection Configuration (#16097)
* fix typo (sparks.driver.extraClassPath -> spark.driver.extraClassPath)

* Fix incorrect Spark session username configuration in DeltaLake connection
2024-05-02 18:04:31 +02:00
Onkar Ravgan
c63d441aa1
Fix #12929: Added config to include/exclude undeployed pipelines while ingestion (#15797) 2024-04-30 10:54:06 +05:30
Pere Miquel Brull
a5e062cf30
MINOR - Metadata Actions - New Actions (#16037)
* MINOR - Metadata Actions - New Actions

* Add actions

* Add actions

* Add actions

* Add actions

* Add actions

* Add actions

* Add support for chart search in UI

* Refactor searchEntity function in AlertsUtil.tsx and getSearchIndexEntityTypeMapping function in SearchClassBase.ts

* add domain

* add domain

* format

* Update setSourceAsValue parameter in searchEntity function

* add missing fields in the advanced search field selection for all the assets

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-04-30 07:11:03 +02:00
sonika-shah
432820d929
Fix #16046 : modify glossaryTerms api endpoint to support querying immediate children with childrenCount (#16056)
* Fix #16046 : modify glossaryTerms api endpoint to support querying immediate children with childrenCount

* childrenCount added in glossary_term_index_mapping.json

* childrenCount added in glossary_term_index_mapping.json
2024-04-30 00:01:07 +05:30
Mayur Singal
eac0466b4d
Fix #16064: Add support for dynamic table type (#16069) 2024-04-29 18:56:52 +05:30
Mohit Yadav
92e5b080cf
Add more information to activity feed (#16007)
* Add more information to activity feed

* Null Or Empty Descriptions

* give owner reference

* - Entity Domain Updates

* - Fix Description

* - Custom Properties Addition
2024-04-29 18:16:21 +05:30
Onkar Ravgan
ceaa9d3e8a
Fix #15611 Parse PowerBI Dax files for lineage (#15975) 2024-04-29 14:55:06 +05:30
Mohit Yadav
33f1e4f4f0
- Fix displayName (#16059) 2024-04-26 15:26:55 +02:00
Imri Paran
4ac5912d4c
MINOR: added TestCase inspection query to backend and sample data (#16003)
* added TestCase inspection query to backend and sample data

* format

* format
2024-04-26 11:49:08 +02:00
sonika-shah
a9773b9527
Add predefined color palette for UI styling assets (#15982)
* Add predefined color palette for UI styling assets - backend

* Update UI components to use customLogoConfig in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.tsx

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.test.tsx

* Remove existing customLogoConfiguration

* Refactor App.tsx and ThemeUtils.ts files

* Update AppearanceConfigSettingsPage.tsx with new imports and default theme values

* Update AppearanceConfigSettingsPage.tsx with new imports and reset functionality

* add cypress test

* Fix color update issue in CustomThemeConfig.spec.ts

* Remove existing customLogoConfiguration -migration changes

* Fix theme configuration initialization in useApplicationStore.ts

* Update CustomThemeConfig.spec.ts to check if the color is updated

* update existing customLogoConfiguration with customUiThemePreference -migration changes

* Update CustomThemeConfig.spec.ts to fix color update check

* Add unit tests for ThemeUtils

* move validateJsonSchema method to JsonUtils

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-25 22:43:02 +05:30
Ayush Shah
3621407642
Fixes #15732: Modify Reference for Tags to EntityName (#15938) 2024-04-25 11:53:46 +05:30
Onkar Ravgan
536aa62748
Added None option to db config (#15994) 2024-04-25 10:20:09 +05:30
Teddy
5606e6209d
FIX #15593 - test case exc summary endpoint improvements (#15992) 2024-04-23 20:30:05 +02:00
Teddy
449a5f2de3
FIX #11951 - ingestion logic for global profiler config (#15948)
* feat: add global metric configuration for the profiler

* style: ran java linting

* fix: renamed disable to disabled

* style: ran java linting

* feat: ometa sdk for profiler setting

* test: ingestion profiler global config tests

* fix: update metric name to use MetricType Enum

* fix: allow bot to retrieve settings

* fix: exclude GX artifacts

* feat: implement global profiler setting logic for ingestion side

* fix: exclude metrics if Metric is empty

* style: ran python linting

* style: ran python linting

* fix: skip empty metrics

* style: ran python linting

* fix: moved GET profiler config to seperate endpoint in system resource

* fix: moved compute metric filter to MetricFilter + renamed container

* fix: test failures

* fix: profiler test case
2024-04-22 22:35:37 +02:00
IceS2
08c114c340
FIXES 15626: Fix issue with not url model store (#15974)
* Changed the MLModelStore storage type to string

* fix checkstyle

* remove unused files

* Update requirements

* fix checkstyle

* Skipping MLFlow intergration on python 3.8

* Hack to allow pytest to parse the mlflow integrations test on python 3.8

* Fix checkstyle
2024-04-22 15:50:44 +02:00
Mayur Singal
c04ec3e922
Fix #14973: Qliksense - Add option to disable hostname validation (#15954) 2024-04-22 18:49:59 +05:30
Mohit Yadav
27e9012a56
Activity Feed Improvements (#15840)
* - Created Specific Schemas for different change Events
- Added Card Style for different Events
- Added Feed Info for runtime card creation and graph drawing in case test Case Result updates

* Review comment

* Test fix

* Revert unrequired changes

* Reverted changes

* Fixed Tests

* fix test

* Add Domain, Data Producs Asset addition/removal Uppdates

* Probale test case failure fix

* Followers Fields
2024-04-22 18:42:40 +05:30
Imri Paran
0a1018648c
Fixes #15566: add dynamodb row count (#15204)
* feat(nosql-profiler): row count

1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.

* added profiler support for mongodb connection

* doc

* use int_admin_ometa in test setup

* - fixed linting issue in gx
- removed unused inheritance

* moved the nosql function into the metric class

* feat(profiler): add dynamodb row count

* feat(profiler): add dynamodb row count

* formatting

* fixed import

* format

* dded dynamodb row count

* format

* removed unused factory file

* removed "validate"

* migrations

* removed validations

* format

* linting

* fixed: test_amundsen.py

* Update schemaChanges.sql
2024-04-22 09:14:52 +02:00
Pere Miquel Brull
b1a4faf171
MINOR - Metadata Actions schemas (#15822)
* MINOR - Metadata Actions schemas

* add type for left side menu bar items

* fix the required fields in automatorAppConfig.json schema

* update constants

* mixin

* create new styled component to show in alerts form card

* Add new translations for ingestion pipeline and location

* Fix import statement in AlertDetailsPage.tsx

* Update automatorAppConfig.json schema to use an array for the 'type' property

* config

* move the advanced search configs to a new classBase for better customization

* Update automatorAppConfig.json schema to add "name" and "description" fields

* Update automatorAppConfig.json schema to remove "name" and "description" fields

* Add new translations for description-plural in multiple language files

* filter Automator

* revert remove

* Update pipeline actions interface and component to handle edit click event

* Update select type for advanced search fields in AdvancedSearchClassBase.ts

* Update ErrorPlaceHolderIngestion component to accept cardClassName prop

* Update FormCardSection component to accept className prop

* Update TagsContainerV2 component to include state property in tag data

* Update TagsContainerV2 component to include defaultState and defaultLabelType properties

* Update ingestionPipelineAPI.ts to include applicationType parameter in getIngestionPipelines function

* Update AdvanceSearchProvider.test.tsx to include pathname in useLocation mock

* work on comments

* fix sonar lint errors

* format

* Update useLocation mock in AdvanceSearchProvider.test.tsx to include pathname

* revert remove

* Update LogsViewerPage interfaces and component to include application_task in IngestionPipelineLogByIdInterface

* Update LeftSidebar component to import LeftSidebarItem interface and fix type annotation in children map function

* fix observability alert test

* revert the form list add method change

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-04-19 12:46:58 +02:00
Ayush Shah
d5b1465406
Fixes #14113 - Allow SSL file uploads (#15828) 2024-04-19 11:38:27 +05:30
Mohit Yadav
bc13659a6d
Inherit sso roles (#15903)
* Use Roles from Provider

* Add Cache For Roles From Users

* Fix Failing Test

* Revert secrets

Update Impl to use loggedInUser and CreateUser for Role assignment and resync

* Revert Expiry check
2024-04-18 20:46:04 +05:30
sonika-shah
45bae608bd
Fixes #10724 : Search with hierarchy api for glossary terms (#15865)
* Fixes #10724 : New search with hierarchy api for glossary terms

* Added query param in existing search api - to search with hierarchy

* Created new json to manage hierarchy and other updates
2024-04-18 14:22:18 +05:30
Teddy
8d377f5c99
Fix #11951 [backend] - Implement Profiler Global Config (#15889)
* feat: add global metric configuration for the profiler

* style: ran java linting

* fix: renamed disable to disabled

* style: ran java linting

* fix: json schema profiler config title
2024-04-16 18:56:21 +02:00
Imri Paran
6349adb0ec
MINOR: added a DELETE method for failedRowsSample (#15878)
* added a DELETE method for failedRowsSample

* fixed tests

* changed required permission to EDIT_SAMPLE_DATA

* changed required permission to DELETE_SAMPLE_DATA

* changed required permission to DELETE_TEST_CASE_FAILED_ROWS_SAMPLE

* changed required permissions

addFailedRowsSample -> EDIT_TEST_CASE_FAILED_ROWS_SAMPLE
getFailedRowsSample -> VIEW_TEST_CASE_FAILED_ROWS_SAMPLE

* removed op EditTestCaseFailedRowsSample

* fixed tests

* format
2024-04-16 17:45:59 +02:00
Teddy
84469e643c
FIX #13475 - Add Parameter Validation for Min/Max tests (#15849)
* feat: add validation field to test case definition parameters

* feat: added parameter validation logic

* fix: set ES testcontainer startup attemp to 3

* style: ran java linting

* fix: added waitForEsAsyncOp in post_trigger_app_200 test
2024-04-15 07:52:43 -07:00
Karan Hotchandani
bd3838868c
disable install for preview apps (#15884)
* disable install for preview apps

* move screenshot path to base class

* MINOR - Add Marketplace preview

* format

* add unit tests

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-04-15 17:32:01 +05:30