859 Commits

Author SHA1 Message Date
Suman Maharana
00be51f299
Fixes #17747: dbt update owners (#19144)
* Fixes 17747: dbt update owners

* update messages

* addressed comments

* py_format

* py_format

* Added tests
2025-01-07 16:37:30 +05:30
Suman Maharana
5b77c14dc8
Fixes #18816 - Airbyte api endpoint (#19207)
* Fix Airbyte api endpoint

* Fix Airbyte api endpoint
2025-01-07 12:41:56 +05:30
Akash Verma
39dcb5baef
Feature : Cockroach db connector (#18961) 2025-01-02 13:07:55 +05:30
Mohit Yadav
06d71f9450
Fix EntityHierarchy Schema (#19104)
* Fix Entity Schema

* fix additionalProperties in entityHierarchy json

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2024-12-20 20:13:04 +05:30
Pere Miquel Brull
d084243d0a
MINOR - Add Gov entities for entity ref & Automator Actions (#19090)
* MINOR - Add Gov entities for entity ref

* MINOR - Add Gov entities for entity ref

* add automator configs

* add automator configs
2024-12-20 07:14:52 +01:00
Mohit Yadav
9d934cbcd3
[Code Improv] Added Mappers (#19111)
* Added Mappers and moved logic from controller to mapper class for resources

* Add missed Fqn

* Fix PyTest Failures

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-12-20 11:40:39 +05:30
Akash Verma
69557e8716
fixes: #15742 Oracle stored package feature (#18852) 2024-12-16 19:35:20 +05:30
Mohit Yadav
50ae01e2ce
Add Algorithm option for validation in yaml (#19049)
* Add algorithm option in authentication

* ENtity Repository code remove

* Keep Default Value

* Fix Test

---------

Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
2024-12-15 01:24:03 +05:30
Mohit Yadav
521619ae32
Correct Imports for Class Converter Factory (#19057) 2024-12-15 01:20:28 +05:30
Karan Hotchandani
497b477be3
remove extra assignees from workflow (#19030) 2024-12-14 09:00:19 -08:00
Sriharsha Chintalapani
cbd2c6ad87
Fix #issue-16041: The persona JSON schema is named Team (#19035) 2024-12-12 19:27:53 -08:00
Keshav Mohta
f4ff43c24c
Fixes #17888 - Implemented Cross Database Lineage (#18831) 2024-12-12 15:13:36 +05:30
Keshav Mohta
cde3a7dd1e
Feature: Cassandra Connector (#18943) 2024-12-12 15:12:55 +05:30
IceS2
c24bc870d0
Parametrizing Flowable configurations and adding a way to handle history cleaning (#18990) 2024-12-12 10:01:39 +01:00
Mayur Singal
6326ae3cb2
MINOR: DB2 Support License (#18928) 2024-12-05 09:54:58 +05:30
Sriharsha Chintalapani
0629750e0a
Minor: fix indexType definition in searchIndex spec (#18912)
* Minor: fix indexType definition in searchIndex spec

* Minor: fix indexType definition in searchIndex spec
2024-12-04 08:50:01 -08:00
Pere Miquel Brull
613fd331e0
MINOR - Clean up configs & add auto classification docs (#18907)
* MINOR - Clean up configs & add auto classification docs

* deprecation notice
2024-12-04 09:32:25 +01:00
Sriharsha Chintalapani
bf00c9c12e
Create a relationship to manage ER based on tableConstraints (#18892)
* Create a relationship to manage ER based on tableConstraints

* Add relationship/remove relationship from entity_relationship table whenever table constriants are updated/added , validate table constraints

* remove findRelatedTables queries

* Add Migrations to add constrait relationship

* remove findRelatedTables code

* Fix postgres migration query

* fix pg migration

* fix test

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-12-03 20:47:34 -08:00
Mayur Singal
9b9509f4b9
MINOR: Mysql Lineage Support Main (#18780)
* MINOR: Mysql Lineage Support Main

* fix test

* fix test

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2024-11-29 20:48:42 +05:30
Siddhant
3453c799f7
FIX: Mask Slack App User & Bot Tokens (#18814)
* fix: mask slack user and bot token.

* refactor: Move the code in SlackApp
2024-11-28 23:59:46 +05:30
mgorsk1
da176767a8
feat: add dbt freshness check test (#18730)
* add dbt freshness check

* docs

* run linting

* add test case param definition

* fix test case param definition

* add config for dbt http, fix linting

* refactor (only create freshness test definition when user executed one)

* fix dbt files class

* fix dbt files class 2

* fix dbt objects class

* fix linting

* fix pylint

* fix linting once and for all

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2024-11-28 18:30:11 +01:00
harshsoni2024
cb33f274fc
Connector: rename microstrategy connector (#18604) 2024-11-28 18:50:42 +05:30
Suman Maharana
9a21e77e15
Added dbt cloud multi projects and jobs filter (#18801)
* Added dbt cloud multi project and jobs filter

* added tests

* change to array type

* updated yaml config

* added migrations
2024-11-28 16:10:34 +05:30
Aniket Katkar
c101c7cf30
Minor: Auto classification UI (#18785)
* Add the auto classification pipeline option in the service ingestion

* Localization changes for other languages

* Improve the logic for getSupportedPipelineTypes function and add unit tests for newly added logic

* Add playwright tests for the auto classification feature

* Improve the getSupportedPipelineTypes function logic to reduce the cognitive complexity

* update md docs

* Add classificationFilterPattern in the UI schema form order

* fix logs from backend for auto classification

* Changes to view the auto classification logs

* Fix the sonar errors

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-11-27 14:51:59 +05:30
IceS2
8cade39bad
MINOR: Increase visibility on flowable process (#18792)
* Add annotation to turn delete_recursive test into the last one to be executed

* Improve Fetching Entities by only fetching the FQN

* Cut back default batchsize to 500

* implement logging for flowable process

* Implement Error Handling Flow

* Remove testing code

* Checking if User is Reviewer when resolving glossary task
2024-11-26 17:04:59 +01:00
Keshav Mohta
10a7f4ea6c
Fixes #15543: Implemented ElasticSearch Index Template Ingestion (#18686) 2024-11-26 14:42:43 +05:30
Kenil Shah
09db201a57
UI: Add option to stop the running application (#18759)
* Add Stop functionality for Applications

* show kill button by default

* Add option to stop the running application

* fix: show stop button for ruuning & activeerror status

* add localisation files

* added unit tests

* update localisation files

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2024-11-26 11:55:47 +05:30
Mayur Singal
40a70357e8
MINOR: Add UDF support for SP - backend (#18767) 2024-11-25 15:27:53 +05:30
IceS2
b3d765dce1
MINOR: Governance Workflows fixes (#18744)
* Add annotation to turn delete_recursive test into the last one to be executed

* Improve Fetching Entities by only fetching the FQN

* Cut back default batchsize to 500
2024-11-24 11:30:56 -08:00
Imri Paran
ee7d043035
[GEN-2109] feat(mongo): added ssl support (#18731)
* feat(mongo): added ssl support

Added SSL support for MongoDB using the SSL manager.

Attached a video demo.

- [Example repository for setting up mongodb with SSL](https://github.com/sushi30/mongodb-docker-ssl-example)
- [MongoDB TLS documentation](https://www.mongodb.com/docs/manual/tutorial/configure-ssl/)

* fixed test_doris.py
2024-11-22 08:54:13 -08:00
Siddhant
702c34c0af
NoSchedule as ScheduleType for Slack App (#18715)
* fix: NoSchedule as scheduleType for Slack App

* fix: remove callback url from the config.

* no render of schedule for noSchedule type

* fix: Migrations and add ScheduleType.NoSchedule

* fix: Migrations

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
2024-11-21 22:29:09 -08:00
Ayush Shah
17ffdf9850
fix: modify fqn to allow quotes with dots (#18719) 2024-11-22 09:33:50 +05:30
Imri Paran
d089ffa2d0
doc(mongodb): mongodb+srv connection scheme (#18712)
- added information about the mongodb+srv connection scheme
- added link to atlas guide for getting the connection string
2024-11-21 18:42:35 +00:00
Siddhant
dad3c9d7f2
refactor: Slack app improvements (#18684) 2024-11-19 19:08:47 +05:30
Pere Miquel Brull
c68a45e7d8
Create new Auto Classification Workflow (#18610) 2024-11-19 08:10:45 +01:00
sonika-shah
853bbc205b
GEN-1983 : Feat - Support Tag Asset Page (#18505)
* Feat :  Support bulk adding tag to Assets

* remove warnings

* add apis for assets remove

* Fix: Add tag page (#18461)

* add tag page which all Assets

* update as per feedbacks

* update as per feedbacks

* add divider in header badge

* remove styling

* update permission and refactoring code

* updated as per comments

* fix sonar cloud issues

* add delete asset functionality

* refactor entityTypeString

* made the top bar fixed to top

* add tests for tag page

* fix check failures issue

* fix tag page check failure

* fix flaky test issue

* add tag page tests

* update the add asset test

* update playwright tests

* add right panel in tag page

* updated as per feedbacks

* remove usage test

* updated as per feedbacks

---------

* Backend: make bulkAssets api async

* Backend: limit bulkAssets api async to Tag Assets page

* Update Tag page Assets API (#18622)

* create branch

* update add asset API

* add websocket on tags page

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>

* add failed case for socket operation

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sweta Agarwalla <105535990+sweta1308@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
2024-11-18 17:57:07 +05:30
Mohit Yadav
66c253ef10
Add Queue Size in Configuration (#18672) 2024-11-18 12:45:08 +01:00
Ayush Shah
6f1df37ba1
Fixes GEN-1260: Add Validators while creating table to escape special characters (#18456) 2024-11-18 15:02:57 +05:30
IceS2
231025df67
MINOR: Improve PeriodicBatchEntityTrigger Workflows performance (#18664)
* Update default configurations

* Refactor how we call the main Workflow in order to increase performance
2024-11-18 08:17:22 +01:00
Sriharsha Chintalapani
88c8fb48f3
Add Edit glossary terms, Edit Tier , Edit Tags as separate permissions (#18331)
* Add EditGlossaryTerms Permission

* Fix #18330: Add EDIT_GLOSSARY_TERM permission and enforce EDIT_TIER permisson

* add edit glossary term permission check in UI

* revert EDIT_GLOSSARY_TERMS operation

* Add EDIT_GLOSSARY_TERMS to common operations

* Add EDIT_TIER to common operations

* add default empty array for tags field, as patch calls can run into issues

* Fix tests

* Fix tests

* added glossary terms

* fix conflicts

* fix permission check for data model

* Add EditGlossaryTerms to DataConsumerPolicy

* Add EditGlossaryTerms,EditTier to DataConsumerPolicy

* fix tests

* Fix migrations for EditTier,EditGlossaryTerms

* add edit tier permission to data consumer

* Fix tests

* fix pytests

* missing test_dbt.py

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-11-15 10:50:15 -08:00
Sriharsha Chintalapani
783ec62e4c
Add Lineage Export Async endpoint using Search (#18553)
* Add Lineage Export Async endpoint using Search

* Lineage api (#18593)

* add api for lineage export

* update API call for lineage async

* use JsonUtils instead of objectMapper to read lineage search response

---------

Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sweta Agarwalla <105535990+sweta1308@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-11-15 09:11:23 -08:00
mgorsk1
3d2dfeb583
feat: use native trino client authentication classes (#16196)
---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2024-11-15 12:54:42 +05:30
Mayur Singal
976ad98874
MINOR: Add missing queueSize in searchIndexingAppConfig (#18643) 2024-11-14 18:10:34 +05:30
Aniket Katkar
bfc12feafc
GEN-1976: Alert improvements (#18448)
* Move the alert components to the components folder from pages folder

* auto generated localization changes and linting changes

* Fix the imports

* Add AlertConfigDetails component

* Add tabs in the alert details page

* auto generated localization changes

* Apply permissions for alert and add owner update functionality

* Change the alert update method from PUT to PATCH

* Add the recent events tab

* auto generated localization changes

* Improve the recent events tab for alert details

* Implement the logic to show alert events count on the alert details page

* auto generated localization and linting changes

* Add the pagination logic for the recent events list on the alert details page

* Fix the logic for previous click in NextPreviousWithOffset

* Localization change for other languages

* Hide pagination when not required

* Add pagination and permissions to API; create eventsRecord API; fix timestamp issue

* fix: remove publisherName from test destination message.

* Fix the recent events tab error

* fix: Resolve PATCH API issue with destination ID creation.

* fix: Delete associated alert metrics upon alert deletion

* feat: Detailed HTTP Response for Destination Test

* Add the test external destination functionality

* localization changes for other languages

* fix: Set status for testing destination in EmailPublisher

* Add advanced configurations for webhook destinations

* Localization changes for other languages

* Fix the destination status not showing
Add the POST and PUT radio button in advanced configs

* Localization changes for other languages

* Update the API to fetch the alert event diagnostic counts

* Improve the AlertConfigDetails component to show the alert configs properly

* fix: Display displayName instead of fullyQualifiedName in templates

* Fix the test destination result not showing the first time

* Fix the test destination status not showing for already existing destinations

* Fix the unit tests

* Fix playwright tests

* refactor: Separate sendMessage and sendTestMessage methods; add statusContext

* fix: Use name or fqn as fallback if displayName is null or empty

* fix: upsert successfulChangeEvent

* eliminate the GovernanceWorkflowChangeEvent option from external destination

* Add code comments

* Fix the unit test

---------

Co-authored-by: Siddhant <86899184+Siddhanttimeline@users.noreply.github.com>
Co-authored-by: Siddhanttimeline <siddhanttimeline9@gmail.com>
2024-11-14 17:39:45 +05:30
Karan Hotchandani
887ffab7e1
MINOR - fix config for workflows (#18628)
* fix config for workflows

* minor spacing

* Fix certification spec

* Fix checkstyle

---------

Co-authored-by: Pablo Takara <pjt1991@gmail.com>
2024-11-13 18:23:12 +01:00
Mohit Yadav
ee327ce0c4
Fix Unlimited Queue Size Issue (#18633) 2024-11-13 22:02:34 +05:30
Mohit Yadav
30ee4fafb5
Update Default Values (#18621) 2024-11-13 13:24:16 +05:30
IceS2
e5441a9969
Initial implementation for Governance Workflows (#17837)
* Initial implementation for Governance Workflows

* Remove file that shouldn't be there

* Initial implementation for the governance workflows

* Refactor JSONSchema for Governance Workflows

* Refactored WorkflowInstanceStatus to keep the stage properly

* Fix Migration for MySQL for the WorkflowInstanceStates change

* Update WorkflowInstanceState to track all the states propertly

* Initial code tro migrate Request type of Tasks

* copy governance schemas into json directory

* update schema names

* Extract WorkflowInstance into its own resource

* Update WorkflowInstance to have the workflow starting variables

* Changed relatedEntity to be an EntityLink

* First iteration on checkEntityAttributes JSONLogic

* Refactored Workflow into TriggerWorkfow and MainWorkflow

* Fixed WorkflowInstance and WorkflowInstanceState

* Fixed WorkflowInstance and WorkflowInstanceState when process is terminated

* Fix UserTask

* json logic changes to workflow

* Fix Patch request. Add exclude filter to EventBasedEntityTrigger

* update query builder

* Fix Migrations

* Fix Migrations

* modify query builder widget

* fix search props

* update widget and use default fields

* Update edges to always have a true/false condition

* made changes to provider for fields

* push progress

* fix grouping of array field

* Remove duplicated migrations

* Fix migrations

* Fix migrations

* Fix Tests and commit some improvements suggestion

* fix unit tests

* Add Certification Workflow

* Fix Tests

* Fix Tests

* fix tests

* add delay in pagination tests

* fix minor config changes for json logic

* change the glossary page

* Remove Table Certification Workflow seed Data

* add json logic and elastic search parsers

* minor schedule interval changes for workflows

* minor fixes to the parsing logic

* certification ui

* fix tests

* fix tests

* add other search fields

* Add Certification to search index

* Update index mapping for certification

* add quick filter for certification

* update locales

* Fix index mapping

* Add missing index updates

* Fix checkstyle

* increase timeout

* skipping user test

* skipping team test

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
2024-11-13 08:47:44 +01:00
Mayur Singal
8e1c839473
MINOR: Summary Card changes require for UI (#18585) 2024-11-13 12:22:47 +05:30
Ayush Shah
6fa03ee66a
Fixes GEN-1994: Remove View Lineage from Metadata Ingestion flow (#18558) 2024-11-13 00:08:55 +05:30