797 Commits

Author SHA1 Message Date
harshsoni2024
01208ef4b4
Connector: add stitch schema (#18452) 2024-11-07 21:08:41 +05:30
sonika-shah
e0d175f78e
GEN-1894 GEN-771 : Improvement - Add displayName field for custom property (#18496) 2024-11-07 16:59:59 +05:30
Chirag Madlani
6fa76ecde9
feat: persona based customisation for glossary and glossary term pages (#18454)
* Initial schemas for UI customization

* feat(ui): persona customization for entity details page
including Glossary & Glossary term detail page too

* WIP glossary term details page customization

* update persona epic

* WIP persona customization

* fix errors and warning

* WIP personalize glossary term

* WIP persona

* WIP glossary terms customization

* WIP glossary terms customization

* update glossary term customization

* remove glossary term reference from generic component

* use more of generic provider

* improved UI UX for customization

* complete glossary term customization
start with glossary customization

* apply backend linting

* support customization for glossary page

* fix issue with new tab on glossary page

* address feedbacks

* fix tests

* fix playwright for customize landing page

* fix tests

* fix tests

* address sonar errors

* fix glossary test

* fix glossary test

* fix failing playwright tests

* address comments

* fix conflict error

* add more widgets

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
2024-11-07 11:13:12 +05:30
Mayur Singal
5660a751e3
GEN-2000: Add Support for PowerBI Report Server (#18513) 2024-11-06 14:55:05 +05:30
Sriharsha Chintalapani
f2c9bc6108
Add LineageSettings to control Lineage Default depth, layers (#18498) 2024-11-04 08:50:43 -08:00
Nicola Coretti
7ebc62dca7
feat: Add support for exasol datasource (#17166)
* Add flake.nix

* Add lockfile for flake

* Update nix environment and document usage

* Add schema for exasol connector

* Add Exasol definitions to databaseService

* Fix error in exasol connector schema

* Add additional connection options/settings to exasol connector

* Add exasol-connector to ui

* Add depdencies for exasol-connector

* Update notes

* Update ingestion code

* Add Basic Documentation for Exasol Connector

* Update flake file

* Add developer notes

* Add python script which can be used as entry point for debugging in ide

* Add config file which can be used for debugging (manual execution)

* Update debug script

* Update developer notes

* Remove old developer notes

* Add .venv to gitignore

* Update dev notes

* Update development notes

* Update ExasolSource

* Establish basic connection to Exasol DB from connector

* Update exasol connector connection settings

* Add service_spec for exasol plugin

* Remove development files

* Remove unused module

* Applied code formatter

* Update exasol dependency constraint(s)

* Add unit test for exasol connection url(s)

* Fixed test expectations for exasol connection url test(s)

* Adjust the test query for the Exasol connection test
2024-10-31 08:11:30 +01:00
Siddhant
f0fc482c46
Fixes #17718 : Notification Debuggability (#18199)
* create EventSubscriptionDiagnosticInfo

* API for failedEvents

* migrations

* add index on source

* list all failedEvents optionally filtered by source

* refactor

* refactor

* getSuccessfullySentChangeEventsForAlert API

* increase defaultValue of limit to 100

* resolve conflicts

* listEvents API with 'failed', 'successful,' and 'unprocessed' query parameters

* fix description

* eventSource as enums

* refactor name.

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-10-30 11:55:51 -07:00
IceS2
57c22b5fbe
Initial JSONSchema specification for Asset Certification (#18392)
* Initial JSONSchema specification for Asset Certification

* Add backend logic to updateCertification

* Add Tests for Asset Certification

* Update certification config and set defaults

* Fix Checkstyle

* Fix Tests

* Fix checkstyle
2024-10-30 11:40:39 +01:00
Teddy
4f8bf05ded
GEN 1931 - Fix entity link accepted chars (#18391)
* fix: entity link accepted characters

* fix: match all characters but `<>|::`

* fix: remove unnecessary negative lookbehind

* fix the test case search not working on add ingestion page in case of special characters in FQN

* style: ran java linting

* fix: failing testst

* style: ran java linting

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-10-29 10:33:08 +01:00
Sriharsha Chintalapani
eeffaee9c4
Update EntityLink.g4, remove entities from EntityLink parser (#18442) 2024-10-29 10:18:49 +05:30
Onkar Ravgan
4a0c8406e9
[ER Diagrams] Add ER diagram APIs and sample data (#18021)
* Add ER diag APIs and sample data

* fix pylint

* formatting fixes2

* fixed es client return

* fixed os client return

* supported TableDetailPage tabs as classBase for supporting collate only tabs

* Added schema Apis

* change the base class to .ts and move the component in the util files

* beautify function arguments

* Added optimizations

* Ingestion changes

* svg dimension change

* supported class base tab in databaseSchema

* supported classBase action button in schema table name column

* added further keys data for constraint modal

* fix sonar issue

* remove old method to override edit action on column and shifted to DisplayNameModal for fields

* supported table right panel component to further extends on collate side

* minor fix around duplicate constraint

* added support to update table constraints and column constraints in the UI

* code optimization and minor fixes

* review comments and multi col fix

* added queryFilter option in NodeSuggestion and tableConstrainst to fetch and use only in service tables

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-10-28 20:26:19 +05:30
Mayur Singal
9d91325af8
Lineage-1: Move view lineage processing to lineage workflow (#18220)
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-28 18:18:22 +05:30
Pere Miquel Brull
7d01373261
MINOR - Renaming MetaPilot to CollateAI (#18393)
* MINOR - Renaming MetaPilot to CollateAI

* MINOR - Renaming MetaPilot to CollateAI

* MINOR - Renaming MetaPilot to CollateAI

* MINOR - Renaming MetaPilot to CollateAI

* fix

* moving limits migrations for 1.5.9
2024-10-28 09:21:15 +01:00
Mayur Singal
ef77535f17
GEN-1664: DI Make Horizontal Axis Configurable (#18134) 2024-10-28 11:11:15 +05:30
harshsoni2024
1a8bba6058
GEN-1911: Quicksight lineage source fix (#18348) 2024-10-24 11:41:37 +05:30
Pere Miquel Brull
1324ad2ceb
MINOR - Metapilot automator config (#18165)
* Prep config

* MINOR - MetaPilot config

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-10-23 12:10:45 +02:00
Sriharsha Chintalapani
9d006885d5
Rbac setting v2 (#18338)
* Add Search RBAC config option in settings

* add null check

* review comments

* resolve

* WIP search rbac page

* enable / disable search rbac switch added

* testing search RBAC

* Remove Domain and Ignore Build RBAC Query for Bots

* Fix Domain Conditions

* Use common code from Search Client

* update page to preferences -> search

* add playwright tests

* add playwright for search RBAC

* update RBACConditionEvaluator

* No results check added

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-10-23 14:53:53 +05:30
Vijay Lakshmanan
4f2ef6fe5c
Fixes #16263: Fixed Mode dashboard ingestion API call (#18355) 2024-10-23 12:03:08 +05:30
Imri Paran
d0fb5137af
[GEN-970] added supportsSystemProfile to connector (#18203)
* added supportsSystemProfile to connector

* reverted unintended changes

* added supportsSystemProfile for bigquery
2024-10-22 07:39:18 +02:00
Mayur Singal
a4d62f6d85
MINOR: Add location path to table entity (#18307) 2024-10-21 10:31:27 +05:30
Sriharsha Chintalapani
9bffbfc3d5
Fix #18007: Disabled Classifications or Tags shouldn't be visible in UI (#18242)
* Fix #18007: Disabled Classifications or Tags shouldn't be visible in UI

* added playwright test for disabled tags should not be visible while search

* replace testing tag to new generated one to avoid flakyness

* added test for checking tags are re-enabling it from disabled state

* fix the playwright test for the wrong column selector

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-10-19 14:18:38 -07:00
Sriharsha Chintalapani
85cc240e1e
Minor: Add an API to list all the properties and custom properties for a given entity type (#18248)
* Minor: Add an API to list all the properties and custom properties for a given entity type

* Minor: Add an API to list all the properties and custom properties for a given entity type

* add custom properties

* add custom properties

* add custom properties
2024-10-19 14:14:34 -07:00
Katarzyna Kałek
141caf4b80
fixed dashboardDataModel enum in spec (#18304)
Co-authored-by: Katarzyna Kałek <kkalek@olx.pl>
2024-10-18 08:28:31 +02:00
Teddy
fc51d7feb4
FIX #17042 - Allow non admin/bot to read profiler global config (#18292)
* fix: custom properties folder name in generation to match expected package name (i.e. customProperties -- uppercase P)

* fix: allow non admin/bot to read profiler global config with the right permission

* style: ran java linting

* fix: custom properties import casing
2024-10-17 09:27:34 +02:00
sonika-shah
2093c24f83
Minor : remove EnumWithDescriptions in basic.json (#18305) 2024-10-17 12:08:39 +05:30
Pere Miquel Brull
7012e73d75
GEN-1166 - Improve Ingestion Workflow Error Summary (#18280)
* GEN-1166 - Improve Ingestion Workflow Error Summary

* fix test

* docs

* comments
2024-10-16 18:15:50 +02:00
Imri Paran
befa1f72e8
MINOR: add supportsDataDiff flag (#18205)
* feat: add supportsDataDiff flag

enables filtering services based on support of the data diff test case

* fixed mysql migration

* fixed conflicts
2024-10-16 11:07:31 +02:00
sonika-shah
62ee79e71e
GEN-1836: Table custom property enhancements (#18239)
* Table custom property enhancements

* ui: remove rowCount

* minor style file changes

* minor change to show no data placeholder

* add icons for property types

* fix type filtering issue

* fix spacing issue for custom properties card

* refactor comments and style file

* fix test

* add playwright test for adding/updating table type property in entity

* add sesrchField for search property by name

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-15 12:41:10 +05:30
sonika-shah
7d736f6195
Feature : Add table-type custom property (#18135)
* Feature : Add table-type custom property

* ui: remove enumWithDescriptions property

* Refactor AddCustomProperty component to remove unused EnumWithDescriptionsConfig import and property

* ui: add support for creating and updating the table-type property

* add rowCount field in table-type and clean logic of validation

* add rowCount field in table-type and clean logic of validation

* add rowCount field in table-type and clean logic of validation

* ui: add support for adding value to table type property part 1

* use rowCount from the API

* default row-count to 10

* fix tooltip to read rowcount from the props

* fix empty row issue

* rename from table-type to table-cp

* Refactor table type property to handle empty rows and display in modal

* Refactor table type property to handle empty rows and display in modal

* use locales for messages

* minor change

* ui: use column layout for rendering custom properties cards

* remove enumwithDescriptions from backend and add tests for table type

* fix unit test

* fix playwright

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-10 13:43:19 -07:00
Sachin Chaurasiya
457f3d919a
GEN-1322: API Entity - Remove Beta (#17967)
* GEN-1322: API Entity - Remove Beta

* minor: add doc for the metadata pipeline

* api service refactor

* api service refactor backend changes

* add apiconnection in test service connection

* pytest fix

* fix java file formatting

* Fix casing of REST in ApiServiceRest.spec.ts

* Refactor REST to Rest in API classes

* minor change

* minor change

* minor change

* fix cashing for API to Api

* add playwright test for api service ingestion

* fix: playwright test

---------

Co-authored-by: harshsoni2024 <harshsoni2024@gmail.com>
2024-10-08 14:39:55 +05:30
Ayush Shah
221327afd4
Refactor ServiceConnectionDetails component to handle specific Airflow connection type (#18100) 2024-10-08 11:29:45 +05:30
Pere Miquel Brull
e81efc8d98
MINOR - Add App Limits (#18072)
* App Limits - Prep extension

* App Limits - Prep extension

* App Limits - Allow to search by name

* App Limits - Allow to search by name

* fix postgres sql

* comments
2024-10-03 11:00:29 +02:00
Siddhant
ed052596e8
[IMPROVEMENT] Add Support for Custom Headers and HTTP Methods in Generic Webhook (#18056)
* Add support for custom headers and JSON payload configuration in webhook

* move testDestination API to EventSubscriptionResource

* support POST and PUT http ops for generic webhook requests

* fix name

* remove json field from webhook

* fix getEmailTestMessage

* refactor exception handling
2024-10-02 09:20:12 +05:30
Mohit Yadav
f1ae49e1a0
[Search] Indexing Fixes (#18048)
* Fix OpenSearch Content Size Issue
entity content is too long [204857600] for the configured buffer limit [104857600]

* Change Type to Long

* Add Payload Size to take dynamic entry

* Migrations for 1.5.6

* Mark Stale Entries Stopped

* Format checkstyle

* Fix failure

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-09-30 23:39:27 +05:30
Teddy
9552886f29
GEN 1211 - Added TTResp and TTReso metrics (#18033)
* fix import issue

* feat: compute test case resolution metrics

* feat: added  index key to resolution index

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-09-30 14:21:41 +02:00
sonika-shah
1b029d2bf2
Add new ‘enumWithDescriptions’ Custom Property to allow adding Enum Keys with Description (#17777)
* Add new ‘metaEnum’ Custom Property to allow adding Enum Keys with Description

* replace JsonNodeFactory method with JsonUtils

* rename property from metaEnum to enumWithDescriptions, and other method optimizations

* ui: add support for creating enumWithDescription property

* minor locale changes

* ui: add edit support for created enumWithDescription property

* Refactor enum description field layout in AddCustomProperty and EditCustomPropertyModal

* add support for adding values to enumWithDescription custom property type

* Refactor custom property input IDs in AddCustomProperty and EditCustomPropertyModal components

* Refactor custom property table rendering logic and UI components

* Refactor custom property table rendering logic and UI components

* Refactor custom property table rendering logic and UI components

* add basic card layout

* Refactor CustomPropertyTable component to improve UI and functionality

* update playwright test part 1

* Refactor PropertyValue component to conditionally render right panel styles

* fix: entity reference property update

* Refactor CustomPropertyTable component to conditionally render right panel styles

* fix: flaky test

* Refactor CustomPropertyTable test to use updated test IDs and remove unnecessary code

* fix flaky test

* improve the playwright test

* add more test

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-09-29 00:45:46 +05:30
Pere Miquel Brull
d26449576a
GEN-1234 - Clean up suggestions when a user is deleted (#17988)
* GEN-1234 - Clean up suggestions when a user is deleted

* add method

* add method

* fix postgres query
2024-09-26 16:22:36 +02:00
Suman Maharana
37b6dc8290
Add Sigma Dashboard Connector (#17855)
* Add Sigma Dashboard Connector

* changed to id instead of name in dashboard entity

* Address Comments

* addressed comments

* Added Docs

* yaml file changes

* fix ui changes
2024-09-26 16:29:35 +05:30
Mohit Yadav
afd7887bd8
[Issue-16487] Add Stop for Search Indexing (#17914)
* Add Stop for Search Indexing

* Update Logs and Message

* test for stop func

* remove test
2024-09-25 18:29:32 +05:30
Sachin Chaurasiya
62ced79671
minor: add supportsMetadataExtraction property in restConnection.json (#17974) 2024-09-24 20:05:57 +05:30
Ayush Shah
8087f48e15
Fixes #17869: Add validations in place for salesforce connection (#17870) 2024-09-20 16:33:38 +05:30
Teddy
33c50efdbf
GEN-1192 - Move Test Case to its Own Resource (#17862)
* feat: indexed test case results

* feat: added indexation logic for test case results

* style: ran java linting

* fix: IDE warnigns

* chore: added test case results migration

* style: ran java linting

* fix: postgres migration column json ref

* empty commit to trigger queued

* chore: extracted test case results to its own resource

* chore: fix failing tests

* chore: move testCaseResult state from testSuite and testCase to dynamic field fetched from test case results search index

* chore: clean up test case repository

* style: ran java linting

* chore: removed testCaseResultSummary and testCaseResult state from db

* fix: test failures

* chore: fix index mapping type for result value

* chore: fix test failure
2024-09-18 11:58:59 +02:00
Sriharsha Chintalapani
c34545adeb
Issue-15768: Support Metric Entity (#17680)
* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Issue-15768: Support Metric Entity

* Fix tests

* Fix tests

* Fix tests

* Minor: Fix tests

* ui: add metricsAPI rest utils

* ui: metric list page part 1

* feat: Add metric translations for multiple languages

* chore: Add "metric" field to SearchIndexingApplication schema

* ui: add create metric page

* ui: metric details page patch 1

* ui: add custom property and lineage support for metric entity

* ui: add expression component

* ui: add metric summary component

* chore: Update tab labels in MetricDetails and MetricVersion components

* ui: show other info like metric type, granularity, etc

* feat: Add support for metric entity in search dropdown

* feat: Rename custom property to Metric in MetricEntity.md

* feat: Add OwnerLabel component to MetricListPage

* Fix expression field in Metric

* chore: update expression to metricExpression

* ui: add metric header component with edit option

* Add metric to SearchIndexApp

* chore: Update expression to metricExpression

* ui: allow metric expression edit

* ui: update metric icon

* minor improvements

* Fix lineage indexing for Metric

* Update GlobalSettingsClassBase.ts to use MetricIcon for metrics in the global settings menu

* Fix error handling in MetricListPage component

* add related metrics

* minor improvements

* Fix relatedTerms patch

* Fix relatedTerms validation

* Add Boolean for deleted

* filter active entity from related metric list

* playwrite e2e part 1

* Refactor MetricSummary component to include RelatedMetrics in the summary panel

* test: add playwright test for metric special cases

* Add 'Metrics' to Explore Tree

* test: add e2e for add metric page

* test: add test for metric listing page content

* Add Boolean for deleted, remove deleted from suggests

* Refactor LineageProvider to handle deleted flag properly

* add playwright for metric listing

* fix test

* Add colored metric icon and update its usage in GlobalSettingsClassBase

* Fixed py_test test_ometa_endpoint for metric

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: SumanMaharana <sumanmaharana786@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-09-16 11:36:31 +05:30
k.nakagaki
3d8e30142c
Fixes 8428: make it possible to choice a sampling method type when we create profile ingestion for the Snowflake (#17831)
* Add test for existing code

* Add sampling method at ingestion.

* add samplingMethodType into UI

* modify init method to use new parameter.

* create descriptions

* execute isort

* fix an unintended change.

* apply py_format

* close section

* specify  init arguments

* fix bug

* apply py_format

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2024-09-15 21:51:17 +02:00
harshsoni2024
4f57f4e710
GEN-309: OpenAPI Connector (#17754)
* configure api service metadata

* add rest api service

* fix test con. pyformat changes

* add models, fix test con.

* improve test con.

* add docs, side doc

* fix model data parse, url error fix

* add tests

* fix pytest errors

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-09-11 13:36:53 +05:30
Ayush Shah
8decd2338f
MINOR: Add Matillion test connection and schema for pipeline service (#17710) 2024-09-10 17:41:28 +05:30
Teddy
1f72a3c9c6
GEN-1213 - logic to index test case result (#17706)
* feat: indexed test case results

* feat: added indexation logic for test case results

* style: ran java linting

* fix: IDE warnigns

* chore: added test case results migration

* style: ran java linting

* fix: postgres migration column json ref

* empty commit to trigger queued

* fix: added index on id field
2024-09-09 07:58:45 +02:00
harshsoni2024
a89c4e9f40
Connector: VertexAI UI, docs (#17649) 2024-09-04 11:45:33 +05:30
sonika-shah
6d32257765
Fixes #16498 : Support custom properties for Data Products (#17641) 2024-08-31 09:43:37 +05:30
Pere Miquel Brull
bcb29b46da
MINOR - Implement SAP Hana Lineage (#17615)
* MINOR - SAP Hana Lineage

* skeleton

* parser

* lineage

* manage formulas

* add cvs

* add cvs

* better typing

* enum

* handle cvs

* saphana docs
2024-08-30 07:42:43 +02:00