2141 Commits

Author SHA1 Message Date
Siddhant
f0fc482c46
Fixes #17718 : Notification Debuggability (#18199)
* create EventSubscriptionDiagnosticInfo

* API for failedEvents

* migrations

* add index on source

* list all failedEvents optionally filtered by source

* refactor

* refactor

* getSuccessfullySentChangeEventsForAlert API

* increase defaultValue of limit to 100

* resolve conflicts

* listEvents API with 'failed', 'successful,' and 'unprocessed' query parameters

* fix description

* eventSource as enums

* refactor name.

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-10-30 11:55:51 -07:00
sonika-shah
b42e211d29
Feat - Add import/export custom property for other entities - databaseService, database, databaseSchema level (#18385)
* Feat - Add import/export custom property for other entities - databaseService, database, databaseSchema level

* Fix java checkstyle

* Fix dbService level export

* minor: improve the inline edit of custom property

* minor: improve the number type icon

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-30 16:32:47 +05:30
IceS2
57c22b5fbe
Initial JSONSchema specification for Asset Certification (#18392)
* Initial JSONSchema specification for Asset Certification

* Add backend logic to updateCertification

* Add Tests for Asset Certification

* Update certification config and set defaults

* Fix Checkstyle

* Fix Tests

* Fix checkstyle
2024-10-30 11:40:39 +01:00
Sriharsha Chintalapani
572822e36a
Minor: add versions to maven build plugins (#18443)
Co-authored-by: Suresh Srinivas <srini30005@gmail.com>
2024-10-29 18:54:50 -07:00
Teddy
4f8bf05ded
GEN 1931 - Fix entity link accepted chars (#18391)
* fix: entity link accepted characters

* fix: match all characters but `<>|::`

* fix: remove unnecessary negative lookbehind

* fix the test case search not working on add ingestion page in case of special characters in FQN

* style: ran java linting

* fix: failing testst

* style: ran java linting

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-10-29 10:33:08 +01:00
Sriharsha Chintalapani
96c3364de7
Minor: Enable Azure DatabaseTokenProvider if there is azure=true in DB_QUERYPARAMS (#18451) 2024-10-29 10:01:57 +01:00
Sriharsha Chintalapani
0114007a22
Parallelize the search indexing process (#18445)
* Parallelize the indexing process

* Parallelize indexing
2024-10-28 23:10:43 -07:00
sonika-shah
f7e09487b5
Fix - Consistent case handling for explore page filters (#18418) 2024-10-28 15:25:37 -07:00
Suresh Srinivas
00c38323bd
Update dependencies for release 1.6 (#18444) 2024-10-28 14:52:55 -07:00
Onkar Ravgan
4a0c8406e9
[ER Diagrams] Add ER diagram APIs and sample data (#18021)
* Add ER diag APIs and sample data

* fix pylint

* formatting fixes2

* fixed es client return

* fixed os client return

* supported TableDetailPage tabs as classBase for supporting collate only tabs

* Added schema Apis

* change the base class to .ts and move the component in the util files

* beautify function arguments

* Added optimizations

* Ingestion changes

* svg dimension change

* supported class base tab in databaseSchema

* supported classBase action button in schema table name column

* added further keys data for constraint modal

* fix sonar issue

* remove old method to override edit action on column and shifted to DisplayNameModal for fields

* supported table right panel component to further extends on collate side

* minor fix around duplicate constraint

* added support to update table constraints and column constraints in the UI

* code optimization and minor fixes

* review comments and multi col fix

* added queryFilter option in NodeSuggestion and tableConstrainst to fetch and use only in service tables

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-10-28 20:26:19 +05:30
Mayur Singal
9d91325af8
Lineage-1: Move view lineage processing to lineage workflow (#18220)
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-28 18:18:22 +05:30
Pere Miquel Brull
de02cee7b1
MINOR - None schedules still need Ingestion Pipelines (#18425) 2024-10-28 12:44:21 +01:00
Mohit Yadav
35d1053dc0
Add Cache-Control and Pragma Header (#18406)
* Add Cache Control and Pragma to Asset Servlet

* Add on null Or Empty
2024-10-28 15:42:37 +05:30
Pere Miquel Brull
7d01373261
MINOR - Renaming MetaPilot to CollateAI (#18393)
* MINOR - Renaming MetaPilot to CollateAI

* MINOR - Renaming MetaPilot to CollateAI

* MINOR - Renaming MetaPilot to CollateAI

* MINOR - Renaming MetaPilot to CollateAI

* fix

* moving limits migrations for 1.5.9
2024-10-28 09:21:15 +01:00
Mayur Singal
ef77535f17
GEN-1664: DI Make Horizontal Axis Configurable (#18134) 2024-10-28 11:11:15 +05:30
Sriharsha Chintalapani
c1a3206538
Add bulk reads for export CSVs (#18412)
* initial draft

* Add bulk reads for export CSVs

* get parentRef and glossaryRef in bulk read

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-10-27 08:56:43 -07:00
Sriharsha Chintalapani
e07bcfa2f9
Minor: Fix custom properties API and add cache (#18401) 2024-10-24 22:24:56 -07:00
Sriharsha Chintalapani
d8f5398efb
Add Azure Token Base Authentication (#18387)
* Add azure token based auth

* Add Azure Token Base Authentication

* Update azure-auth.md

* Update azure-auth.md

* feat: Add `azure-identity-extensions` library for passwordless database connection

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Akash-Jain <15995028+akash-jain-10@users.noreply.github.com>
2024-10-24 11:29:46 +02:00
Teddy
cd63a218c7
fix: added test case domain, tags and owners field to search incident index (#18380) 2024-10-24 10:46:08 +02:00
Mayur Singal
913072868a
MINOR: Not Include Soft Deleted Assets in DI (#18383) 2024-10-24 11:12:50 +05:30
Mayur Singal
d145d29e9a
MINOR: Fix web analytic activity being reset (#18344) 2024-10-23 17:59:26 +05:30
Sriharsha Chintalapani
9d006885d5
Rbac setting v2 (#18338)
* Add Search RBAC config option in settings

* add null check

* review comments

* resolve

* WIP search rbac page

* enable / disable search rbac switch added

* testing search RBAC

* Remove Domain and Ignore Build RBAC Query for Bots

* Fix Domain Conditions

* Use common code from Search Client

* update page to preferences -> search

* add playwright tests

* add playwright for search RBAC

* update RBACConditionEvaluator

* No results check added

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-10-23 14:53:53 +05:30
Mayur Singal
fa64a8d0fc
MINOR: Fix LocationPath Index (#18364) 2024-10-23 10:59:08 +05:30
sonika-shah
74cfb9d14f
Fix #18334 - Fix issues in zh language search index mapping (#18335) 2024-10-23 08:27:36 +05:30
Sriharsha Chintalapani
9bd93c15ab
Minor: Add ViewAll policy to enable default OrganizationPolicy to allow users to view metadata (#18353)
* Minor: Add ViewAll policy to enable default OrganizationPolicy to allow uses to view metadata

* Minor: Add ViewAll policy to enable default OrganizationPolicy to allow uses to view metadata
2024-10-22 11:51:13 +05:30
Pere Miquel Brull
93b4c66704
GEN-927 - Add bot default roles (#18256)
* GEN-927 - Add bot default roles

* GEN-927 - Add bot default roles
2024-10-21 09:17:45 +02:00
Mayur Singal
a4d62f6d85
MINOR: Add location path to table entity (#18307) 2024-10-21 10:31:27 +05:30
Sriharsha Chintalapani
9bffbfc3d5
Fix #18007: Disabled Classifications or Tags shouldn't be visible in UI (#18242)
* Fix #18007: Disabled Classifications or Tags shouldn't be visible in UI

* added playwright test for disabled tags should not be visible while search

* replace testing tag to new generated one to avoid flakyness

* added test for checking tags are re-enabling it from disabled state

* fix the playwright test for the wrong column selector

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-10-19 14:18:38 -07:00
Sriharsha Chintalapani
85cc240e1e
Minor: Add an API to list all the properties and custom properties for a given entity type (#18248)
* Minor: Add an API to list all the properties and custom properties for a given entity type

* Minor: Add an API to list all the properties and custom properties for a given entity type

* add custom properties

* add custom properties

* add custom properties
2024-10-19 14:14:34 -07:00
Sriharsha Chintalapani
868231e051
Fix Search RBAC, avoid processing Operations other than ViewAll, ViewBasic, ALL (#18328) 2024-10-19 14:13:56 -07:00
Teddy
781989e5bc
MINOR - live index on test suite creation (#18317)
* fix: live index on test suite creation

* fix: make live indexing use entityInterface
2024-10-18 12:07:11 +02:00
Sriharsha Chintalapani
c78cda6757
Minor: Fix rbac for multiple rules (#18057)
* Minor: Search RBAC, fix condition evaluation for single policy multiple rules

* Minor: Search RBAC, fix condition evaluation for single policy multiple rules

* add test for complex policies

* Add viewAll policy to organization

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-10-17 19:15:06 -07:00
Ethan
e708a3242e
feat: update version (#18259) 2024-10-17 16:18:37 -07:00
Teddy
ff95f84aa9
MINOR: added testSuite field back to lineage response (#18313)
* fix: added testSuite field back to lineage response

* style: fix style
2024-10-17 08:59:10 -07:00
Teddy
d20ee5cc8a
MINOR - added test case results to search reindex (#18277)
* feat: added test case results to search reindex

* fix: failing typescript test case
2024-10-17 11:50:46 +02:00
Teddy
28727b1c16
fix: remove test case result index from all index (#18295) 2024-10-17 09:28:14 +02:00
Teddy
fc51d7feb4
FIX #17042 - Allow non admin/bot to read profiler global config (#18292)
* fix: custom properties folder name in generation to match expected package name (i.e. customProperties -- uppercase P)

* fix: allow non admin/bot to read profiler global config with the right permission

* style: ran java linting

* fix: custom properties import casing
2024-10-17 09:27:34 +02:00
sonika-shah
b175f6cb7e
Minor : increase MAX_AGGREGATE_SIZE in search (#18299) 2024-10-16 13:43:22 -07:00
Siddhant
42eea35aff
correct negative values of the changeCount. (#18279) 2024-10-16 17:51:00 +00:00
Sriharsha Chintalapani
6ffce0f4a5
Minor: Add an option to execute data migrations in ext migrations (#18267)
* Minor: Add an option to execute data migrations in ext migrations

* Minor: Add an option to execute data migrations in ext migrations
2024-10-16 08:56:29 -07:00
sonika-shah
30497528bc
feat: add Import/export support for table type custom property in glossary (#18262)
* import/export support for table type custom property

* remove logs

* Add backend tests

* ui: add custom type property parsing

* print rows in the order of columns defined in config

* ui: handle special chars pasring for custom property import/export

* ui:improve table type property parsing

* ui:fix sql input styling

* ui: improve the parsing for table type property

* ui: fix parsing for markdown, string and sqlQuery properties

* ui: fix parsing for table type property

* ui: fix unit test

* fix backend tests

* remove unwanted comments

* ui: add playwright test for table type import/export part1

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-16 15:44:38 +05:30
sonika-shah
f366827208
GEN-1662 : Fix - on dashboard soft delete, chart should not be visible (#18283) 2024-10-15 21:34:58 -07:00
sonika-shah
831cccf71e
Minor : Fix exception in search due to exception in database.displayName and databaseSchema.aggregation (#18270)
* Minor : Update database.displayName in search indexes from text to keyword

* Update for databaseSchema.displayName and serviceType in search indexes from text to keyword

* Revert "Update for databaseSchema.displayName and serviceType in search indexes from text to keyword"

This reverts commit 16408d81a234b505cfe794dd983f282612c10c9f.

Revert commit changes - 1

* Revert "Minor : Update database.displayName in search indexes from text to keyword"

This reverts commit 2264563f726df07db85ddfea09826e5cfe1fd076.

* print rows in the order of columns defined in config
2024-10-16 08:23:41 +05:30
Teddy
c6586d338d
fix: add domain params for test suite search list filter (#18263) 2024-10-15 20:45:19 +02:00
Imri Paran
be82086e25
MINOR: add column case sensitivity parameter (#18115)
* fix(data-quality): table diff

- added handling for case-insensitive columns
- added handling for different numeric types (int/float/Decimal)
- added handling of boolean test case parameters

* add migrations for table diff

* add migrations for table diff

* removed cross type diff for now. it appears to be flaky

* fixed migrations

* use casefold() instead of lower()

* - implemented utils.get_test_case_param_value
- fixed params for case sensitive column

* handle bool test case parameters

* format

* testing

* format

* list -> List

* list -> List

* - change caseSensitiveColumns default to fase
- added migration to stay backward compatible

* - removed migration files
- updated logging message for table diff migration

* changed bool test case parameters default to always be false

* format

* docs: data diff

- added the caseSensitiveColumns parameter

requires: https://github.com/open-metadata/OpenMetadata/pull/18115

* fixed test_get_bool_test_case_param
2024-10-15 16:29:43 +02:00
sonika-shah
62ee79e71e
GEN-1836: Table custom property enhancements (#18239)
* Table custom property enhancements

* ui: remove rowCount

* minor style file changes

* minor change to show no data placeholder

* add icons for property types

* fix type filtering issue

* fix spacing issue for custom properties card

* refactor comments and style file

* fix test

* add playwright test for adding/updating table type property in entity

* add sesrchField for search property by name

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-10-15 12:41:10 +05:30
Pere Miquel Brull
9496da1997
MINOR - Improve exception messages for JWT Filter (#18231)
* MINOR - Improve exception messages for JWT Filter

* MINOR - Improve exception messages for JWT Filter
2024-10-14 10:22:11 +02:00
Sriharsha Chintalapani
da6ce5c63b
Minor: Add deleteEntityByFQNPrefix (#18243) 2024-10-13 09:22:22 -07:00
Siddhant
07b039b8b8
Migrations fixes for #17967 (#18232)
* fix migrations.

* fix logging
2024-10-11 13:51:12 +00:00
Imri Paran
4d775cff0b
GEN-1793: fix app config from resource (#18228)
* fix: app config from resource

* removed useless logging
2024-10-11 10:59:43 +02:00