393 Commits

Author SHA1 Message Date
sonika-shah
0bf7577175
Fixes #16788 : Fix feed and count api query performance (#16993)
* Fixes #16788 : Fix feed and count api query performance

* Fixes #16788 : Fix feed and count api query performance

* Fixes #16788 : Fix feed and count api query performance

* postgres fixes

* rename thread_entity indexes
2024-07-11 14:18:33 +05:30
Pere Miquel Brull
27f4d9799e
REL #460-CL - Metadata Actions Create Permissions (#16955)
* REL #460-CL - Metadata Actions Create Permissions

* format
2024-07-09 11:13:57 +02:00
Pere Miquel Brull
97a733b704
FIX #16231 - Ingestion Pipeline w/ Bot Assignment (#16941)
* FIX #16231 - Ingestion Pipeline w/ Bot Assignment

* format
2024-07-08 11:19:16 +02:00
Sriharsha Chintalapani
74573ee4fa
Issue #10933: New Entity: API Services (#16878)
* API Service Schemas

* Create API Service Schemas

* Create API Service Schemas

* Add API Collection APIs

* Add API Endpoint APIs

* Fix #10933: New Entity: API Services

* Fix Postgres schema creation for API entities

* domain ref

* fix RESTConnection.json

* Fix openAPI reserved keyword

* fix ref

* Fix for Java Heap Space Issue

* fix APISchema spec

* fix APISchema spec

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-07-05 06:35:32 -07:00
sonika-shah
be243c5cea
Fix #16788: Optimize feed query performance issues introduced in 1.4.2 (#16862) 2024-07-01 19:58:47 -07:00
IceS2
927bf56738
Moved Delta Migrations to schemaChanges.sql (#16789) 2024-06-28 15:40:41 +00:00
Mohit Yadav
b05d29ccee
[DuplicateLogin] User name issue for loggedInUserApi (#16762)
* User Email Based lookup for loggedInUser

* Add Unique on name

* support email only login

* fix cypress

* fix playwright

* reset user for 404
inject enableSelfSignup for auth config

* Add Self Signup Flag

* Add error messages

* Better check

* Change to Custom Exception Message

* Update message as suggested

* fix claim issue for confidential type

* fix sonar

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
2024-06-24 18:21:32 +05:30
IceS2
f0049853ec
FIXES 14885: Initial deltalake implementation for s3 (#16665)
* Initial deltalake implementation for s3

* Fix styles

* Fix test_amundsen

* Fix UnitTests

* Fix Checkstyle

* Fix integration tests due to datalake client refactor

* Fix unit tests

* Fix tests

* Fix Integration DeltaLake Storage test

* Skip delta storage integration test for python 3.8

* DeltaLake JSONSchema changes migrations

* Update import name

* Add some comments based on sonarcloud suggestions

* Update DeltaLake documentation

* Resolve some comments
2024-06-20 12:08:21 +05:30
Huanjie Guo
1175cf9dec
bugfix: change double quote to single quote (#15888)
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-06-17 10:51:38 +05:30
Mohit Yadav
3db41f08e2
Fix Test Suite Filter (#16615)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-06-12 10:40:05 +05:30
Mohit Yadav
77dfe1f6af
Fix Postgres Application listing (#16600)
* Fix Postgres Application listing

* Fix Listing
2024-06-11 13:18:50 +05:30
Teddy
ff00175fb1
MINOR - Data Quality Preparatory Work (#16523)
* chore: added min/max bound and dynamic support flag

* chore: added migration for dynamic attribute

* style: fix pythin checkstyle
2024-06-10 11:03:55 +02:00
Sriharsha Chintalapani
edcac89389
Add appType as part of schema in ingestion pipeline (#16519) 2024-06-03 21:42:40 -07:00
Teddy
451d73593e
[MINOR] partition migration issue with redshift servics (#16452)
* fix: partition migration issue with redshift servics

* chore: typo in sql comment
2024-05-29 10:49:27 +02:00
Imri Paran
a4c516d2c7
Fixes 16305: Added Test Case for Matching Enum (#16362)
* Added Test Case for Matching Enum

1. Implemented the test case using the `matchEnum` parameter.
2. Added integration tests.
3. Added migrations.

* fix tests

* fixed tests

* format

* fixed tests

* clear search cache before running ingestion

* format

* changed scopt of aws fixture

* moved migrations to 1.5.0
2024-05-28 09:30:30 +02:00
Ayush Shah
9416a7ac5f
Fix Migrations: Add postgres migrations (#16403) 2024-05-24 14:31:41 +05:30
Pere Miquel Brull
c33d1bbb18
FIX - Query checksum & fix update (#16392)
* FIX - Query checksum & fix update

* FIX - Query checksum & fix update

* FIX - Query checksum & fix update

* FIX - Query checksum & fix update

* FIX - Query checksum & fix update
2024-05-22 23:49:42 +05:30
Ashish Gupta
a5e30a0cdd
Activity feed card fixes (#16166)
* Activity feed card ui fixes

* Fix feed card for test suite

* fix testCase and testSuite feed card title and links

* Fix the profile picture not loading in the feeds

* fix the custom property and bot icon

* fix the deleted feed card by adding opacity on it

* Fix feed card hover effect
Add edit and delete button for posts

* - Fix Activityt feed event

* - Ignore workflow

* - Ignore pipelienStatus as well

* fix test case card feed

* localization keys

* sonar fix

* changes made as per comments and fix sonar issue

* Update schemaChanges.sql

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-05-10 11:04:17 +05:30
Mohit Yadav
f606212484
- Fix Search Indexing issues in Ops (#16194)
- BatchSize and recreate updated
- added conditional Init of resources
2024-05-09 17:56:38 +05:30
Mohit Yadav
b3e0470882
Migration Fixes (#16131)
* - Migration Fixes

* - Migration Fixes

* - Typo fix

* - Make warn to error

* - Update Migration to show Query Status

* - Fix Query in POstgres

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-05-07 22:07:25 +05:30
Mohit Yadav
0769d71ee7
Fixes Test Suite Reference in Table Schema (#16129)
* Fixes Test Suite Reference in Table Schema

* fix: fix test suite to interact with entity reference

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2024-05-06 19:03:23 +05:30
Suman Maharana
488078da8a
Add DDL query ingest (#15860) 2024-05-06 18:03:50 +05:30
sonika-shah
c6afcce52d
update SettingsCache and fix migration issues (#16052)
* update SettingsCache and fix migration issues

* java checkstyle change
2024-04-29 09:26:09 +05:30
sonika-shah
a9773b9527
Add predefined color palette for UI styling assets (#15982)
* Add predefined color palette for UI styling assets - backend

* Update UI components to use customLogoConfig in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.tsx

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.test.tsx

* Remove existing customLogoConfiguration

* Refactor App.tsx and ThemeUtils.ts files

* Update AppearanceConfigSettingsPage.tsx with new imports and default theme values

* Update AppearanceConfigSettingsPage.tsx with new imports and reset functionality

* add cypress test

* Fix color update issue in CustomThemeConfig.spec.ts

* Remove existing customLogoConfiguration -migration changes

* Fix theme configuration initialization in useApplicationStore.ts

* Update CustomThemeConfig.spec.ts to check if the color is updated

* update existing customLogoConfiguration with customUiThemePreference -migration changes

* Update CustomThemeConfig.spec.ts to fix color update check

* Add unit tests for ThemeUtils

* move validateJsonSchema method to JsonUtils

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-25 22:43:02 +05:30
Imri Paran
0a1018648c
Fixes #15566: add dynamodb row count (#15204)
* feat(nosql-profiler): row count

1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.

* added profiler support for mongodb connection

* doc

* use int_admin_ometa in test setup

* - fixed linting issue in gx
- removed unused inheritance

* moved the nosql function into the metric class

* feat(profiler): add dynamodb row count

* feat(profiler): add dynamodb row count

* formatting

* fixed import

* format

* dded dynamodb row count

* format

* removed unused factory file

* removed "validate"

* migrations

* removed validations

* format

* linting

* fixed: test_amundsen.py

* Update schemaChanges.sql
2024-04-22 09:14:52 +02:00
Ayush Shah
d5b1465406
Fixes #14113 - Allow SSL file uploads (#15828) 2024-04-19 11:38:27 +05:30
Mohit Yadav
d1d964867d
Move migration for apps to 1.3.3 all together (#15944) 2024-04-18 14:26:05 +05:30
Mohit Yadav
dfe6204b55
Remove SQls from 1.3.2 (#15917) 2024-04-16 18:51:03 +05:30
Teddy
84469e643c
FIX #13475 - Add Parameter Validation for Min/Max tests (#15849)
* feat: add validation field to test case definition parameters

* feat: added parameter validation logic

* fix: set ES testcontainer startup attemp to 3

* style: ran java linting

* fix: added waitForEsAsyncOp in post_trigger_app_200 test
2024-04-15 07:52:43 -07:00
Pere Miquel Brull
6ed319c2dc
MINOR - delete mysql ES pipelines (#15868) 2024-04-11 16:37:49 +02:00
Mohit Yadav
ed7fa24c39
Fix Migration SQL , not executing from Handle (#15873)
* Fix Migration SQL , not executing from Handle

* fix
2024-04-11 19:41:50 +05:30
harshsoni2024
eff5394504
fix qlik db migration changes (#15852) 2024-04-11 15:48:32 +05:30
Mohit Yadav
3dc4a4f3d1
Fix SQLs (#15859)
* Fix SQLs

* Update SQls

* Update bootstrap/sql/migrations/native/1.3.3/postgres/schemaChanges.sql

* Update bootstrap/sql/migrations/native/1.3.3/mysql/schemaChanges.sql

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-04-11 15:13:16 +05:30
Imri Paran
740480fe28
MINOR: change systemApp to system (#15757) 2024-04-01 18:50:43 +05:30
harshsoni2024
884029f031
Fix #14089: Add QlikCloud connector support (#15610) 2024-03-28 20:15:22 +05:30
Pere Miquel Brull
f28f196c34
MINOR - Move app migrations to 1.3.2 (#15690) 2024-03-26 08:43:15 +01:00
Sriharsha Chintalapani
8377128c93
Application cannot be installed with None as schedule type (#15524)
* Application cannot be installed with None as schedule type

* MINOR: Add / Fix GCS and ADLS - docs, bugs (#15502)

Add GCS and ADLS docs

* Add stack trace while throwing an error to debug (#15522)

* Fix #15533: Fix name & display name for kafka json schema parser (#15534)

* MINOR: postgres add ssl options in yaml (#15538)

* Small change to fix the Glossary TErm Tasks not fetching Owner automatically (#15535)

* remove the DataInsightAlert spec as the alerts test are covered in ObservabilityAlerts spec (#15531)

* MINOR: fix help dropdown item font and icon sizes (#15511)

* fix help dropdown item font and icon sizes

* added unit test for navbarUtils

* fix sonar failure

* changes as per comments

* minor fixes

* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue (#15528)

* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue

* fixed failing unit test

* fixed unit test for Data quality test

* reverting e2eLabeler changes

* chore(ui): separate routes as per categories (#15512)

* chore(ui): seprate routes as per categories

* add tests

* fix test

* domain path fix

* [MINOR] GX logging hierarchy (#15542)

* fix: GX module logging hierarchy

* style: ran python linting

* - Use Entity Type (#15546)

* removed docs 1.4.x (#15550)

* Suggestions Alert new design (#15532)

* update ux of suggestions alert

* locales

* minor changes

* fix descriptions

* minor css fixes

* Fix #11868: Duplicated queries cannot be created (#15519)

* Fix #11868: Duplicate query should throw an error of entityExists

* Fix #11868: Duplicate query should throw an error of entityExists

* fix test

* fix test

* Fix uniquee constraint for checksum in Postgres

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* added OpenMetadataApplication.getDao (#15549)

* Add the entityUrl to the header for ThreadMessages (#15552)

* feat: AppResource (#15555)

forbid modification of system app

* MINOR: Skip source hash generation for service (#15516)

* sync the documentation roadmap page with the getcollate roadmap (#15551)

* Application cannot be installed with None as schedule type

* None type should not register in scheduler

* scheduleTimeline fix

* fix payload of schedule type none and change it to scheduleTimeline

* fix unit test

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: IceS2 <pjt1991@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Imri Paran <imri.paran@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-03-19 18:23:25 +05:30
Mayur Singal
88ab7475e7
MINOR: Restructure dbServiceName field in dashboard and pipeline (#15548) 2024-03-15 12:42:47 +05:30
Pere Miquel Brull
fd403bae9a
MINOR - Review query performance (#15553)
* MINOR - Review query performance

* MINOR - Review query performance

* MINOR - Review query performance

* MINOR - Review query performance
2024-03-14 06:37:38 +01:00
Sriharsha Chintalapani
d0efaac877
Fix #11868: Duplicated queries cannot be created (#15519)
* Fix #11868: Duplicate query should throw an error of entityExists

* Fix #11868: Duplicate query should throw an error of entityExists

* fix test

* fix test

* Fix uniquee constraint for checksum in Postgres

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-03-13 13:02:26 +01:00
Teddy
c7afe29441
ISSUE #15336 - Added type and status for test case list (#15457)
* chore: clean up TestCaseResource tests

* fix: added status and type query params for test cases

* style: ran java linting

* fix: refactor addCondition method in ListFilter

* style: ran java linting
2024-03-08 09:01:07 +01:00
Teddy
6536ce7c94
fix: move migration to 1.3.1 (#15463) 2024-03-05 15:30:43 +01:00
Imri Paran
ec3eb297f5
add migration for MongoDB supportsProfiler = true (#15254)
* add migration for MongoDB supportsProfiler = true
2024-02-27 08:41:55 +01:00
Pere Miquel Brull
7a22a5c3f9
MINOR - Delete Reindex Ingestion Pipelines (#15344) 2024-02-26 14:36:34 +01:00
Mohit Yadav
0892da51b6
Clustering jobs across servers (#15277)
* - Added Jdbc Store for Quartz Job
- Added tables schema for Mysql and Postgresql

* Catch Object Already Exists

* Drop tables extra statements

* Use Proper Dialects
2024-02-21 21:46:30 +05:30
Teddy
e72c80f264
MINOR - fix test definition deletion (#15203)
* fix: added provider for testDefinition entities

* fix: changed testDefinition <-> testCase relationship + allow recursive deletion of testDefinition

* fix: added migration steps

* style: ran java linting
2024-02-15 18:22:26 +00:00
Mohit Yadav
9fae19dd7a
format cron from unix to quartz for internal apps , for others use unix (#15070)
* format cron from unix to quartz for internal apps , for others use unix

* Add Migration Files for 131

* Add Migrations

* Fix Json Issue and Add Exception Handling for force

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-02-14 19:59:14 +05:30
Sriharsha Chintalapani
2e95fcb98d
Fix #14786: Suggestions API (#14821)
* Fix #14786: Suggestions API

* Handle suggestions in ometa

* Minor: Optimise Databricks Client (#14776)

* MINOR - Fix SP topology context & Looker usage context (#14816)

* MINOR - Fix SP topology context & Looker usage context

* MINOR - Fix SP topology context & Looker usage context

* Fix tests

* Fixes #14598: Fix Tags / Labels ingestion on includeTags as False (#14782)

* fix(ui): password error message for char limits (#14808)

* fix(ui): password error message for char limits

* fix java side code

* Fixes #13556: Support for Salesforce table description ingestion (#14733)

* ISSUE-13556: Add suport for Salesforce table description ingestion

* ISSUE-13556: Remove unnecessary blank line

* ISSUE-13556: Fix to get description for each table

---------

Co-authored-by: Teddy <teddy.crepineau@gmail.com>

* MINOR - Better handling of Ingestion Pipeline Status (#14792)

* MINOR - Better handling of Ingestion Pipeline Status

* format

* format

* MINOR: Added table validation for cost analysis data (#14793)

* Added validation for cost analysis source

* centralized life cycle logic

* CYPRESS: simplify side navigation click in cypress (#14818)

* simplify side navigation click in cypress

* make sidbar item uses common enum

* fix cypress failure of outside import

* fix(#14326): tier dropdown is not working in advance search (#14780)

* improvement in advance search based on custom property

* fix a reading undefined property issue

* wip: advance search based on tier

* some code cleanup and improvement

* some fixes

* fix: ui flicker when advanceSearched is apply and refresh the page

* some cleanup

* no need to call customproperty api call, if entity not suppport customProperties

* minor change

* fix: autocomplete not working in tier search option in advance search modal

* added unit test for advance search provider component

* some cleanup

* added testcase for open modal

* added testcase for resetAllFilters method

* removed unwanted code

* added e2e test for testing tier advance search

* fix: e2e search flow for single field

* fix: string field not working after giving listValues in TierSearch

* fix: group query e2e test fix

* used asyncFetch way to get the tierOptions synchronously

* some cleanup

* remove unwanted lines

* some cleanup

* fix: selected option show option value instead of option title

* fix(minor): update skip icon for executions (#14809)

* Fixes #14803: ignore capitalization when confirming deletes  (#14804)

* ignore case when confirming deletes

* Test confirmation of deletes works when case differs 

Added test case for 'delete' as the confirmation text.

* minor(config): update openmetadata-ui code reviewers (#14823)

* Add Tests

* Add list/accept/reject apis

* initial ui changes

* localisation

* show suggestion for empty description

* ui feedbacks

* Fix permission check for entities without owner

* Fix entityLink and add tests

* Add update suggestion WIP

* Fix test

* Fix PUT and Pagination

* Fix styling

* update test

* Update status

* add OM server connection in apps

* add permissions check

* Fix CI

* Remove TODO

* Fix feedResourceTest

* fix unit tests

* add private configs for apps

* add private configs for apps

* fix update application icons

* minor center align icon

* add private configs for apps

* Format

* Fix pydantic gen

* Remove token

* Update name

* Rework private conf

* Fix apps

* Fix apps

* Format

* Format

* show metapilot only if its installed

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: kwgdaig <18678754+kwgdaig@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Abhishek Porwal <80886271+Abhishek332@users.noreply.github.com>
Co-authored-by: Carlo Q <carlo@machina.bio>
Co-authored-by: karanh37 <karanh37@gmail.com>
2024-01-31 18:51:09 -08:00
Mohit Yadav
270e76bdfb
[Fix] Migrate Ingestion pipeline for Data Insights to be of user provider type (#14942)
* Data Insight Report Link Fix

* Make the Data Insights Pipeline as User
2024-01-30 13:09:44 +05:30
Chirag Madlani
c936ac3a41
feat(ui): Data observability and notifications (#14806)
* Centralize EventType

* Format better for readability

* Add Handling for Thread as Change Event

* Centralize logic for Entity Message Creation

* Add Thread Parent Message in Conversation

* Add Task and Announcement

* Support for multiple destinations

* Delete existing alerts for schema changes

* Delete all tables data for event subscription

* Centralize logic for Notifications and Observability

* Change Field Name and Remove redundant code

* Test Fixes

* Add more fields

* Add Optional

* Logical Test Case Addition Chnage Event

* Add Filter By Owner

* Fix Email Entity Url

* added data observability settings page

* localization changes for other languages

* Handler closure and cleanup

* complete create observibility flow

* fix searchIndex method error

* Update Entity Observability Filter

* working observability creation

* complete creation flow

* fix action with arguments

* fix form validations and form data formulation for 'input.actions' field

* localization changes for other languages

* Fixes for Data Observability

* Add mention notification

* Only apply mention to conversation

* fIx test case

* bug fixes for add and edit observability alert form functionality

* fIx test case

* add feature to config internal along with external destinations in alert form

* Fix Formatting

* Fix Url Redirection

* add observability alert details page

* localization changes for other languages

* remove filter and action required rules

* fix styling for observability form and details page

* add delete functionality on observability alert details page

* update view page layout and add a warning in configure destination step

* localization changes for other languages

* feat(ui): support notification with new flow (#14848)

* feat(ui): support notification with new flow

* fix notifications

* add missing filters for notification alert creation flow

* update the filter hiding logic to show at the start always

* add check to show Actions section in observability alert creation form

* code cleanup and improvements

* add breadcrumb to add notification alert page

* Add General Metadata Filters

* remove code smells and code cleanup for add notification page

* add new filter

* show activity feed alert on notifications list page

* add notification alert details page changes

* localization changes for other languages

* show icons for all the entities in the alert trigger dropdown

* redirect to notification alert listing page after deleting the notification alert

* change add alert page title

* add logic to disable all destinations except email for users

* fixed unit tests

* code improvements and cleanups

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2024-01-26 13:56:18 +05:30