415 Commits

Author SHA1 Message Date
Sachin Chaurasiya
6e99fe7bda
fix(#15683): backend validation error for the following property types (#16881)
* fix(#15683): backend validation error for the following property types

* feat: Update supported date and datetime formats

* test: add e2e playwright test for custom properties

* chore: remove cypress test for custom properties

* Add date, dateTime, time backend support for custom property

* fix: update custom property display names in AddCustomProperty component

* feat: Update supported date and time formats

* chore: Update supported date and time formats

* feat: Update date and time formats in CustomProperty

* test: add playwright test

* Update supported date and time formats

* feat: Add tests for custom properties in Playwright in 2 parts

* Update supported date and time formats

* Update supported date and time formats

* chore: Update date format to uppercase in PropertyValue component

* feat: Add support for custom date format in PropertyValue component

* Update supported date and time formats

* test: add playwright test for time, dateTime, timeInterval and date properties

* Update supported time formats

* chore: Add focus to time and date pickers before filling values

* chore: Add focus to time and date pickers before filling values

* Remove date, dateTime, time from type_entity

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
2024-07-15 11:37:38 +05:30
Pere Miquel Brull
f2f8443c76
MINOR - Clean automations_workflow in 1.4.5 (#17006) 2024-07-12 13:54:46 +02:00
sonika-shah
0bf7577175
Fixes #16788 : Fix feed and count api query performance (#16993)
* Fixes #16788 : Fix feed and count api query performance

* Fixes #16788 : Fix feed and count api query performance

* Fixes #16788 : Fix feed and count api query performance

* postgres fixes

* rename thread_entity indexes
2024-07-11 14:18:33 +05:30
Pere Miquel Brull
27f4d9799e
REL #460-CL - Metadata Actions Create Permissions (#16955)
* REL #460-CL - Metadata Actions Create Permissions

* format
2024-07-09 11:13:57 +02:00
Pere Miquel Brull
97a733b704
FIX #16231 - Ingestion Pipeline w/ Bot Assignment (#16941)
* FIX #16231 - Ingestion Pipeline w/ Bot Assignment

* format
2024-07-08 11:19:16 +02:00
Sriharsha Chintalapani
74573ee4fa
Issue #10933: New Entity: API Services (#16878)
* API Service Schemas

* Create API Service Schemas

* Create API Service Schemas

* Add API Collection APIs

* Add API Endpoint APIs

* Fix #10933: New Entity: API Services

* Fix Postgres schema creation for API entities

* domain ref

* fix RESTConnection.json

* Fix openAPI reserved keyword

* fix ref

* Fix for Java Heap Space Issue

* fix APISchema spec

* fix APISchema spec

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2024-07-05 06:35:32 -07:00
sonika-shah
be243c5cea
Fix #16788: Optimize feed query performance issues introduced in 1.4.2 (#16862) 2024-07-01 19:58:47 -07:00
IceS2
927bf56738
Moved Delta Migrations to schemaChanges.sql (#16789) 2024-06-28 15:40:41 +00:00
Mohit Yadav
b05d29ccee
[DuplicateLogin] User name issue for loggedInUserApi (#16762)
* User Email Based lookup for loggedInUser

* Add Unique on name

* support email only login

* fix cypress

* fix playwright

* reset user for 404
inject enableSelfSignup for auth config

* Add Self Signup Flag

* Add error messages

* Better check

* Change to Custom Exception Message

* Update message as suggested

* fix claim issue for confidential type

* fix sonar

---------

Co-authored-by: Chira Madlani <chirag@getcollate.io>
2024-06-24 18:21:32 +05:30
IceS2
f0049853ec
FIXES 14885: Initial deltalake implementation for s3 (#16665)
* Initial deltalake implementation for s3

* Fix styles

* Fix test_amundsen

* Fix UnitTests

* Fix Checkstyle

* Fix integration tests due to datalake client refactor

* Fix unit tests

* Fix tests

* Fix Integration DeltaLake Storage test

* Skip delta storage integration test for python 3.8

* DeltaLake JSONSchema changes migrations

* Update import name

* Add some comments based on sonarcloud suggestions

* Update DeltaLake documentation

* Resolve some comments
2024-06-20 12:08:21 +05:30
Huanjie Guo
1175cf9dec
bugfix: change double quote to single quote (#15888)
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-06-17 10:51:38 +05:30
Mohit Yadav
3db41f08e2
Fix Test Suite Filter (#16615)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2024-06-12 10:40:05 +05:30
Mohit Yadav
77dfe1f6af
Fix Postgres Application listing (#16600)
* Fix Postgres Application listing

* Fix Listing
2024-06-11 13:18:50 +05:30
Teddy
ff00175fb1
MINOR - Data Quality Preparatory Work (#16523)
* chore: added min/max bound and dynamic support flag

* chore: added migration for dynamic attribute

* style: fix pythin checkstyle
2024-06-10 11:03:55 +02:00
Sriharsha Chintalapani
edcac89389
Add appType as part of schema in ingestion pipeline (#16519) 2024-06-03 21:42:40 -07:00
Teddy
451d73593e
[MINOR] partition migration issue with redshift servics (#16452)
* fix: partition migration issue with redshift servics

* chore: typo in sql comment
2024-05-29 10:49:27 +02:00
Imri Paran
a4c516d2c7
Fixes 16305: Added Test Case for Matching Enum (#16362)
* Added Test Case for Matching Enum

1. Implemented the test case using the `matchEnum` parameter.
2. Added integration tests.
3. Added migrations.

* fix tests

* fixed tests

* format

* fixed tests

* clear search cache before running ingestion

* format

* changed scopt of aws fixture

* moved migrations to 1.5.0
2024-05-28 09:30:30 +02:00
Ayush Shah
9416a7ac5f
Fix Migrations: Add postgres migrations (#16403) 2024-05-24 14:31:41 +05:30
Pere Miquel Brull
c33d1bbb18
FIX - Query checksum & fix update (#16392)
* FIX - Query checksum & fix update

* FIX - Query checksum & fix update

* FIX - Query checksum & fix update

* FIX - Query checksum & fix update

* FIX - Query checksum & fix update
2024-05-22 23:49:42 +05:30
Ashish Gupta
a5e30a0cdd
Activity feed card fixes (#16166)
* Activity feed card ui fixes

* Fix feed card for test suite

* fix testCase and testSuite feed card title and links

* Fix the profile picture not loading in the feeds

* fix the custom property and bot icon

* fix the deleted feed card by adding opacity on it

* Fix feed card hover effect
Add edit and delete button for posts

* - Fix Activityt feed event

* - Ignore workflow

* - Ignore pipelienStatus as well

* fix test case card feed

* localization keys

* sonar fix

* changes made as per comments and fix sonar issue

* Update schemaChanges.sql

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-05-10 11:04:17 +05:30
Mohit Yadav
f606212484
- Fix Search Indexing issues in Ops (#16194)
- BatchSize and recreate updated
- added conditional Init of resources
2024-05-09 17:56:38 +05:30
Mohit Yadav
b3e0470882
Migration Fixes (#16131)
* - Migration Fixes

* - Migration Fixes

* - Typo fix

* - Make warn to error

* - Update Migration to show Query Status

* - Fix Query in POstgres

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-05-07 22:07:25 +05:30
Mohit Yadav
0769d71ee7
Fixes Test Suite Reference in Table Schema (#16129)
* Fixes Test Suite Reference in Table Schema

* fix: fix test suite to interact with entity reference

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2024-05-06 19:03:23 +05:30
Suman Maharana
488078da8a
Add DDL query ingest (#15860) 2024-05-06 18:03:50 +05:30
sonika-shah
c6afcce52d
update SettingsCache and fix migration issues (#16052)
* update SettingsCache and fix migration issues

* java checkstyle change
2024-04-29 09:26:09 +05:30
sonika-shah
a9773b9527
Add predefined color palette for UI styling assets (#15982)
* Add predefined color palette for UI styling assets - backend

* Update UI components to use customLogoConfig in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in applicationConfig

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.tsx

* Update UI components to use customLogoConfig and customTheme in AppearanceConfigSettingsPage.test.tsx

* Remove existing customLogoConfiguration

* Refactor App.tsx and ThemeUtils.ts files

* Update AppearanceConfigSettingsPage.tsx with new imports and default theme values

* Update AppearanceConfigSettingsPage.tsx with new imports and reset functionality

* add cypress test

* Fix color update issue in CustomThemeConfig.spec.ts

* Remove existing customLogoConfiguration -migration changes

* Fix theme configuration initialization in useApplicationStore.ts

* Update CustomThemeConfig.spec.ts to check if the color is updated

* update existing customLogoConfiguration with customUiThemePreference -migration changes

* Update CustomThemeConfig.spec.ts to fix color update check

* Add unit tests for ThemeUtils

* move validateJsonSchema method to JsonUtils

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2024-04-25 22:43:02 +05:30
Imri Paran
0a1018648c
Fixes #15566: add dynamodb row count (#15204)
* feat(nosql-profiler): row count

1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.

* added profiler support for mongodb connection

* doc

* use int_admin_ometa in test setup

* - fixed linting issue in gx
- removed unused inheritance

* moved the nosql function into the metric class

* feat(profiler): add dynamodb row count

* feat(profiler): add dynamodb row count

* formatting

* fixed import

* format

* dded dynamodb row count

* format

* removed unused factory file

* removed "validate"

* migrations

* removed validations

* format

* linting

* fixed: test_amundsen.py

* Update schemaChanges.sql
2024-04-22 09:14:52 +02:00
Ayush Shah
d5b1465406
Fixes #14113 - Allow SSL file uploads (#15828) 2024-04-19 11:38:27 +05:30
Mohit Yadav
d1d964867d
Move migration for apps to 1.3.3 all together (#15944) 2024-04-18 14:26:05 +05:30
Mohit Yadav
dfe6204b55
Remove SQls from 1.3.2 (#15917) 2024-04-16 18:51:03 +05:30
Teddy
84469e643c
FIX #13475 - Add Parameter Validation for Min/Max tests (#15849)
* feat: add validation field to test case definition parameters

* feat: added parameter validation logic

* fix: set ES testcontainer startup attemp to 3

* style: ran java linting

* fix: added waitForEsAsyncOp in post_trigger_app_200 test
2024-04-15 07:52:43 -07:00
Pere Miquel Brull
6ed319c2dc
MINOR - delete mysql ES pipelines (#15868) 2024-04-11 16:37:49 +02:00
Mohit Yadav
ed7fa24c39
Fix Migration SQL , not executing from Handle (#15873)
* Fix Migration SQL , not executing from Handle

* fix
2024-04-11 19:41:50 +05:30
harshsoni2024
eff5394504
fix qlik db migration changes (#15852) 2024-04-11 15:48:32 +05:30
Mohit Yadav
3dc4a4f3d1
Fix SQLs (#15859)
* Fix SQLs

* Update SQls

* Update bootstrap/sql/migrations/native/1.3.3/postgres/schemaChanges.sql

* Update bootstrap/sql/migrations/native/1.3.3/mysql/schemaChanges.sql

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-04-11 15:13:16 +05:30
Imri Paran
740480fe28
MINOR: change systemApp to system (#15757) 2024-04-01 18:50:43 +05:30
harshsoni2024
884029f031
Fix #14089: Add QlikCloud connector support (#15610) 2024-03-28 20:15:22 +05:30
Pere Miquel Brull
f28f196c34
MINOR - Move app migrations to 1.3.2 (#15690) 2024-03-26 08:43:15 +01:00
Sriharsha Chintalapani
8377128c93
Application cannot be installed with None as schedule type (#15524)
* Application cannot be installed with None as schedule type

* MINOR: Add / Fix GCS and ADLS - docs, bugs (#15502)

Add GCS and ADLS docs

* Add stack trace while throwing an error to debug (#15522)

* Fix #15533: Fix name & display name for kafka json schema parser (#15534)

* MINOR: postgres add ssl options in yaml (#15538)

* Small change to fix the Glossary TErm Tasks not fetching Owner automatically (#15535)

* remove the DataInsightAlert spec as the alerts test are covered in ObservabilityAlerts spec (#15531)

* MINOR: fix help dropdown item font and icon sizes (#15511)

* fix help dropdown item font and icon sizes

* added unit test for navbarUtils

* fix sonar failure

* changes as per comments

* minor fixes

* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue (#15528)

* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue

* fixed failing unit test

* fixed unit test for Data quality test

* reverting e2eLabeler changes

* chore(ui): separate routes as per categories (#15512)

* chore(ui): seprate routes as per categories

* add tests

* fix test

* domain path fix

* [MINOR] GX logging hierarchy (#15542)

* fix: GX module logging hierarchy

* style: ran python linting

* - Use Entity Type (#15546)

* removed docs 1.4.x (#15550)

* Suggestions Alert new design (#15532)

* update ux of suggestions alert

* locales

* minor changes

* fix descriptions

* minor css fixes

* Fix #11868: Duplicated queries cannot be created (#15519)

* Fix #11868: Duplicate query should throw an error of entityExists

* Fix #11868: Duplicate query should throw an error of entityExists

* fix test

* fix test

* Fix uniquee constraint for checksum in Postgres

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* added OpenMetadataApplication.getDao (#15549)

* Add the entityUrl to the header for ThreadMessages (#15552)

* feat: AppResource (#15555)

forbid modification of system app

* MINOR: Skip source hash generation for service (#15516)

* sync the documentation roadmap page with the getcollate roadmap (#15551)

* Application cannot be installed with None as schedule type

* None type should not register in scheduler

* scheduleTimeline fix

* fix payload of schedule type none and change it to scheduleTimeline

* fix unit test

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: IceS2 <pjt1991@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Imri Paran <imri.paran@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2024-03-19 18:23:25 +05:30
Mayur Singal
88ab7475e7
MINOR: Restructure dbServiceName field in dashboard and pipeline (#15548) 2024-03-15 12:42:47 +05:30
Pere Miquel Brull
fd403bae9a
MINOR - Review query performance (#15553)
* MINOR - Review query performance

* MINOR - Review query performance

* MINOR - Review query performance

* MINOR - Review query performance
2024-03-14 06:37:38 +01:00
Sriharsha Chintalapani
d0efaac877
Fix #11868: Duplicated queries cannot be created (#15519)
* Fix #11868: Duplicate query should throw an error of entityExists

* Fix #11868: Duplicate query should throw an error of entityExists

* fix test

* fix test

* Fix uniquee constraint for checksum in Postgres

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2024-03-13 13:02:26 +01:00
Teddy
c7afe29441
ISSUE #15336 - Added type and status for test case list (#15457)
* chore: clean up TestCaseResource tests

* fix: added status and type query params for test cases

* style: ran java linting

* fix: refactor addCondition method in ListFilter

* style: ran java linting
2024-03-08 09:01:07 +01:00
Teddy
6536ce7c94
fix: move migration to 1.3.1 (#15463) 2024-03-05 15:30:43 +01:00
Imri Paran
ec3eb297f5
add migration for MongoDB supportsProfiler = true (#15254)
* add migration for MongoDB supportsProfiler = true
2024-02-27 08:41:55 +01:00
Pere Miquel Brull
7a22a5c3f9
MINOR - Delete Reindex Ingestion Pipelines (#15344) 2024-02-26 14:36:34 +01:00
Mohit Yadav
0892da51b6
Clustering jobs across servers (#15277)
* - Added Jdbc Store for Quartz Job
- Added tables schema for Mysql and Postgresql

* Catch Object Already Exists

* Drop tables extra statements

* Use Proper Dialects
2024-02-21 21:46:30 +05:30
Teddy
e72c80f264
MINOR - fix test definition deletion (#15203)
* fix: added provider for testDefinition entities

* fix: changed testDefinition <-> testCase relationship + allow recursive deletion of testDefinition

* fix: added migration steps

* style: ran java linting
2024-02-15 18:22:26 +00:00
Mohit Yadav
9fae19dd7a
format cron from unix to quartz for internal apps , for others use unix (#15070)
* format cron from unix to quartz for internal apps , for others use unix

* Add Migration Files for 131

* Add Migrations

* Fix Json Issue and Add Exception Handling for force

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2024-02-14 19:59:14 +05:30
Pere Miquel Brull
aaa649a637
MINOR - OpenMetadata OPS to load collate-service jars first (#15143) 2024-02-12 10:47:49 +01:00