88 Commits

Author SHA1 Message Date
Mohit Yadav
ebdd7f7fd9
Centralize OIDC Flow to handler, and refresh logic (#17082)
* Centralize OIDC Flow to handler, and refresh logic

* Remove forced condition

* Return on success
2024-07-18 23:53:50 +05:30
Pere Miquel Brull
7e98ece3e5
MINOR - Pydantic V2 warnings and better exception msg (#16916) 2024-07-04 14:54:41 +02:00
Sriharsha Chintalapani
3289dd15b5
Minor: Do not start limits if its disabled (#16875) 2024-07-01 22:53:31 -07:00
Sriharsha Chintalapani
5571851e53
Pluggable API/Features Limits (#16782)
* Limits

* Limits

* - Mismatched Types

* Update Limits config response

* Update Limits feature response

* Limits

* Limits

* - Mismatched Types

* Update Limits config response

* Update Limits feature response

* Limits: add entity resource enforcer

* Limits: fix rebase

* update limits enforcement

* Add OperationContext to limits

* chore: Bump versions to `1.4.0`

* chore: Bump Ingestion Versions to `1.4.0.1` for Release

* chore: Bump Ingestion Versions to `1.4.0.1`  in Dockerfiles for Release

* Remove Retry From Abstract Event Consumer (#16405)

(cherry picked from commit f8ed079731cc238dc136306fe018c5df35dd2f3b)

* Fix Migrations: Add postgres migrations (#16403)

(cherry picked from commit 9416a7ac5fa8fd9695063b108501790d813e8e6e)

* Add Null Check for isAdmin (#16407)

* Remove Retry From Abstract Event Consumer

* - Add Check for null Or Empty in isAdmin

* - Fix Test

(cherry picked from commit fe2db2d63c5495b6c288d4252a19ab77481b6de0)

* Fix OpenLineage ingestor (#16416)

* Fix OpenLineage ingestor

* py format

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>

* Minor: added whats new for 1.4.1 (#16420)

* Minor: added whats new for 1.4.1

* added note in to whats new

* Fix SSL issue (#16412)

* chore: Bump Versions for `1.4.1` Release

* chore(release): Prepare Branch for `1.4.2`

* [MINOR] partition migration issue with redshift servics (#16452)

* fix: partition migration issue with redshift servics

* chore: typo in sql comment

(cherry picked from commit 451d73593e813151c24f2c1d17efb3dcdebb71c8)

* minor(ui): update what's new for 1.4.2 (#16457)

(cherry picked from commit d55981adfd2321de706e4a043828bb473a4b05f1)

* fix:  ingestion for dbt > 1.8.0 resource_type is not an enum (#16415)

* fix: resource_type is not an enum

* feat: add log to display finis

* improve readability

* use getattr to be compatible

* format

* Add Cache Query Param for Limits

* Only Parse view  query (#16470)

* add limit check during user creation via PUT

* add limit check during user creation via PUT

* MINOR: Kafka Setup SSL Arg Fix (#16469)

* Fix#16404 - Show Node level lineage by default (#16445)

* default to node layer

* update cypress

* code cleanup

* fix cypress

(cherry picked from commit f0cda8464f34a21f45f18fa557e980fb2f105d8e)

* Invalidate count of data asset after hard delete. add limit exception to ingestion client

* - Remove Change Description from Lineage (#16488)

(cherry picked from commit 9e5c5529a84dfc781382b3a3b6abd80ee41f11f5)

* - Non Indexable fields should be remvoed at the end (#16499)

(cherry picked from commit f0b0f7a9426ca601d3bfee3989d4ce47e732a7af)

* fix announcement not redirect from landing page (#16506)

* fix announcement not redirect from landing page

* minor changes

* change in cypress test

(cherry picked from commit ee7cddd169a3a1fb1e598e80035c2fc15a5a129b)

* Fix Schema Field Null Issue (#16510)

(cherry picked from commit 022772943f1b33f6230cb35547d1da6acfaf6cfa)

* feat(ui): limits integration with application (#16206)

* feat(ui): limits integration with application

* support pipelineSchedules via limit api

* enforce limit to all the modules

* update banner styling

* update

* support disable option for ManageButton

* limit version

* fix spotlight

* update tests

* Add name and version history to resource limits
Refactor the getEntityIcon function and add new icon mappings

* limit version

* hide access token tab

* fix version for all the entity

* fix tests

* fix DQ tests

* Add fallback for the icon

* Revert the fallback icon changes

* Apply the limit to the add ingestion button in the service details page

* Fix the data quality tab add test button not working

* fix banner styling

* minor fix

* Fix ingestion component unit test

* Add InlineAlert component

* update entityNameLabels mapping object

* Fix the incorrect link in LimitBanner

* update pricing page url

* Create the GlobalSettingsClassBase

* Update URLs for pricing page and upgrade options

* fix global settings uncaught error

* add parameters to the resource limit API

* implement inline alerts for service and alert creation form

* update PRIVILEGES for docker

* fix layout issues

* fix tests

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>

* Add token limitations

* Add token limitations

* Add appType as part of schema in ingestion pipeline (#16519)

* #16489: fix the redirect issue to new tab for tags and glossary (#16512)

* fix the redirect issue to new tab for tags and glossary

* fix the redirect on cancel icon and unit test issue

* changes as per comments

(cherry picked from commit 8d312f0853609cfef260739cf789d459838a3421)

* Fix  #16229 - Tag and Service filters for test cases (#16484)

* fix: added test case support for tags (inherit from table/column)]"

* feat: add tag and service filter for test cases

* feat: add tier query param

* fix: tests

(cherry picked from commit 6b00dde90285924445567ee7c396c89f0fcf3f1d)

* fix: None type is not iterable (#16496)

(cherry picked from commit 656da03b14ca24171cf7924b9dd33663e6bed423)

* minor(ui): refresh token for OIDC SSO (#16483)

* minor(ui): refresh token for OIDC SSO

* remove frame window timeout issue

* increase iFrame timeout for oidc

(cherry picked from commit 1a6c4c972052836a9b3cfa273b7ea1aa3202eafe)

* feat(ui): support tag & tier filter for test case (#16502)

* feat(ui): support tag & tier filter for test case

* fix tag filter

* allow single select for tier

* added service name filter

* update cypress for tags, tier & service

* add specific add for filters

* fix tier api call

(cherry picked from commit 5b71d79e8ac2d08a154882dfe71b9b3a0f73bffc)

* minor: sanitize activity feed editor content (#16533)

* Add appType as part of schema in ingestion pipeline (#16519)

* Fixed quicksight conn (#16537)

* fix: saml auth for new user not created (#16543)

* fix: saml auth for new user not created

* doc: add comment

* Fix#16491 -  fix lineage edge description update (#16538)

* fix lineage edge description update

* fix tests

(cherry picked from commit dff0aa8dbedcd4064ad63765cadda65bb998772e)

* CYPRESS: fix announcement cypress (#16536)

* fix announcement cypress

* changes as per comments

* fix the cypress failure

(cherry picked from commit fcb87b5866ba06aa7a6db516677e311c24053db7)

* [MINOR] Fix Test Failure for EventRegistration

* [MINOR] Fix Test Failure for EventRegistration

* [MINOR] Fix Test Failure for EventRegistration

[MINOR] Fix Test Failure for EventRegistration

* Fix Event Handlers registration Issue (#16544)

* Fix Event Handlers Issue

* Review Comments

(cherry picked from commit d374e48b7938e8ad3514dc5cf8dff619a12595e3)

* [MINOR] Fix Test Failure for EventRegistration

(cherry picked from commit 4563ad4fd10f9790c21fe744d8fc131ebd028ac8)

* Fix Topic Schema missing messageSchema (#16545)

(cherry picked from commit b612dd90c07f564d38392b1ccfe0de1505a4867b)

* Add limits exception cache in rest client

* MINOR: Ignore Cluster Information from columns (#16495)

* minor: improve the block editor initial content history (#16540)

* Minor: fixed data quality page type issue (#16556)

* #16521: fix issue in userProfilePage for roles. teams and displayName (#16527)

* fix update on roles and backlink them in user profile page

* fix teams, displayName and profile pic issue

* sonar fix

* fix cypress issue

* minor changes

(cherry picked from commit 98945cb2db87ebb325d3a72131f049abffcba345)

* Empty quick filters (#16402)

* initial commit for empty quick filters

* update progress

* fix field title

* cleanup

* add tests

* unit tests

* fix encoding of search query

* add cypress tests

* add cypress

* fix flaky cypress

* fix review comments

* revert tooltip changes

* fix tests

* fix tests

(cherry picked from commit 5930cd7a7a4bef73f6850848c85118eb64843e2d)

* Fix #16278 : Search to display Draft glossaryTerms on Explore page (#16462)

* Fix #16278 : Search to display Draft glossaryTerms as well on Explore page

* add term status quick filter

* change aggregation key for status field

* change aggregation key for status field

* add lowercase_normalizer in status filed for aggregate api

* add cypress tests

* fix cypress

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
(cherry picked from commit ae5e9d61cc9e6a39d65972987de9149a421395b1)

* [FIX] GlossaryTerm reviewers should be user or team only (#16372)

* add teams as reviewer

* Check Users to be reviewers

* Reviewers can be a team or user

* Fix check by id or name

* Review can be team or user both

* Validate Reviewers

* add multi select control

* - Fix Reviewers

* - Centralize Reviewer Relationship to EntityRepository

* - Sort

* add team as reviewer for glossary terms

* locales

* cleanup

* - Update Reviewer should remove existing reviewers

* fix selectable owner control

* fix code smells

* fix reviewer issue

* add glossary cypress

* fix patch issue on reviewers set to null

* update cypress tests

* fix cypress

* fix cypress

* fix reviewers in glossary task and supported cypress

* fix pytest

* Fix

* fix cypress

* fix code smells

* Inherited Reviewers need to be present always

* filter out inherited users

* fix cypress

* fix backend tests failure

* fix backend tests failure -checkstyle

* restrict owner to accept task in case of reviewer present

* fix pytest

---------

Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: sonikashah <sonikashah94@gmail.com>
(cherry picked from commit 9ec3d94e3b8445e63a7d77239c92c92a32536bf2)

* Add testSuite tags, domain field and check for TestCase limits

* fix owner not showing after refersh in teams page (#16567)

(cherry picked from commit 119fcf8959732a980b75e1f795a9f2dc5288cd27)

* [ISSUE-16503] Fix createUser to use EntityResource (#16549)

* Fix createUser to use EntityResource

* fix broken tests

* Fix Tests - 3

(cherry picked from commit aeb020ae3b0cbab3a2ee5995c61480cdd1eae405)

* what's new for 1.4.2 (#16568)

(cherry picked from commit c86468d9929e433922886852381269b46d69c832)

* address feedbacks

* fix error for bots page

* update banner text

* allow force fetch limit

* fix ingestion schedule

* Revert "Merge branch '1.4.2' into limits"

This reverts commit 8e965207a23ba527d0f5ba91463c1869077bf091, reversing
changes made to 4d16531965fb0d489a4afdebd45ab5b7f3d1eb5c.

* Merge 1.4.2 (#16578)

* fix explore page conflicts

* fix tests

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>

* fix subheader

* Updating glossary reviewers should propagate reviewers in glossary term (#16580)

* highlight inherited reviewer in glossary

* locales

* use glossary name for search query

* fix glossary version cypress

* add union datatype for subfields

* Adding reviewer to glossary also adds them as an assignee to the task

* add glossary approval cypress

---------

Co-authored-by: sonikashah <sonikashah94@gmail.com>
(cherry picked from commit 4c8bf1cac14074df87dafe7a719e2795b0a29895)

* Update documentation for Search Index apis (#16539)

(cherry picked from commit d3123c49143652015c416d271d9fd0f9cfa9e324)

* cypress: fixed flakiness and announcment cypress (#16579)

* fetch latest limit for create / delete operations

* guard datAsset limit got topic, dashboard, mlmodel etc

* Fix: Ensure correct index mapping in Elasticsearch for clusterAlias (#16589)

* Fix: Ensure correct index mapping in Elasticsearch for clusterAlias

* Fix: Ensure correct index mapping in Elasticsearch for clusterAlias

(cherry picked from commit 8723b8c36afe31410c31d1ebbdafe7b1770921fa)

* cypress: fixed cypress AUT for mysql (#16446)

* cypress: fixed cypress AUT for mysql

* minor fix

* skip announcment redirection cypress

* Minor: Ensure correct index mapping in Elasticsearch for clusterAlias (#16598)

(cherry picked from commit 04543722a6f6e2b1eaf7a451ebb1c176862bc346)

* Fix Postgres Application listing (#16600)

* Fix Postgres Application listing

* Fix Listing

(cherry picked from commit 77dfe1f6af53d187ff7a61fdb1e1416de7178f5a)

* fix limit related issue

* Fix Automations limits invalidation during the uninstall

* cypress: fixed 1.4.2 AUT cypress (#16602)

* cypress: fixed 1.4.2 AUT cypress

* fix cypress around announcement,user,glossary, lineage and mydata

* searchIndexApplication fix and minor changes

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>

* test: add updateJWTTokenExpiryTime util (#16606)

(cherry picked from commit 8c173bed6a279cb0a648bd30632ea6ebdf4a2a90)

* OSS changes for adding automator cypress tests (#16611)

* Fix Test Suite Filter (#16615)

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
(cherry picked from commit 3db41f08e27f388495040e5b23cc7bee5ae665f1)

* MINOR: Fix Profiler for SSL Enabled Source (#16613)

* Add Test Suite SSL (#16619)

* MINOR: Fix ssl connection in usage & lineage (#16625)

* Fix owner notification (#16629)

* - Fix Task notification not getting sent to owners

* - Fix Task notification not getting sent to owners

(cherry picked from commit cc2d581eb0524604b6dcf0523e9ca96e0b8a6ce3)

* chore(release): Prepare Branch for `1.4.3`

* - Fix User Signup (#16667)

(cherry picked from commit b4cba8a850ecd7a25aeff6ca7dea0dc432d43d86)

* - Fix User Signup - p2

(cherry picked from commit d9ae6f6db9891f8e9bf7ad49c561a71dd50103da)

* - Update What's new (#16669)

- fix vulnerability

(cherry picked from commit 1dcb1bd46f9da49764f4c61a7ac5048dd2fa956b)

* Minor: Fix incorrect alert on signup page (#16666)

* Fix Application enforceLimits during install

* Wrap the add test button with limits wrapper for column profile tab

* fix errors

* fix tests

* fix pylint

* fix tests

* fix limits

* pylint

* fix schedule options

* fix glossary spec failure

* Add domain & tags to testSuite

* Update airflow-apis-tests-3_9.yml

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Chira Madlani <chirag@getcollate.io>
Co-authored-by: Pablo Takara <pjt1991@gmail.com>
Co-authored-by: Akash-Jain <15995028+akash-jain-10@users.noreply.github.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Maxim Martynov <martinov_m_s_@mail.ru>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Antoine Balliet <antoine.balliet@gorgias.com>
Co-authored-by: Suman Maharana <sumanmaharana786@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-01 14:59:25 +05:30
Teddy
47a13e27cf
MINOR: Improve filtering method (#16810)
* fix: improve filtering method

* fix: filter type

* fix: failing tests

* fix: revert log level
2024-06-27 14:15:28 +02:00
Mohit Yadav
5ddbe8b210
Fix settings for email not encrypted on init (#16541)
* Fix Email Settings not encrypted

* Fix Email Settings not encrypted

* Move fernet to secure test
2024-06-05 20:52:44 +05:30
Teddy
9f49b41acf
fix: remove classifier config from openmetadata config file (#15998)
BREAKING CHANGE: if you previously used your own classifier this will remove its support
2024-04-24 11:41:40 +02:00
Mohit Yadav
27e9012a56
Activity Feed Improvements (#15840)
* - Created Specific Schemas for different change Events
- Added Card Style for different Events
- Added Feed Info for runtime card creation and graph drawing in case test Case Result updates

* Review comment

* Test fix

* Revert unrequired changes

* Reverted changes

* Fixed Tests

* fix test

* Add Domain, Data Producs Asset addition/removal Uppdates

* Probale test case failure fix

* Followers Fields
2024-04-22 18:42:40 +05:30
Mohit Yadav
bc13659a6d
Inherit sso roles (#15903)
* Use Roles from Provider

* Add Cache For Roles From Users

* Fix Failing Test

* Revert secrets

Update Impl to use loggedInUser and CreateUser for Role assignment and resync

* Revert Expiry check
2024-04-18 20:46:04 +05:30
Pere Miquel Brull
4db6dce2f9
MINOR - Add /status endpoint (#15394)
* MINOR - Add `/validate` endpoint

* rename endpoint

* fix test

* add new migration

* Status

* Simplify migration client

* Simplify migration client

* Simplify migration client
2024-03-25 07:05:13 +01:00
Mohit Yadav
91184ecf5d
Using Backend for Oidc Flow (#15557)
* - Initial Draft of Using Backend for Oidc Flow

* - Review Comments

* - Add Session Handler

* - Add Nonce Validation

* Typo

* support backend auth flow from client side

* Add Refresh Logic

* Remove oidc Config from /auth

* Auth Logout Handling

* Add Refresh as separate endpoint

* return empty on no refresh token

* support refresh token and logout

* fix route issues

* fix path issues

* Redirect to Logout on know session

* fix store issues

* exclude logout api from domain filter

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2024-03-19 17:15:30 +05:30
Pere Miquel Brull
060a9d7f68
MINOR - Prepare AI chat feed (#15510)
* MINOR - Prepare AI chat feed

* Rename ai to chatbot

* Rename ai to chatbot
2024-03-18 10:29:52 +01:00
Imri Paran
d3fe165347
added OpenMetadataApplication.getDao (#15549) 2024-03-13 15:25:21 +01:00
Pere Miquel Brull
5b2427743c
MINOR - Reconfigure app metrics (#15460)
* MINOR - Reconfigure app metrics

* Format
2024-03-07 15:19:21 +01:00
Pere Miquel Brull
88c3c42b3f
#15425 - Move SecretsManagerUpdateService to migrate command (#15444)
* #15425 - Move SecretsManagerUpdateService to migrate command

* #15425 - Move SecretsManagerUpdateService to migrate command
2024-03-05 06:47:16 +01:00
Mohit Yadav
f2b1622c0f
Add Dao Get Call based on type (#15429)
* Add Dao Get Call based on type

* Keep Jdbi as Attr
2024-03-02 23:43:28 +05:30
Mohit Yadav
92fd428380
Fix Cache Init (#15196) 2024-02-15 19:19:03 +05:30
mohitdeuex
84d2aaaf30 Check if Smtp is Enabled to set random password 2024-02-15 16:28:10 +05:30
Teddy
e6cacb19f1
MINOR - DQ incident severity classifier (#15149)
* feat: added severity classifier for DQ incidents

* feat: added severity classifier tests

* style: ran java linting
2024-02-13 07:02:09 +01:00
07Himank
795bd5c867
Added Index Factory Config (#14950)
* Index Factory Config

* CI handling

* checkstyle
2024-02-01 19:50:51 +05:30
Mohit Yadav
dad886eaba
Remove LMAX Disruptor , use queue and Quartz (#14422)
* Remove LMAX Disruptor , use queue and Quartz

* Spotless Fix

* Spotless formatting

* Review Comments

* Improvements

* Improvements to add Retry and DLQ

* Spotless fix

* Update Stats

* Fix stats

* Fix delete Failures

* Fix Failing Tests

* Fix Failing Tests

* Fix Failing Tests

* Postgres Fix

* Fix Schema Change for Change Event Table

* Failing test

* Fix Error for non ChangeDescriptions

* Adds Observability On Top of Event Subscription

* Make List Separate

* Add more filters for Test Suite

* Populate Default Schema Updates to Look for

* typo

* Add matchFieldChange and other in paramAdditionalContext

* Review Comments

* Spotless fix

* Merge issue

* Update Filter as per new Syntax

* Fix Failing Test
2024-01-09 00:13:45 +05:30
Suresh Srinivas
496ae26cef
Code clean up related to issues flagged by Sonar and IDE (#14491) 2023-12-22 16:08:04 -08:00
Suresh Srinivas
451a20b3cd
Adopt Java 17 features (#14430)
* Adopt Java 17 features

* Make the code comments compact
2023-12-18 17:52:01 -08:00
Sriharsha Chintalapani
1294f93e15
Minor: Migrate to latest google code style library to support Java 17 and beyond (#14429)
* Minor: add spotless and use simplecontext

* Remove context from rule evaluaiton

* Fix EventSubscription tests

* Minor: Migrate to latest google code style library to support Java 17 and beyond

* Minor: Ignore code style migration from git blame
2023-12-18 12:56:17 -08:00
07Himank
db921645f3
changes needed for adding knowledgeCenterIndex (#14223)
* chnages needed for adding knowledgeCenterIndex

* fix tests

* CI fix
2023-12-11 16:10:52 +05:30
Mohit Yadav
2ae0333551
Fix Email Issues (#13962)
* - Added Unauthenticated Server Emails
- Removed redundant OM_URI
- Add Test Email in SystemResources

* remove default port

* remove fernet
2023-11-14 09:04:53 +05:30
Mohit Yadav
3580e4512d
Add DataInsightsApp (#13549)
* Add DataInsightsApp

* Add app Screenshots

* Data insight app UI (#13566)

* fix: add data insights report application

* fix: minor ui feedbacks

* fix: minor css border

* fix: added carousel code for marketplace app details

* fix: add app screenshots

* fix: update app screenshots

* fix: type errors

* Add External App Ingestion Schema

* Add Deploy and Trigger under App resource

* Add pipeline Status and logs as single response

* Fix tests

* Fix tests

---------

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Sriharsha Chintalapani <mail@harsha.io>
2023-10-17 19:51:33 -07:00
Sriharsha Chintalapani
e79ce5fe07
Remove jdbi unit of work (#13550)
* Remove JDBI Unit of Work transactions

* Remove JDBI Unit of Work transactions

* Remove JDBI Unit of Work transactions
2023-10-12 16:07:50 -07:00
Mohit Yadav
ab1a156b77
Build Applications Automations (#13318)
* [WIP] Applications

* Remove JobRunr Dependency

* Cleanup Files

* Reformatted

* Add Api to delete schedule for app

* Fix Applications issue

* added missing tables for application automation

* fixed buil failure

* Add Tables for Applications

* Migrate Search Reindexing Job to New Apps

* Fixed Merge Conflicts

* Fix Build

* Add DataInsightReportJob as App

* Review Comments Fix

* update err msgs

* Add Features fields

* Add Default Search Index Application

* Fix Failures

* Add Data Insights application

* Fix trigger and add update fields

* Add Stats context

* Fix Scheduler Initialization

* Fix Stats Issue

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: 07Himank <himank07mehta@gmail.com>
Co-authored-by: 07Himank <112613760+07Himank@users.noreply.github.com>
2023-10-10 11:32:37 +05:30
Pere Miquel Brull
c9b3724003
Support migration extensions (#13436)
* Support migration extensions

* Prep test

* Prep test
2023-10-07 16:00:47 -07:00
Suresh Srinivas
d0f3ee6a2a
Removing redunant throws exceptions (#13468) 2023-10-06 15:43:45 -07:00
Sriharsha Chintalapani
1a90c5c1b3
Refactor Repositories: Remove passing around CollectionDAO (#13427)
* Refactor repository registration

* Refactor repository registration

* Refactor repository registration

* Refactor repository registration
2023-10-03 23:41:06 -07:00
Sriharsha Chintalapani
eb84c524aa
Fix repository registration (#13425)
* Fix failing repository registration in external

* Fix failing repository registration in external

* Scan classPath to find Repository annotation

* Remove additioanl config, register any classes with Repository or Collection annotation

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-10-03 21:03:56 -07:00
Sriharsha Chintalapani
8578aee04f
Search refactor (#13397)
* Refactor Search

* Refactor Search

* Fix propgation bugs

* Fix propgation bugs

* Fix glossary term search

* Fix glossary term search

* Only issue search requests if the client is configured properly

* Only issue search requests if the client is configured properly

* Fix glossary index

* add documentation for sharded tables (#13361)

* Cost analysis agg (#13408)

* feat: updated DI workflow to inherit from BaseWorkflow + split processor and producer classes

* feat: __init__.py files creation

* feat: updated workflow import classes in code and doc

* feat: moved kpi runner from runner to processor folder

* fix: skip failure on list entities

* feat: deleted unused files

* feat: updated status reporter

* feat: ran linting

* feat: fix test error with typing and fqn

* feat: updated test dependencies

* feat: ran linting

* feat: move execution order up

* feat: updated cost analysis report to align with new workflow

* feat: fix entity already exists for pipeline entity status

* feat: ran python linting

* feat: move skip_on_failure to method

* feat: added unusedReport to DI

* feat: added aggregated unused report

* feat: ran linting

* feat: reverted compose file changes

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>

* Add Java 17 support (#12895)

* Add Java 17 support

* Change Test HTTP client provider

* Create Tests HTTP Client once

* Create Tests HTTP Client once

* fix(CI): Update CI to use jdk 17 and dockerfiles as well

---------

Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>

* Refactor Search

* Refactor Search

* Fix propgation bugs

* Fix propgation bugs

* Fix glossary term search

* Fix glossary term search

* Only issue search requests if the client is configured properly

* Only issue search requests if the client is configured properly

* Fix glossary index

* Merge main

* fix style

* deleted field propagation

* Fix style

* close the test client

* Changing to jersey connector

* Fix Authentication Exception headers

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Akash-Jain <Akash.J@deuexsolutions.com>
2023-10-03 14:56:53 -07:00
Sriharsha Chintalapani
11f1df437b
Fix SettingsCache initialization (#13395)
* Fix SettingsCache initialization

* Fix SettingsCache initialization

* Fix SettingsCache initialization
2023-10-01 15:47:25 -07:00
Suresh Srinivas
2e2f9a1edb
Separate initialization of Repository and Resources (#13389)
* Separate initialization of Repository and Resources

* Change that speeds up the test. Still WIP

* Rebasing the branch with main

* Revert back previous changes and use Repositories from registry for migration

* Fixing test failures

* Code cleanup

* Fix test failures
2023-09-30 22:13:16 -07:00
Artiom Darie
7d2f8dc2bb
Fixes issue-11740: Added support for the om service to connect to AWS RDS using IAM roles (#11913)
* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Changed intial implementation accordingly. Added better flexibility for different auth prodvider impl

* ISSUE-11740: Clean up unnecessary classes

* ISSUE-11740: Clean up unnecessary properties

* ISSUE-11740: Clean up unnecessary properties

* ISSUE-11740: Clean up unnecessary properties

* ISSUE-11740: Clean up unnecessary properties

* ISSUE-11740: Clean up unnecessary properties

* ISSUE-11740: Code formatting

* ISSUE-11740: Added support for the om service to connect to AWS RDS using IAM roles

* ISSUE-11740: Moved docs to 1.2 version

---------

Co-authored-by: artiom.darie <artiom.darie@adswizz.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-09-27 09:59:24 -07:00
Mohit Yadav
6d4e425f36
Remove Cleanups for JdbiUnitOfWorkProvider + added cases for autocomm… (#13163)
* Remove Cleanups for JdbiUnitOfWorkProvider + added cases for autocommit in case of non transactional events

* remove unnecessary clear

* typos
2023-09-12 20:28:42 -07:00
Mohit Yadav
5f5aeef7f0
Transaction for Delete @UnitOfWork (#13115)
* Fix #12494: Jdbi Transactions opens a new connection per DAO causing recursive delete to not to rollback

* Fix #12494: Jdbi Transactions opens a new connection per DAO causing recursive delete to not to rollback

* rebase with main

* Fix styling

* Add jdbiUnitOfWork

* Improvements for Change Event Repository and missing Annotations

* Add connection commit/rollback

* push schemaChange fix

* Improvements for Change Event Repository and missing Annotations

* Pass wrapped collectionDAO everywhere

* Pass wrapped collectionDAO everywhere

* Separate event handlers transactions

* Fix checkstyle

* Wrap PUT , POST, and DELETE in transactions

* Add Patch for transactions

* Add primary key to tag.id

* Proxy internal @CreateSQLObjects

* Fix exception handling in ManagedHandleInvocationHandler

* Java typo

* Update schema files

* Checkstyle fix and conflicts resolve issue

* Remove @JdbiUnitOfWOrk from Feed Repository

* remove unnecessary @JdbiUnitOfWork annotation

* Test Failures fix

* Test Failures fix

* Increase wait time , changeEventDAO takes time due to handle.commit()

* commit change

* No need to use commit in ChangeEventHandler

* Add a lookup for avoiding transaction that are not started and should not be rolled back

* remove JdbiUnitOfWork.java

---------

Co-authored-by: Sriharsha Chintalapani <harsha@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-09-08 23:19:44 +05:30
Mohit Yadav
9712337b01
[SecurityHeaders] Fixed issue with Security Headers (#12921) 2023-08-18 17:48:17 +05:30
Sriharsha Chintalapani
524e611edc
Deprecate Flyway and use Native migrations (#12865)
* Add ascii_bin for fqnhash, Add Constraints on timeseries

* Attemp of moving migrations to files

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Move upstream migrations to new path
2023-08-17 14:39:05 -07:00
Sriharsha Chintalapani
da0c79cf78
Fix bulk soft-delete or restore from search indexes (#12807)
* Fix bulk soft-delete or restore from search indexes

* Fix bulk soft-delete or restore from search indexes

* Add entityRestored to list events api
2023-08-09 23:24:10 -07:00
Suresh Srinivas
340a029807
Fixes #12802 Add support for generic entity cache (#12803)
* Add entity cache in EntityRepository for all the entities

* Set entity fields only when it is not already set

* Clone entities from the cache

* Remove SubjectCache

* Remove RoleCache

* Remove PolicyCache

* Remove TagLabelCache

* Deleting lingering files

* Test failure fixes

* Remove getEntityReference methods from EntityDAO

* Move quoteName logic to EntityRepository

* Fix test failures
2023-08-08 16:42:54 -07:00
Sriharsha Chintalapani
67b68a70c9
Fix #7712: Align AuthProvider values in the JSON schema definition (#11508)
* Fix #7712: Align AuthProvider values in the JSON schema definition

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Handle new providers

* Lint

* Fix #7712: Align AuthProvider values in the JSON schema definition

* Handle new providers

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Lint

* fix rebase

* Fix test

* Fix test

* Fix Usage of Auth Provider in User Resource

* remove unused function

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2023-07-26 11:34:58 -07:00
Mohit Yadav
9a643c3a41
Fix Missing We Config on Error Page Response in #UI (#12428) 2023-07-13 21:16:05 +05:30
mohitdeuex
989914f84e Add server migration validation 2023-07-03 20:15:13 +05:30
Suresh Srinivas
4d4993effc
12142.task1 - Add support for Domains (#12143) 2023-07-02 11:32:52 -07:00
Sriharsha Chintalapani
d346101249
Update fqn size (#12188)
* Fix #10198: Filter tags and classifications if the classification is disabled

* Fix #10198: Fix tests

* Fix #10198: Fix tests

* Fix #10198: Fix Postgres tests

* Fix #10198: Fix Postgres tests

* Fix fqnHash column size to be 768

* Fix Failing test due to primary_key size limit

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-06-29 00:31:10 +05:30
Sriharsha Chintalapani
33c752d2f5
interfaces for db migrations (#12057)
* interfaces for db migrations

* Add Data Migration for entities to 1.1

* Migration for FQN hash

* Added Tag Usage And Fixed some transaction issue in testing

* Added SERVER_CHANGE_LOG validations

* Update Path for files

* Added Postgres Migration

* Test Suite Migration [WIP]

* remove ingestion pipeline

* moved Migration to Table Initializer

* init Data source config in TableInitializer

* make checksum ignore configurable

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-06-26 13:03:44 +05:30
Pere Miquel Brull
520d5e2657
Add Pipeline Service Status Job to publish to Prometheus (#12046)
* Add Pipeline Service Status Job to publish to Prometheus

* Add Pipeline Service Status Job to publish to Prometheus

* Add config for health check interval
2023-06-22 06:53:56 +02:00