1565 Commits

Author SHA1 Message Date
Mayur Singal
1386b43607
Fix #6141: Ingestion Pipeline Status Updates (#8216)
* Fix #6141: Ingestion Pipeline Status Updates

* List Pipeline Status API & Improvements

* Rename State field to PipelineState in UI

* Convert Pipeline Status array to single object

* fix braking UI

* Rebase Fixes

* Profiler, TestStuite & DataInsigts Pipeline

* py_format

* fix logs page not loading
add pipelineStatus endpoint

* fix recent run changes

* Fix Tests

* address review comments for ui

* fix failing checks

* fix unit tests

* fix cypress test

* remove loader test as not using it

* wait for API when we click on the tab

* fix cypress waitFor api

* fix failed cypress tests

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2022-11-03 14:37:26 +05:30
Suresh Srinivas
d47f5738cb
Add support for system provided Tags, Glossaries, Policies, and Roles #8484 (#8508) 2022-11-02 14:56:31 -07:00
Mayur Singal
fa68a1f18f
Fix #6754: Added User Details for Usage (#8449) 2022-11-02 14:07:41 +05:30
Suresh Srinivas
8a008022c8
Fixes #7004 Backend : conditionAllow should not be applicable to create operation (#8483) 2022-11-01 12:46:18 -07:00
Parth Panchal
af85b9629d
Fixed#8239: Query mismatch while searching for - and _ makes data overlap (#8476)
* Fixed#8239: Query mismatch while searching for - and _ makes data overlap

* Fixed#8239: Query mismatch while searching for - and _ makes data overlap
2022-11-01 18:20:03 +05:30
Suresh Srinivas
e7a4a7a3e5
Fixes #6681 Add support for policy evaluation API (#8466) 2022-10-31 15:54:32 -07:00
Parth Panchal
ca9c571179
Fixed#8354: Support Login For User from multiple devices with different refresh tokens (#8454)
* Fixed#8354: Support Login For User from multiple devices with different refresh tokens

* Fixed#8354: Support Login For User from multiple devices with different refresh tokens
2022-10-31 19:17:26 +05:30
Sajitha
a731b3fdfd
Fix sonar issues (#8421)
* Fix sonar issue

* Remove duplicate constant
2022-10-30 12:32:45 +01:00
Teddy
f027c26cc1
Remove authorizer for the collect endpoint (#8398) 2022-10-28 12:45:55 +02:00
Sajitha
debba08190
Fix sonar: Use try-with-resources and handle exception (#8414)
* Fix sonar: Use try-with-resources and handle exception

* Fix formatting
2022-10-28 13:04:54 +05:30
Suresh Srinivas
a6438c6347
Fixes #8401 Remove lingering authorizeAdmin in TagResource (#8403) 2022-10-27 15:37:44 -07:00
Mohit Yadav
6efe6df96b
[LDAP] Added Ldap-Authentication (#8256)
* [LDAP] Add Ldap-Authentication

* Refactor Basic to Service Layer

* added read login config

* pom updates

* review comments

* review comments

* move authenticator after authorizer

* merge conflicts resolve

* add init method , for initializing authenticator

* sonar fix
2022-10-27 22:56:20 +05:30
Suresh Srinivas
1f4d16dab0
Remove unused methods in ResultList implementations (#8378)
* Fixes 8371 Remove bot flag from authorizeAdmin method

* Remove authorizeAdminOrBot method

* Fix merge errors

* Remove unused constructor methods
2022-10-27 07:58:03 -07:00
Nahuel
1732d85dd6
Fix: Update Maven dependencies (#8383)
* Update Maven dependencies

* Update Jetty dependency
2022-10-27 16:21:30 +02:00
Suresh Srinivas
ed8c74e9b7
Fixes 8371 Remove bot flag from authorizeAdmin method (#8372)
* Fixes 8371 Remove bot flag from authorizeAdmin method

* Remove authorizeAdminOrBot method

* Fix merge errors
2022-10-26 20:36:17 -07:00
Shivam Shrey
0babaa52e0
ISSUE-7848: Fix variable names as per naming convention (#8228)
* ISSUE-7848: Rename local variables to match regular expression for camel case

* ISSUE-7848: Refactor package name as per naming convention
2022-10-26 05:54:40 -07:00
Rohit Agarwal
31d5835387
Fixed Issue#8342 (#8365) 2022-10-26 05:47:42 -07:00
mosiac1
ad7e4a2341
UI (#6189) | Refactor Explore and ExplorePage, add Logical Query Builder (#8140)
* Add logical query builder and Explore page refactor

* Add file headers

* Change search aggreagation keys

* Move query builder utils, remove comments

* Add comments

* Rename AdvancesSearch.constants.ts

* Add review changes

* Handle deleted elements correctly in Explore

The `deleted` field needs special handling because `showDeleted === false` should return only documents where `deleted` is false, but `showDeleted === true` should return documents regardless of their `deleted` field.

* Fix facet filter issue

Issue was the following: the user set a facet filter on `aggregationKey` and `value`. Then user changes the query builder so that in the new result there is no bucket in `aggregationKey` with `value`. The facet filter would still be used, but not visible to the user and not removable.

* Fix facetFilter tests

* Move isFilterObject type guard

* Use antd tabs for Explore component

* fix cypress failure

* miner fix in cypress

* fix typo

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2022-10-26 03:47:49 -07:00
Teddy
83b048c07c
Fixes #8206 -- Implement data insight refinement for entity report data (#8351)
* added data insight module to refine and ingest data

* Added json schema for data insight workflow

* Added mixin for data insight

* Added tests for data insight workflow and methods

* Fixed python style

* Fixed python linting

* Fixed unit test

* Added data insight worflow + workflow logging

* Fix python style

* Renamed analytic -> data_insight + added test to make + fixed style

* Fixed java style

* Fixed code smells + python style

* Ignore ES linting for now

* Fixed comments from review

* fix python formatting

* Fixe PipelineType attribute for data insight
2022-10-26 11:18:08 +02:00
Suresh Srinivas
e9ebb4dd75
Fixes #8343 Differentiate create from update in PUT operations (#8344) 2022-10-25 04:14:44 -07:00
Suresh Srinivas
f12b09687d
Fixes #8326 Add roles to system bots during initialization and use po… (#8327)
* Fixes #8326 Add roles to system bots during initialization and use policy based authorization for bots

* Remove var usage and instead use specific types for variables

* Fix bot can't create entities
2022-10-24 17:53:11 -07:00
Suresh Srinivas
82b81572ac
Update javadoc version (#8329) 2022-10-23 22:07:58 -07:00
Mahesh Pai R
0a1c701c76
Corrected null checks (#8320) 2022-10-23 05:43:37 -07:00
Suresh Srinivas
f668a58668
Fixes #8394 Use Awaitility pollInterval instead of iteration loop (#8325) 2022-10-23 01:40:38 -07:00
ssandy83
e6f686e857
ISSUE-1987: Replace thread.sleep with awaitility library (#8305)
* ISSUE-1987: Replace thread.sleep with awaitility library

* ISSUE-1987: Replace thread.sleep with awaitility library

* ISSUE-1987: increase polling period to 100ms

Co-authored-by: ssundaresha2 <sandeep_sundaresha@intuit.com>
2022-10-22 07:22:32 -07:00
Suresh Srinivas
0d75eb9ff2
Fixes #8313 Deprecate botPrincipals from open metadata configuration (#8314) 2022-10-21 20:49:41 -07:00
Finn McClusky
3f7c8dc2e2
FIX 8196: Remove the create-ingestion-bot and update-ingestion-bot from bootstrap_storage.sh (#8202)
* ISSUE-8196: removed create-ingestion-bot, update-ingestion-bot options and unused functions from table initializer

* java checkstyle fix

Co-authored-by: Finn McClusky <finn.mcclusky@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2022-10-21 22:41:02 +05:30
Suresh Srinivas
229b56e7b3
Fixes #8236 - Initialize bots from JSON data files (#8304) 2022-10-21 08:38:52 -07:00
Pere Miquel Brull
339f123f92
Fix #7477 - Paginate Airflow logs (#8145)
* Paginate last dag logs

* clean logic

* Prep pagination

* Prep pagination

* Format

* Update docker/local-metadata/docker-compose.yml

Co-authored-by: Nahuel <nahuel@getcollate.io>

* UI : Add pagination support for airflow logs

* Addressing review comments

Co-authored-by: Nahuel <nahuel@getcollate.io>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-10-21 17:32:56 +05:30
Sriharsha Chintalapani
1ef189540f
Revert "Fix ISSUE-3790: Remove unnecessary public modifiers (#8268)" (#8295)
This reverts commit 24f3ae763eaecc707d56fc62bea59b792eb7ad8b.
2022-10-20 10:04:35 -07:00
Shivam Shrey
24f3ae763e
Fix ISSUE-3790: Remove unnecessary public modifiers (#8268) 2022-10-20 08:58:52 -07:00
Teddy
342e4a24da
Fixes #8154 - Add collect endpoint for web analytic events (#8193)
* Initial implementation of report endpoint

* Modified model files

* Added migration

* Added tests for analytics/report endpoint

* Finalized implementation of the reports endpoint

* Fixed maven test failure

* Fixed java style

* Added policy for reportDefinition

* Fixed minor typos in code

* Added database migration for web analytic event type

* Added logic for collect endpoint

* Added JSON Schema for collect endpoint

* Added index on table columns + added resource description for web event entities

* Fixed policy tests

* Updated add web analytic function to return the body of the request + updated test

* Added referer URL to PageView event

* Changed collection name to align with API endpoint

* updated initSeed method to initializeEntity

* cleaned up PR by removing unused filed from branch

* Fixed requested changes from review

* Clean up names

* Fixed comma in SQL statement
2022-10-20 16:38:54 +02:00
Teddy
816111a671
Fixes #8121 -- Report endpoint for data insight (#8122)
* Initial implementation of report endpoint

* Modified model files

* Added migration

* Added tests for analytics/report endpoint

* Finalized implementation of the reports endpoint

* Fixed maven test failure

* Fixed java style

* Added policy for reportDefinition

* Fixed minor typos in code

* Renamed classes from AnalyticReport to ReportDefinition

* Java style formatting

* Update add report results to return request body

* Refactor report endpoint to simplify it and remove dependency on entity resource.

* Removed import *

* Fixed java style

* fixed typos in code

* Fix import path
2022-10-20 16:37:09 +02:00
Mohit Yadav
36bd428616
Fixes issue with Table name containing excluded endpoints (#8289)
* Endpoint should be more specific, causes issue in case we have enitities containing login, config etc in name

* Endpoint should be more specific, causes issue in case we have enitities containing login, config etc in name
2022-10-20 17:50:21 +05:30
Pere Miquel Brull
ee658bfb1f
Revert "Fixes #8236 - Initialize bots from JSON data files (#8237)" (#8281)
This reverts commit 9facb03cafce52735ef2b615d75205486985fc55.
2022-10-20 11:08:47 +02:00
Suresh Srinivas
9facb03caf
Fixes #8236 - Initialize bots from JSON data files (#8237) 2022-10-19 16:21:57 -07:00
Mohit Yadav
90ff4ac1ed
[BOT] User Cache for accessing Jwt For Bot (#8267)
* [BOT] User Cache for accessing Jwt For Bot

* [BOT] Review comments addressed
2022-10-20 00:00:43 +05:30
Suresh Srinivas
d4e463a789
Refactor initializing system entities from data files for better reusability (#8234) 2022-10-18 17:57:30 -07:00
Mohit Yadav
7af8c5418c
Adding Exception to Logger , and update batch jobs status as well (#8217) 2022-10-18 18:59:11 +05:30
Sriharsha Chintalapani
a4d8cbb818
Revert "ISSUE-1986: Replace System.out, System.err by a logger (#8139)" (#8163)
This reverts commit 1d3234ffe8ddf5e1f4f37a68be4226bf5ce5f67b.
2022-10-17 12:43:45 +02:00
Shivam Shrey
1d3234ffe8
ISSUE-1986: Replace System.out, System.err by a logger (#8139) 2022-10-14 06:39:00 +02:00
Mohit Yadav
71fcb17e59
Added Context Info in ES reindexing (#8123)
* Added Context Info in ES reindexing

* Fix File Formatting

* Update openmetadata-spec/src/main/resources/json/schema/settings/eventPublisherJob.json

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>

* revert python file

Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2022-10-13 18:32:06 +05:30
Mohit Yadav
cba9240fe9
[Email] Added explicit senderMail in configuration (#8126)
* [Email] Added explicit senderMail in configuration

* Fix failing tests

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-10-13 18:24:32 +05:30
mosiac1
d8a4cbeeb9
UI (#6189) | Update SearchResource and add new searchAPI frontend implementation (#8083)
* Update SearchResource and add new searchAPI frontend implementation

SearchResource now supports `query_filter` and `post_filter` as json
encoded Elasticsearch filters; `fetch_source` and
`include_source_fileds` for reducing response sizes.

* Remove

* Use async/await for searchAPI

* Move formatting to util functions
2022-10-13 09:30:56 +05:30
Parth Panchal
cf28194e1e
Fixed#8064: Count API should support include filter (#8079)
* Fixed#8064: Count API should support include filter

* Fixed#8064: Count API should support include filter
2022-10-11 16:51:54 +05:30
Mohit Yadav
5d0fcb7483
[BasicAuth] Init default password to "admin" in case of no passwords (#8072)
* Added websocket updates on jobchannel

* Default Password to be "admin" for uninitialized PWDs
2022-10-11 12:14:59 +05:30
Suresh Srinivas
f4acc4af1f
Fixes #8068 Remove ConstructorType annotation from Collection (#8069) 2022-10-10 23:32:48 -07:00
Suresh Srinivas
de7a616788
Add log layout for tests and also printing ISO8601 timestamp (#8071) 2022-10-10 18:42:59 -07:00
Nahuel
13b76dfd88
Remove UserRepository constructor which could cause NPE (#8047) 2022-10-10 10:50:01 +02:00
Sriharsha Chintalapani
ba08cfd115
Fix #8043: add log4j dependency openmetadata-service (#8044) 2022-10-08 13:55:34 +05:30