5980 Commits

Author SHA1 Message Date
Teddy
dcd0bbb566
Fixes 8470 -- Implements ES aggregation for web analytic data (#8566)
* Moved webanalytics type in its own folder

* Added data insight chart api endpoint

* Jave formatting

* Added resource descriptor

* Added metadata entity endpoint

* Added aggregation endpoint for dataInsight

* Fix tag name

* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight

* added domo to test subpackage

* cleaned up branch by removing commit from issue-8353 that were not merged in main

* Added web analytics data refinement

* Added get_status function

* Added from __futur__ for typing

* Added dailyActiveUsers aggregation

* Added page views entities aggregation and active users aggregation
2022-11-08 17:25:31 +01:00
Milan Bariya
4d16be2608
Fix: Hive connection issue (#8583)
* Fix: Hive connection issue

* Change based on comments
2022-11-08 16:22:37 +01:00
Teddy
2ffb88f6aa
Added bit -> binary column type conversion for mssql (#8581) 2022-11-08 16:21:55 +01:00
michizhou
da06d183db
feat: add QuickSight connector (#8442) 2022-11-08 19:54:49 +05:30
Shailesh Parmar
1538972341
UI: Fixed Data is getting round off at the time of formatting in en-US (#8576) 2022-11-08 16:55:05 +05:30
Aniket Katkar
00bac5c929
Fix #8481 (UI): Changed how team assets are shown similar to explore page (#8526)
* Changed how entity details are shown in 'My Data' and 'Following' tabs on user details page

* Changed how team assets are shown similar to explore page

* removed isUndefined as isEmpty will cover that

* - Renamed 'GlossaryTermAssets' interface to 'AssetsDataType' to use it commonly and made the required changes for where it was used
- Code optimisation to reuse repeated code blocks
- Localization changes for 'UserPage'

* - Code optimisation by using common util functions
- Localization changes for 'TeamsPage' and 'TeamDetailsV1'
- Fixed spelling mistakes
2022-11-08 16:42:00 +05:30
Mayur Singal
710e3d785f
Fix #8495: Update pipeline sources for pipeline as edge lineage (#8556) 2022-11-08 14:13:49 +05:30
Shailesh Parmar
abc9247899
UI: Updated lineage UI as per new mock (#8568)
* UI: Updated lineage UI as per new mock

* fixed function icon issue in edit mode

* removed unit test related to pipeline label name

* center nodes on expand/collapse column lineage view

* addressing comments
2022-11-08 13:48:28 +05:30
Abhishek Pandey
73b370b5e2
schema-filter-added-in-datalake-for-bucket (#8516)
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2022-11-08 10:57:16 +05:30
Mayur Singal
83102d6697
Fix #8455: fix dbt test errors when run not passed (#8545) 2022-11-08 10:50:27 +05:30
Sachin Chaurasiya
ef579ce9b4
Fix #8540 Session ID does not reset for web analytic (#8562) 2022-11-07 23:21:40 +05:30
Aniket Katkar
ecee66fce9
Fixed wrong field names showing in the validation messages for add and edit announcement modal. (#8564) 2022-11-07 23:07:31 +05:30
Teddy
b44972ef60
Fixes #8470 - Implements refinement functions for web analytics events (#8528)
* Moved webanalytics type in its own folder

* Added data insight chart api endpoint

* Jave formatting

* Added resource descriptor

* Added metadata entity endpoint

* Added aggregation endpoint for dataInsight

* Fix tag name

* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight

* added domo to test subpackage

* cleaned up branch by removing commit from issue-8353 that were not merged in main

* Added web analytics data refinement

* Added get_status function

* Added from __futur__ for typing

* Fixed typos brought up during reviews
2022-11-07 17:08:20 +01:00
Nahuel
6e1f4fdd92
Schedule security scan every Monday midnight (#8558) 2022-11-07 15:01:01 +01:00
Aniket Katkar
af7f8b691a
Fixed issue where wrong data was being sent to the PATCH API call for table resulting in error while removing tags for columns (#8557) 2022-11-07 19:30:28 +05:30
Aniket Katkar
e44fafad88
Fix #7737 (UI) : Changed how entity details are shown in 'My Data' and 'Following' tabs on user details page (#8524)
* Changed how entity details are shown in 'My Data' and 'Following' tabs on user details page

* removed isUndefined as isEmpty will cover that

* - Renamed 'GlossaryTermAssets' interface to 'AssetsDataType' to use it commonly and made the required changes for where it was used
- Code optimisation to reuse repeated code blocks
- Localization changes for 'UserPage'
2022-11-07 19:21:12 +05:30
Shailesh Parmar
5fee438b87
Feat UI: Added Tracing for lineage and Column lineage (#8542)
* Feat: Added tracing feature for lineage

* fixed stack overflow issue for tracing logic

* restrict highlighing column lineage on click of node

* added support for column lineage tracing

* added unit test for util function

* updated icon color as per font color
2022-11-07 17:00:51 +05:30
Pere Miquel Brull
8d15985117
Docs - Run connectors from MWAA (#8548)
* Run connectors from MWAA

* Run connectors from MWAA

* Update url

* Add backend conn

* Add MWAA extraction
2022-11-07 09:43:42 +01:00
Pere Miquel Brull
b23d8353fc
Docs best practices (#8541) 2022-11-07 09:27:53 +01:00
Vj-L
987e3a66fb
Updated the document for docker deployment (#8527)
* Updated the document for docker deployment

* Updated the document for docker deployment

* Updated the document for docker deployment

Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
2022-11-07 11:00:00 +05:30
Suresh Srinivas
292672a970
Clean up code warnings (#8532) 2022-11-04 11:24:46 -07:00
Pere Miquel Brull
a7347e1873
Fix Dagster migration script (#8539) 2022-11-04 09:43:52 +01:00
Sachin Chaurasiya
833883e24a
Fix data insight Json Schema filename and content (#8534)
* Fix Json Schema filename

* Remove formatting

* FIx required property and change data type from array to object

* added items to array type

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2022-11-04 14:07:36 +05:30
Sriharsha Chintalapani
5c5514a8b9
Fix mysql migration script syntax (#8536) 2022-11-04 08:31:33 +01:00
Sachin Chaurasiya
aa7c31ec31
Fix #8490 Wrong message in the required message on Policy page (#8522)
* Fix #8490 Wrong message in the required message on Policy page

* Fix unit test

* rename functions to conditionFunctions

* Address review comments

* Address review comments

* Fix cypress test data test id.

* FIx unit test
2022-11-04 11:20:36 +05:30
Mayur Singal
a49c261cbe
Add Queued Status in Ingestion Pipelines (#8529) 2022-11-04 11:19:50 +05:30
NiharDoshi99
1c1f8660f2
Fixed: dagster to connect to cloud (#8387)
* Fixed: dagster to connect to cloud

* Fixed: changed test_cases

* Fixed: lineage between the assets and jobs

* Changes as per comments

* Fix python checkstyle

* Fix as per comments

* Fix added migration scripts

* minor fixes
2022-11-03 18:19:50 +01:00
Tushar Mittal
6f2c93089c
feat: add SageMaker connector (#8435)
* feat: add sagemaker connector

Signed-off-by: Tushar Mittal <chiragmittal.mittal@gmail.com>

* fix: fix linting errors and update imports

Signed-off-by: Tushar Mittal <chiragmittal.mittal@gmail.com>

* test: add unit tests for sagemake source

Signed-off-by: Tushar Mittal <chiragmittal.mittal@gmail.com>

Signed-off-by: Tushar Mittal <chiragmittal.mittal@gmail.com>
2022-11-03 18:19:20 +01:00
Aniket Katkar
d93b46ef31
(UI) #7790 : Added support to preserve search text for users and admins list page (#8478)
* Added support for preserving search text for users and admins list page

* added comments for better understanding of code

* Used URLSearchParams for adding parameters to url for userListPage

* renamed search parameter from 'name' to 'search'
2022-11-03 22:32:07 +05:30
Milan Bariya
6a7704a746
Fix: Filter Databricks history by date (#8520)
* Fix: Filter Databricks history by date

* Fix: Filter Databricks history by date

* Change based on comment
2022-11-03 22:08:56 +05:30
Milan Bariya
158bd4b9cd
Add: databricks troubleshoot page (#8523) 2022-11-03 14:27:50 +01:00
Shailesh Parmar
0f0489a7d4
Feat: Added pipeline as edge from ui (#8503)
* Feat: Added pipeline as edge from ui

* fixed failing unit test

* removed pipeline

* added unit test for util function
2022-11-03 17:46:49 +05:30
Pere Miquel Brull
c326cc1370
Move logging to DEBUG (#8519) 2022-11-03 13:04:33 +01:00
Chirag Madlani
60a97ea8bc
fix(ui): redirect to sign-in when user is not authorized (#8512) 2022-11-03 16:08:50 +05:30
Teddy
f4cc2b57a7
Fixes #8353 - Implement Aggregation Endpoint for dataInsight (#8464)
* Moved webanalytics type in its own folder

* Added data insight chart api endpoint

* Jave formatting

* Added resource descriptor

* Added metadata entity endpoint

* Added aggregation endpoint for dataInsight

* Fix tag name

* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight

* Fix pageViewEvent import issue

* Changed dataType to PERCENTAGE and NUMBER

* Cleaned up PR by removing metadata resource and ingestionPipeline sink logic

* Update openmetadata-service/src/main/resources/json/data/dataInsight/totalEntitiesByTier.json

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>

* Update openmetadata-service/src/main/resources/json/data/dataInsight/totalEntitiesByType.json

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-11-03 14:51:14 +05:30
Mayur Singal
1386b43607
Fix #6141: Ingestion Pipeline Status Updates (#8216)
* Fix #6141: Ingestion Pipeline Status Updates

* List Pipeline Status API & Improvements

* Rename State field to PipelineState in UI

* Convert Pipeline Status array to single object

* fix braking UI

* Rebase Fixes

* Profiler, TestStuite & DataInsigts Pipeline

* py_format

* fix logs page not loading
add pipelineStatus endpoint

* fix recent run changes

* Fix Tests

* address review comments for ui

* fix failing checks

* fix unit tests

* fix cypress test

* remove loader test as not using it

* wait for API when we click on the tab

* fix cypress waitFor api

* fix failed cypress tests

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2022-11-03 14:37:26 +05:30
Onkar Ravgan
8273197b48
Fix #7767: Added new PowerBi APIs and lineage (#8485)
* Added new APIs and lineage

* Addressed review comments

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-11-03 08:31:31 +01:00
Mihir Kulkarni
0bc2d92aba
#7994 Updating password error message to mention numbers. (#8103) 2022-11-03 13:01:14 +05:30
Sachin Chaurasiya
63ceeb2a33
Fix #8497 Update Table styling (#8498) 2022-11-03 10:52:47 +05:30
Shailesh Parmar
91b395bc17
Fixed Cypress flakiness for add owner test in service spec (#8504) 2022-11-03 10:10:32 +05:30
Suresh Srinivas
d47f5738cb
Add support for system provided Tags, Glossaries, Policies, and Roles #8484 (#8508) 2022-11-02 14:56:31 -07:00
Sachin Chaurasiya
f3b9e2c59c
Fix UI(#8467) : Only allow admins to create and update bot and user entities (#8491)
* FIx UI : Only allow admins to create and update bot and user entities. #8467

* Add helper text for ADMIN_ONLY_ACTION
2022-11-02 22:43:46 +05:30
Pere Miquel Brull
370288f1ff
Airflow connections docs (#8493) 2022-11-02 12:07:54 +01:00
Tushar Mittal
2a65df5f36
feat: add Kinesis connector (#8452)
Signed-off-by: Tushar Mittal <chiragmittal.mittal@gmail.com>
2022-11-02 16:12:45 +05:30
Pere Miquel Brull
119763afc4
Bump datamodel-code-generator (#8492) 2022-11-02 10:31:44 +01:00
Mayur Singal
fa68a1f18f
Fix #6754: Added User Details for Usage (#8449) 2022-11-02 14:07:41 +05:30
Chirag Madlani
ef5108c47b
chore(ui): remove unused files (#8487) 2022-11-02 13:55:42 +05:30
Sachin Chaurasiya
e7e646e9a8
Revert #7292 : Add ConditionalAllow check for Create Operation Permission (#8488) 2022-11-02 11:57:42 +05:30
Milan Bariya
fbe9e59dcd
Fix: Athena connector fails to ingest complex types (#8453)
* Fix: Athena connector fails to ingest complex types

* Fix: Athena connector fails to ingest complex types

* Fix: Athena connector fails to ingest complex types

* Fix: Change based on comments
2022-11-02 07:06:39 +01:00
Suresh Srinivas
8a008022c8
Fixes #7004 Backend : conditionAllow should not be applicable to create operation (#8483) 2022-11-01 12:46:18 -07:00