5980 Commits

Author SHA1 Message Date
NiharDoshi99
46afe69811
improvement in pii tagging (#10696)
* improvement in pii tagging

* fix conflict and changes as per comment

* Added confidence field

* changes as per comments

* Apply suggestions from code review

Co-authored-by: Teddy <teddy.crepineau@gmail.com>

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2023-03-28 19:37:48 +05:30
Pere Miquel Brull
4853dfc6c8
Deprecate test_connection endpoint (#10798) 2023-03-28 15:52:36 +02:00
Teddy
fc9b64b52a
Fix histogram labels (#10797)
* fix: added logic to format bin labels for histogram metric

* fix: histogram labels

* fix: added types for arguments
2023-03-28 12:59:45 +02:00
Anuj359
52c9a0f0df
Limiting cypress actions on openmetadata-ui (#10796)
* Limiting cypress actions on openmetadata-ui

* Limiting cypress actions on openmetadata-ui

* Updating cypress actions pull_request_target

---------

Co-authored-by: Anuj Joshi <anuj@Anujs-Air.fritz.box>
2023-03-28 15:38:39 +05:30
Chirag Madlani
bae95c5314
feat(ui): consistent user dropdown for owner and reviewers (#10617)
* feat(ui): consistent user dropdown for owner and reviewers

* feat(ui): consistent user dropdown for owner and reviewers

* make component working with pages
update styles
add few more components like count details, color update on tags

* remove AddUserModal component

* fix owner edit for netity page

* add tests for selectable list

* remove reference for OwnerWidgetWrapper
update tests to reduce console error on test run

* remove count

* fix sonarcloud and ui

* remove OwnerWidget

* minor fixes

* verify reviewer and owner modal behavior

* fix tests

* fix minor UI glitches

* fix unit tests

* fix cypress

* fix cypress failures

* fix search and glossary cypress failure

* minor fix

* fix remaining cypress tests

* remove intercept for terms

* fix issues

* address comments

* skip falky test to avoid failures
2023-03-28 14:39:41 +05:30
Parth Panchal
8f424e1787
Fixed#10565: Make Prometheus Mappings Configurable (#10671)
* Fixed#10565: Make Prometheus Mappings Configurable

* Fixed#10565: Make Prometheus Mappings Configurable

* Fixed#10565: Make Prometheus Mappings Configurable

* Addressing comments and fixed nullpointer in tests

* Fixed#10565: Make Prometheus Mappings Configurable

* Addressing comments

* Fixing tests
2023-03-28 12:51:52 +05:30
Shailesh Parmar
4b260f1736
fixes 10357: Update Partitionning Setting Flow for Profiler (#10743)
* initial commit for #10357

* added form based on partition condition

* localization sync

* added unit test

* updated sql editor with common component

* updated form based on switch

* addressing comment

* added form type and provided to form instance

* added default value for partitionValues in initialValue field
2023-03-28 12:13:16 +05:30
Milan Bariya
65297e5b1d
Fix Sample data ingestion which have enum type columns (#10756)
* Fix Sample data ingestion which have enum type columns

* Change based on comments

* Change based on comments

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-28 11:52:12 +05:30
Ashish Gupta
21d5de6ee4
Fixes #10771 : Supported more properties in Object Store detail Page header (#10772)
* Added more properties in Object Store detail Page

* localisation key added

* bytes converted to kb and mb

* changes as per comments

* changes as per comments

* reverted last commit changes
2023-03-28 11:48:22 +05:30
Milan Bariya
3d1bbb1037
Add a configurationto skip Snowflake transient and tmp tables (#10665)
* Add a configurationto skip  Snowflake transient and tmp tables

* Fix Python checkstyle

* add separate query for transient tables

* Move skipTempTables into SnowflakeConnection

* Fix merge conflict

* change skip word to include

* Add title in json file

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-28 06:29:59 +02:00
Pere Miquel Brull
78d7dd8789
[WIP] - Test Connection - Prepare the new test connection ingestion+UI logic (#10660)
* Prepare the new test connection ingestion logic

* Update test assert

* Update Test Connection for SQA Sources

* Correct return type and method doc

* Handle decryption

* Non SQA Database Sources

* Add the run_automation script in ingestion-base

* Dashboard Test Connection Changes

* Pipeline, Messagin, MlModel & Metadata Sources

* ui: test connect flow-1

* Unmask connection parameters before sending to Ariflow

* ui: test connect flow-2

* Address review comments and pylint

* pytest fix

* ui: test connect flow-3 (refactoring and style fix)

* ui: test connect flow-4 (fix test connection status logic)

* sync local file

* ui: test connect flow-5 (fix lowercase issue and styling)

* ui: test connect flow-5 (show toast notifications)

* test: add unit test

* ui: test connect flow-5 (update service page test connection button)

* Databrick fix & pytest fix

* pylint

* Update test

* Fix merge

* S3 Test connection

* add style for mandatory step

* sync locales

* chore: add service name in workflow request

* Unmask using original service connection parameters

* Fix test connection unmasking

* Wrap inspector function to eliminate error outside test conn

* Fix linting

* fix:cy test

* Fix linting

* address comment

* refactor and fix connection type casing issue

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-03-28 06:29:13 +02:00
Mohit Yadav
13edb4ccdf
Return proper message in case invalid or missing token (#10788) 2023-03-27 21:56:39 +00:00
Mohit Yadav
29119dda89
Add docs for SAMl and AWS saml setup (#10787) 2023-03-28 01:05:21 +05:30
Shailesh Parmar
e2913c68a3
cypress: skipping tagsAddRemove spec for mlmodel entity (#10786) 2023-03-27 18:54:24 +00:00
Mohit Yadav
bdb6a16430
Fix Query Used In (#10784) 2023-03-27 23:03:33 +05:30
Teddy
31d4662abe
Fixes #10775 -- BQ partitionning for datetime dtype (#10780)
* fix: BQ partitionning for datetime dtype

* Update ingestion/src/metadata/profiler/orm/functions/datetime.py

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update ingestion/src/metadata/profiler/orm/functions/datetime.py

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update ingestion/src/metadata/profiler/orm/functions/datetime.py

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-27 15:28:34 +02:00
dependabot[bot]
4db2a74c26
chore(deps): bump spring-core from 5.3.24 to 5.3.26 (#10742)
Bumps [spring-core](https://github.com/spring-projects/spring-framework) from 5.3.24 to 5.3.26.
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v5.3.24...v5.3.26)

---
updated-dependencies:
- dependency-name: org.springframework:spring-core
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nahuel <nahuel@getcollate.io>
2023-03-27 12:01:36 +00:00
Nahuel
da808bbe75
Fix: Update when GH actions are triggered (#10777) 2023-03-27 12:16:10 +02:00
Pere Miquel Brull
cc3d2b448b
Clean Airflow lineage test (#10745) 2023-03-27 11:46:34 +02:00
Ashish Gupta
c50390da80
Make the modal not closable on outside click (#10748)
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-03-27 12:57:59 +05:30
Ashish Gupta
f54efc649b
Fixes #10543 : Fix searchbar persistence issue after user get deleted by search. (#10753)
* Fix the searchbar on user page after delete action

* minor changes
2023-03-26 20:06:03 +05:30
Cristian Calugaru
3ffde9a293
Fixes #9064 - Ingestion for S3 bucket containers (#10515)
* Prep skeleton for object store connector

* First of of ingestion piepline for bucket containers.

* addressed PR review comments.

* swaped random with secrets due to security warning.

* linter issues.

* added prefix population for containers.

* more linting

* removed temp file.

* object store unit tests, PR comments and minor refactorings

* docs update and new TestConnection approach.

* linting

* removed isStructured field from containermeta schema

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-26 11:35:34 +02:00
Sachin Chaurasiya
0f3e929b75
doc: update doc for how to add new language support (#10734)
* doc: update doc for how to add new language support

* chore: minor fix

* address comments
2023-03-25 13:08:17 +05:30
Shailesh Parmar
5bbac9185c
fixed issue test result graph not displaying full axis values #10704 (#10715)
* fixed issue test result graph not displaying full axis values #10704

* added additional test case for digitFormatter function
2023-03-25 10:43:39 +05:30
Suresh Srinivas
ea8e1fda46
Fixes #10759 A Non-Privileged user can add new 'Roles' to Teams (#10760) 2023-03-24 19:23:22 -07:00
Chirag Madlani
c4c510443b
fix(test): redshiftWithDBT cypress failure for lineage (#10754)
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2023-03-25 00:30:17 +05:30
Nahuel
ef759c7e88
Fixes#8038: Change how status is handled after running workflow (#10710)
* Change how status is handled after running workflow

* Reset changes in config files

* Add auxiliary Summary class

* Improve failures handling

* Pylint error

* Pylint error

* Show result in table

* Add test

* Fix setup.py

* Add comments
2023-03-24 17:59:06 +01:00
Shailesh Parmar
0b50380b97
ui: unit test for query as entity ui changes (#10699)
* added edit support for description, tags and owner for queries

* updated style of left panel and center panel of query tab

* translation sync

* full entity ref

* added ui changes for query used in other table

* translation-sync

* move const value to constant file

* implemented permission API for edit actions

* implemented pagination

* sync-language

* translation sync

* implemented vote up and vote down functionality for query

* refactor code

* added filter dropdown for query entity

* translation-sync

* fixed unit test and refactor query files

* localization-sync

* fixed failing unit test and cursor blink issue in readOnly mode for editor

* refactor

* addressing comments

* updated edit condition

* updated description heading

* added unit test for QueryCardExtraOption component

* added unit test for tableQueryRightPanel

* updated floating query used by other table element

* translation sync

* updated default paging value

* updated unit test with new changes

* added unit test for QueryUsedByOtherTable component

* updated query card test

* updated expand collapse icon for query

* added border top for card footer

* addressing comment

* import fix

* fixed cypress for postgress

* removing py changes,

* updated unit test as per new change

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-03-24 09:32:51 -07:00
Mohit Yadav
11032a2d48
SAML feature (#10669)
* Added Saml Feature

* add localization

* fix tests

* added collate copyright

* remove thread.getType

* fixed keystore issues and updated review comments

* remove private key getter

* update test saml config

* update test saml config

* Update openmetadata-service/src/main/java/org/openmetadata/service/security/saml/OMMicrometerHttpFilter.java

Co-authored-by: Nahuel <nahuel@getcollate.io>

* review comments

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Nahuel <nahuel@getcollate.io>
2023-03-24 21:56:49 +05:30
Jan-Pieter van den Heuvel
c46732a4ff
Fix year of latest release in releases documentation page (#10664) 2023-03-24 17:08:21 +01:00
Sachin Chaurasiya
c6746507d9
fix#10662: Update tier functionality is not working as expected on the containers page (#10720)
* fix#10662: Update tier functionality is not working as expected on the containers page

* chore: remove unwanted change

* test: add unit test

* chore: add container entity icon

* chore: fix spacing issue

* chore: add support for lineage info drawer

* fix: locale missing key issue

* refactor: entity info drawer
2023-03-24 18:13:44 +05:30
Mohit Yadav
fb02cbfeed
fix failing Py test due to backend change (#10749) 2023-03-24 17:41:52 +05:30
Aniket Katkar
218e07808a
ui: explore page filters refactoring (#10524)
* changed behavior of quick dropdown filters to be same as left panel filters
- AND condition across different fields
- OR condition inside the same field filters
Applying left panel filters will now sent data as `post_filter` to prevent aggregations updating

* Added a separate API call without any filters to get all the left panel filter options

* separated url parameter variables to store 'facet' 'quick' and 'advanced search' filters
updated the logic to change facet filters according to selection.

* Fixed following bugs
- facet filters updating if filters are applied from quick dropdown or advanced search
- count for the filters were not updating with filter selection
- optimized API calls on explore page

* updated facet filters behavior where only counts will updated on filter selection instead of actual filters list

* Removed unnecessary code
Removed code smells

* worked on comments to improve the code

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-03-24 14:15:50 +05:30
Pere Miquel Brull
4913aeb53b
Docs - Add Airflow version support (#10707) 2023-03-24 06:31:38 +01:00
karanh37
f7dc89f727
feat: add glossary and tags to global search (#10677)
* feat: add global search for glossary and tags

* fix: missing localization keys

* fix: select issues

* feat: add glossary and tags in explore page

* fix: review comments

* fix: add missing icons

* feat: added glossary term summary panel

* feat: add tags panel

* fix: update glossary icon

* fix: separate out interfaces for summary entities

* fix: remove fragment

* fix: jest tests
2023-03-24 11:01:06 +05:30
Ashish Gupta
9e2b2d1926
Fix the breaking remove owner from Services (#10717) 2023-03-24 10:54:51 +05:30
Ashish Gupta
99cac0ea82
Fix the icon on lineage summary panel (#10733) 2023-03-24 10:54:28 +05:30
Shailesh Parmar
13e846d894
Feat: Query tab improvement (#10635)
* added edit support for description, tags and owner for queries

* updated style of left panel and center panel of query tab

* translation sync

* full entity ref

* added ui changes for query used in other table

* translation-sync

* move const value to constant file

* implemented permission API for edit actions

* implemented pagination

* sync-language

* translation sync

* implemented vote up and vote down functionality for query

* refactor code

* added filter dropdown for query entity

* translation-sync

* fixed unit test and refactor query files

* localization-sync

* fixed failing unit test and cursor blink issue in readOnly mode for editor

* refactor

* addressing comments

* updated edit condition

* updated description heading

* updated floating query used by other table element

* translation sync

* updated default paging value

* updated expand collapse icon for query

* added border top for card footer

* addressing comment

* import fix

* fixed cypress for postgress

* removing py changes,

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-03-24 10:29:50 +05:30
Shailesh Parmar
c188a63b51
fix: issue App Analytics summary not adjusting to the datetime [UI] #10571 (#10728) 2023-03-24 10:29:21 +05:30
Sriharsha Chintalapani
d911e0caba
Add query index (#10682)
* Add query index

* add QueryUsage ChangeEvent

* fix test

* Add query index

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-03-23 21:26:27 -07:00
Sachin Chaurasiya
cc27a6ce0d
chore(ui): Bring back the count for services, teams, and users, and remove the count for glossaries on the landing page. (#10719) 2023-03-24 09:23:34 +05:30
Suresh Srinivas
b80fc39632
Fixes #10740 Inherited ownership must not overwrite manually entered ownership (#10741) 2023-03-23 17:15:55 -07:00
NiharDoshi99
bfa0905093
update druid docs (#10735) 2023-03-23 22:16:14 +05:30
Milan Bariya
48cc249b11
Fix glue ingestion issue (#10732) 2023-03-23 20:53:43 +05:30
Ashish Gupta
c24710a811
UI : Fix the overflow issue of tags (#10673)
* Fix the overflow issue of tags

* Changes as per comments
2023-03-23 15:45:23 +05:30
Onkar Ravgan
c9feba826f
Kinesis bug fixed (#10578)
* Kinesis bug fixed

* Added sample data fix

* Fixed typo

* Added constansts

* Added try catch for sample data

* Added fixes on review comments

* Modified models

* Added sample data py models

* Fix pylint

* fixed code smell

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-23 13:38:56 +05:30
Mohit Yadav
acb0e71a3e
Fix Email sent everytime a admin owns an Entity during startup (#10697)
* Fix Email sent everytime a admin owns an Entity during startup

* fix
2023-03-23 11:58:54 +05:30
Teddy
3d8e7e6d41
Fixes #10290 - NUMERIC type assigned the wrong dtype in converter.py (#10708)
* fix: profiler refactor

* fix: catch division by zero error

* fix: instantiated new column object from name to delegate type handler back to dbapi

* fix: reverted columns instantiation and updated NUMBER type to NUMERIC

* fix: updated doc string for process_pii_sensitive method
2023-03-23 07:22:19 +01:00
Mayur Singal
bbce9c5aa4
Fix #10489: Handle unknown data types & store raw data type (#10563) 2023-03-23 11:41:29 +05:30
Teddy
b2e1eed842
doc: added best practice section to profiler (#10712)
* doc: added best practice section to profiler

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-22 19:36:42 +01:00