10673 Commits

Author SHA1 Message Date
Milan Bariya
65297e5b1d
Fix Sample data ingestion which have enum type columns (#10756)
* Fix Sample data ingestion which have enum type columns

* Change based on comments

* Change based on comments

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-28 11:52:12 +05:30
Ashish Gupta
21d5de6ee4
Fixes #10771 : Supported more properties in Object Store detail Page header (#10772)
* Added more properties in Object Store detail Page

* localisation key added

* bytes converted to kb and mb

* changes as per comments

* changes as per comments

* reverted last commit changes
2023-03-28 11:48:22 +05:30
Milan Bariya
3d1bbb1037
Add a configurationto skip Snowflake transient and tmp tables (#10665)
* Add a configurationto skip  Snowflake transient and tmp tables

* Fix Python checkstyle

* add separate query for transient tables

* Move skipTempTables into SnowflakeConnection

* Fix merge conflict

* change skip word to include

* Add title in json file

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-28 06:29:59 +02:00
Pere Miquel Brull
78d7dd8789
[WIP] - Test Connection - Prepare the new test connection ingestion+UI logic (#10660)
* Prepare the new test connection ingestion logic

* Update test assert

* Update Test Connection for SQA Sources

* Correct return type and method doc

* Handle decryption

* Non SQA Database Sources

* Add the run_automation script in ingestion-base

* Dashboard Test Connection Changes

* Pipeline, Messagin, MlModel & Metadata Sources

* ui: test connect flow-1

* Unmask connection parameters before sending to Ariflow

* ui: test connect flow-2

* Address review comments and pylint

* pytest fix

* ui: test connect flow-3 (refactoring and style fix)

* ui: test connect flow-4 (fix test connection status logic)

* sync local file

* ui: test connect flow-5 (fix lowercase issue and styling)

* ui: test connect flow-5 (show toast notifications)

* test: add unit test

* ui: test connect flow-5 (update service page test connection button)

* Databrick fix & pytest fix

* pylint

* Update test

* Fix merge

* S3 Test connection

* add style for mandatory step

* sync locales

* chore: add service name in workflow request

* Unmask using original service connection parameters

* Fix test connection unmasking

* Wrap inspector function to eliminate error outside test conn

* Fix linting

* fix:cy test

* Fix linting

* address comment

* refactor and fix connection type casing issue

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-03-28 06:29:13 +02:00
Mohit Yadav
13edb4ccdf
Return proper message in case invalid or missing token (#10788) 2023-03-27 21:56:39 +00:00
Mohit Yadav
29119dda89
Add docs for SAMl and AWS saml setup (#10787) 2023-03-28 01:05:21 +05:30
Shailesh Parmar
e2913c68a3
cypress: skipping tagsAddRemove spec for mlmodel entity (#10786) 2023-03-27 18:54:24 +00:00
Mohit Yadav
bdb6a16430
Fix Query Used In (#10784) 2023-03-27 23:03:33 +05:30
Teddy
31d4662abe
Fixes #10775 -- BQ partitionning for datetime dtype (#10780)
* fix: BQ partitionning for datetime dtype

* Update ingestion/src/metadata/profiler/orm/functions/datetime.py

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update ingestion/src/metadata/profiler/orm/functions/datetime.py

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update ingestion/src/metadata/profiler/orm/functions/datetime.py

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-27 15:28:34 +02:00
dependabot[bot]
4db2a74c26
chore(deps): bump spring-core from 5.3.24 to 5.3.26 (#10742)
Bumps [spring-core](https://github.com/spring-projects/spring-framework) from 5.3.24 to 5.3.26.
- [Release notes](https://github.com/spring-projects/spring-framework/releases)
- [Commits](https://github.com/spring-projects/spring-framework/compare/v5.3.24...v5.3.26)

---
updated-dependencies:
- dependency-name: org.springframework:spring-core
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nahuel <nahuel@getcollate.io>
2023-03-27 12:01:36 +00:00
Nahuel
da808bbe75
Fix: Update when GH actions are triggered (#10777) 2023-03-27 12:16:10 +02:00
Pere Miquel Brull
cc3d2b448b
Clean Airflow lineage test (#10745) 2023-03-27 11:46:34 +02:00
Ashish Gupta
c50390da80
Make the modal not closable on outside click (#10748)
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-03-27 12:57:59 +05:30
Ashish Gupta
f54efc649b
Fixes #10543 : Fix searchbar persistence issue after user get deleted by search. (#10753)
* Fix the searchbar on user page after delete action

* minor changes
2023-03-26 20:06:03 +05:30
Cristian Calugaru
3ffde9a293
Fixes #9064 - Ingestion for S3 bucket containers (#10515)
* Prep skeleton for object store connector

* First of of ingestion piepline for bucket containers.

* addressed PR review comments.

* swaped random with secrets due to security warning.

* linter issues.

* added prefix population for containers.

* more linting

* removed temp file.

* object store unit tests, PR comments and minor refactorings

* docs update and new TestConnection approach.

* linting

* removed isStructured field from containermeta schema

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-26 11:35:34 +02:00
Sachin Chaurasiya
0f3e929b75
doc: update doc for how to add new language support (#10734)
* doc: update doc for how to add new language support

* chore: minor fix

* address comments
2023-03-25 13:08:17 +05:30
Shailesh Parmar
5bbac9185c
fixed issue test result graph not displaying full axis values #10704 (#10715)
* fixed issue test result graph not displaying full axis values #10704

* added additional test case for digitFormatter function
2023-03-25 10:43:39 +05:30
Suresh Srinivas
ea8e1fda46
Fixes #10759 A Non-Privileged user can add new 'Roles' to Teams (#10760) 2023-03-24 19:23:22 -07:00
Chirag Madlani
c4c510443b
fix(test): redshiftWithDBT cypress failure for lineage (#10754)
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2023-03-25 00:30:17 +05:30
Nahuel
ef759c7e88
Fixes#8038: Change how status is handled after running workflow (#10710)
* Change how status is handled after running workflow

* Reset changes in config files

* Add auxiliary Summary class

* Improve failures handling

* Pylint error

* Pylint error

* Show result in table

* Add test

* Fix setup.py

* Add comments
2023-03-24 17:59:06 +01:00
Shailesh Parmar
0b50380b97
ui: unit test for query as entity ui changes (#10699)
* added edit support for description, tags and owner for queries

* updated style of left panel and center panel of query tab

* translation sync

* full entity ref

* added ui changes for query used in other table

* translation-sync

* move const value to constant file

* implemented permission API for edit actions

* implemented pagination

* sync-language

* translation sync

* implemented vote up and vote down functionality for query

* refactor code

* added filter dropdown for query entity

* translation-sync

* fixed unit test and refactor query files

* localization-sync

* fixed failing unit test and cursor blink issue in readOnly mode for editor

* refactor

* addressing comments

* updated edit condition

* updated description heading

* added unit test for QueryCardExtraOption component

* added unit test for tableQueryRightPanel

* updated floating query used by other table element

* translation sync

* updated default paging value

* updated unit test with new changes

* added unit test for QueryUsedByOtherTable component

* updated query card test

* updated expand collapse icon for query

* added border top for card footer

* addressing comment

* import fix

* fixed cypress for postgress

* removing py changes,

* updated unit test as per new change

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-03-24 09:32:51 -07:00
Mohit Yadav
11032a2d48
SAML feature (#10669)
* Added Saml Feature

* add localization

* fix tests

* added collate copyright

* remove thread.getType

* fixed keystore issues and updated review comments

* remove private key getter

* update test saml config

* update test saml config

* Update openmetadata-service/src/main/java/org/openmetadata/service/security/saml/OMMicrometerHttpFilter.java

Co-authored-by: Nahuel <nahuel@getcollate.io>

* review comments

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Nahuel <nahuel@getcollate.io>
2023-03-24 21:56:49 +05:30
Jan-Pieter van den Heuvel
c46732a4ff
Fix year of latest release in releases documentation page (#10664) 2023-03-24 17:08:21 +01:00
Sachin Chaurasiya
c6746507d9
fix#10662: Update tier functionality is not working as expected on the containers page (#10720)
* fix#10662: Update tier functionality is not working as expected on the containers page

* chore: remove unwanted change

* test: add unit test

* chore: add container entity icon

* chore: fix spacing issue

* chore: add support for lineage info drawer

* fix: locale missing key issue

* refactor: entity info drawer
2023-03-24 18:13:44 +05:30
Mohit Yadav
fb02cbfeed
fix failing Py test due to backend change (#10749) 2023-03-24 17:41:52 +05:30
Aniket Katkar
218e07808a
ui: explore page filters refactoring (#10524)
* changed behavior of quick dropdown filters to be same as left panel filters
- AND condition across different fields
- OR condition inside the same field filters
Applying left panel filters will now sent data as `post_filter` to prevent aggregations updating

* Added a separate API call without any filters to get all the left panel filter options

* separated url parameter variables to store 'facet' 'quick' and 'advanced search' filters
updated the logic to change facet filters according to selection.

* Fixed following bugs
- facet filters updating if filters are applied from quick dropdown or advanced search
- count for the filters were not updating with filter selection
- optimized API calls on explore page

* updated facet filters behavior where only counts will updated on filter selection instead of actual filters list

* Removed unnecessary code
Removed code smells

* worked on comments to improve the code

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-03-24 14:15:50 +05:30
Pere Miquel Brull
4913aeb53b
Docs - Add Airflow version support (#10707) 2023-03-24 06:31:38 +01:00
karanh37
f7dc89f727
feat: add glossary and tags to global search (#10677)
* feat: add global search for glossary and tags

* fix: missing localization keys

* fix: select issues

* feat: add glossary and tags in explore page

* fix: review comments

* fix: add missing icons

* feat: added glossary term summary panel

* feat: add tags panel

* fix: update glossary icon

* fix: separate out interfaces for summary entities

* fix: remove fragment

* fix: jest tests
2023-03-24 11:01:06 +05:30
Ashish Gupta
9e2b2d1926
Fix the breaking remove owner from Services (#10717) 2023-03-24 10:54:51 +05:30
Ashish Gupta
99cac0ea82
Fix the icon on lineage summary panel (#10733) 2023-03-24 10:54:28 +05:30
Shailesh Parmar
13e846d894
Feat: Query tab improvement (#10635)
* added edit support for description, tags and owner for queries

* updated style of left panel and center panel of query tab

* translation sync

* full entity ref

* added ui changes for query used in other table

* translation-sync

* move const value to constant file

* implemented permission API for edit actions

* implemented pagination

* sync-language

* translation sync

* implemented vote up and vote down functionality for query

* refactor code

* added filter dropdown for query entity

* translation-sync

* fixed unit test and refactor query files

* localization-sync

* fixed failing unit test and cursor blink issue in readOnly mode for editor

* refactor

* addressing comments

* updated edit condition

* updated description heading

* updated floating query used by other table element

* translation sync

* updated default paging value

* updated expand collapse icon for query

* added border top for card footer

* addressing comment

* import fix

* fixed cypress for postgress

* removing py changes,

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-03-24 10:29:50 +05:30
Shailesh Parmar
c188a63b51
fix: issue App Analytics summary not adjusting to the datetime [UI] #10571 (#10728) 2023-03-24 10:29:21 +05:30
Sriharsha Chintalapani
d911e0caba
Add query index (#10682)
* Add query index

* add QueryUsage ChangeEvent

* fix test

* Add query index

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-03-23 21:26:27 -07:00
Sachin Chaurasiya
cc27a6ce0d
chore(ui): Bring back the count for services, teams, and users, and remove the count for glossaries on the landing page. (#10719) 2023-03-24 09:23:34 +05:30
Suresh Srinivas
b80fc39632
Fixes #10740 Inherited ownership must not overwrite manually entered ownership (#10741) 2023-03-23 17:15:55 -07:00
NiharDoshi99
bfa0905093
update druid docs (#10735) 2023-03-23 22:16:14 +05:30
Milan Bariya
48cc249b11
Fix glue ingestion issue (#10732) 2023-03-23 20:53:43 +05:30
Ashish Gupta
c24710a811
UI : Fix the overflow issue of tags (#10673)
* Fix the overflow issue of tags

* Changes as per comments
2023-03-23 15:45:23 +05:30
Onkar Ravgan
c9feba826f
Kinesis bug fixed (#10578)
* Kinesis bug fixed

* Added sample data fix

* Fixed typo

* Added constansts

* Added try catch for sample data

* Added fixes on review comments

* Modified models

* Added sample data py models

* Fix pylint

* fixed code smell

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-23 13:38:56 +05:30
Mohit Yadav
acb0e71a3e
Fix Email sent everytime a admin owns an Entity during startup (#10697)
* Fix Email sent everytime a admin owns an Entity during startup

* fix
2023-03-23 11:58:54 +05:30
Teddy
3d8e7e6d41
Fixes #10290 - NUMERIC type assigned the wrong dtype in converter.py (#10708)
* fix: profiler refactor

* fix: catch division by zero error

* fix: instantiated new column object from name to delegate type handler back to dbapi

* fix: reverted columns instantiation and updated NUMBER type to NUMERIC

* fix: updated doc string for process_pii_sensitive method
2023-03-23 07:22:19 +01:00
Mayur Singal
bbce9c5aa4
Fix #10489: Handle unknown data types & store raw data type (#10563) 2023-03-23 11:41:29 +05:30
Teddy
b2e1eed842
doc: added best practice section to profiler (#10712)
* doc: added best practice section to profiler

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-22 19:36:42 +01:00
Pere Miquel Brull
51f4e0b170
UI - Add ES log retrieval (#10667) 2023-03-22 17:52:58 +01:00
Pere Miquel Brull
ba7690f14e
Update PR template (#10700)
* Update PR template

* Update PR template
2023-03-22 16:50:13 +01:00
Ashish Gupta
f15a752706
UI : Added show deleted table in Database schema level (#10672)
* Added show deleted table in Database scheam level

* Changes as per comments

* fix cypress issue

* make switch label static

* other language

* fix cypress issue
2023-03-22 16:16:08 +05:30
Pere Miquel Brull
ebbd611825
Update counts after #10265 (#10693) 2023-03-22 06:32:18 +01:00
Pere Miquel Brull
e2a2bcc8da
Fix search by email index keyword (#10698)
* Fix search by email index keyword

* Fix search by email index keyword
2023-03-21 20:50:47 -07:00
Mayur Singal
9f99296ea0
Bigquery Docs Fix (#10692)
* Bigquery Docs Fix

* typo fix

* keycloak sso

* GCP Create Custom Role
2023-03-21 16:55:57 +01:00
Suresh Srinivas
b8e0ae489a
Fixes #10679 - Add policy functions inAnyTeam and hasAnyRole (#10680) 2023-03-21 07:24:41 -07:00