797 Commits

Author SHA1 Message Date
Ayush Shah
0f9c2c2164
Add Labels on DatabaseSchema Level (#10547) 2023-03-13 14:13:51 +00:00
Teddy
7ec667808c
Fixes #10498 - Add support for nifi client certificate auth. (#10499)
* feat: Added nifi support for client cert auth

* feat: fix code linting

* feat: addressed comments for nifi connections

* feat: fixed linting
2023-03-10 11:09:40 +01:00
NiharDoshi99
9b0b06fe25
Added flag for overriding owners (#10425)
* added flag for overriding owners

* docs update

* adding owners in patch call

* adding owners in patch call

* fix: python test case

* changes as per comment

* changes as per comment

* ui changes
2023-03-10 15:01:56 +05:30
Pere Miquel Brull
81dec813a0
Don't store the OM connection in the Ingestion Pipeline or Workflow (#10448)
* Do not store OM connection

* Migration to remove the server connection

* Update tests

* Add workflow masking and secrets manager

* Fix failing test

---------

Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
2023-03-09 17:32:40 +01:00
NiharDoshi99
ddbc0311e8
fix: mark table deleted dependencies (#10476) 2023-03-09 09:15:05 +00:00
Suresh Srinivas
4c6d184ef5
Fixes #10480 Glossary rename results in rename of Classification with… (#10486)
* Fixes #10480 Glossary rename results in rename of Classification with the same name

* Rename TagSource Tag to Classification
2023-03-09 00:30:36 -08:00
Mayur Singal
c199f13ed0
Fix #10402: Add support for AssumeRole for AWS (#10417) 2023-03-08 15:43:33 +05:30
Pere Miquel Brull
050da1e2d1
Add service type to container (#10441) 2023-03-06 14:44:30 +01:00
Pere Miquel Brull
477a5223eb
Fix #10401 - Add Automations Workflow Resource & PUT service test connection result (#10437)
Fix #10401 - Add Automations Workflow Resource & PUT service test connection result (#10437)
2023-03-06 14:44:16 +01:00
Mohit Yadav
df2001418e
add tags to schema Level (#10381)
=
2023-03-06 12:45:07 +05:30
Pere Miquel Brull
1e3652d35e
Fix #10382 - Test Connection Definitions, Results and Resource (#10383)
* Prep the skeleton

* Add DAO and tables

* Add Test Connection Definition Resource

* Format

* Add ResourceDescriptors & missing props

* Simplify APIs to scope for GET and LIST
2023-03-05 10:17:08 +01:00
Teddy
5208b6f684
Fixes #4368 - Add Histogram Metric (#10422) 2023-03-03 21:56:32 +01:00
Nahuel
247016307d
Fix#8648: Mask sensitive info from API responses (#10307)
* Mask sensitive info from API responses

* Rename converter classes

* Add missing Java classes from JSON schemas and class converters

* Update test service connection schema

* Update datalakeConnection JSON schema and fix some tests

* Fix AlertsRuleEvaluatorResourceTest and minor error in run_local_docker.sh

* Fix Pipeline and Database service tests

* Minor refactor

* Fix CsvUtilTest

* Fix EventMonitorFactoryTest

* Fix CloudWatchEventMonitorTest

* Update datalake metadata

* Update bigquery metadata

* Fix test connection functionality

* Fix OMeta service api test

* Update gcsValues title and revert changes in GH actions

* Mask sensitive enabled by default for local docker

* Add missing tests

* Address PR comments

* Address PR comments

* fix ui breaks on gcsValues.json

* Address PR comments

* Minor refactor

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-03-03 18:10:01 +00:00
NiharDoshi99
1ff76f5e65
pii tagging using spacy (#10256)
* WIP: pii tagging using spacy

* added test cases and changes as per comment

* fix python checkstyle

* fix python checkstyle

* added score, test_cases and docs update

* solved merge conflict

* fix python checkstyle

* remove pii tagging using regex

* fix python test

* lib changes and added some test case

* changed as per comment

* fix: python test

* fix: changes to get source_config

* fix: changes as per comment
2023-03-03 18:33:18 +05:30
Mayur Singal
a11f7d7e37
Fix #8981: Move Connection Options and Connection Args to main properties (#10301) 2023-03-01 17:12:28 +05:30
Sachin Chaurasiya
4ce038557e
feat(ui)#10274: UI - allow to configure OM logo (#10316)
* feat(ui)#10274: UI - allow to configure OM logo

* chore: use logo config to render the brand and login page logo

* fix : cy tests

* doc: update the doc

* chore: remove check for filePath logo location type

* test: add unit test

* - removed FilePath
- Added configs to docker

* address comment

* remove config from test yaml

* update logo aspect ratio

* remove type

* chore: update ui logic as per backend config

* update doc

* address comments

* Revert "address comments"

This reverts commit c4f9da88918343cab08cd5e603cba29bd05b2ec9.

* address comments

---------

Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-03-01 14:09:44 +05:30
Teddy
754074f1be
Fixes #7758 - Added Column value and Integer Range Partitionning (#10350)
* feat(profiler): renamed  module to

* feat(profiler): added dbt-artifacts-parser to test setup.py

* feat(profiler): refactor workflow and interface

* feat(profiler): linting

* feat(profiler): removed old profiler modules

* feat(profiler): added support for value and integer range partition

* feat(profiler): fixed linting

* feat(profiler): added partitionning support for datalake profiler

* feat(profiler): removed `ProfilerInterfaceArgs` class

* feat(profiler): address comments

* feat(profiler): Added `OTHER` as an `IntervalType` for UI type generation
2023-03-01 08:20:38 +01:00
Pere Miquel Brull
203e2b5fff
Simplify Pipeline Service Client Response and prep test connection for ingestion base (#10334)
* Simplify Response and prep test connection

* Format
2023-02-25 15:52:14 -08:00
Cristian Calugaru
3f61c3e05d
Issue 9064 - New Object Store Service and Container models (#10283)
* First cut of the store service model

* Second cut of the store service model

* reverted accidental rename of property

* Third version for the object container schema definition.

* final draft version for container and object store service related entities

* final first draft of api's and entities for the object store service and container

* formatting

* removed test file

* formatting

* test fixes

* formatting

* more fixed tests

* fixed tests, addressed PR comments

* added type for object store connections

* addressed PR comments

* formatting

* renamed objectstoreservice to service in schemas. Changed type to fullyQualifiedEntityName in createContainer schema
2023-02-24 16:34:08 -08:00
Onkar Ravgan
21a30af25c
Added uri in jsonchema for aws credentials (#10315)
* Added uri in jsonchema

* fixed sample data
2023-02-24 22:17:38 +05:30
Mohit Yadav
960ae2918e
- Config Changes for Logo and Jwt Expiry (#10287)
* - Changes for Logo
- Make Jwt Token Expiry Configurable

* change config time

* fix tests

* review comments

* fix failing python

* moved login to application config as well

* fix login Config

* Added Nav Bar and Login Page Config for Logos

* Removed UI Api calls

* fix: unit test

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-02-23 22:57:57 +05:30
Pere Miquel Brull
34a54461bd
Pipeline Service Client - Pass the service to run and deploy (#10294)
* Pass de service to run and deploy

* Generalize type map

* Format
2023-02-23 11:40:18 +01:00
Mayur Singal
0fafbf70ba
Support postgres multiple DB ingestion for user with no access (#10207) 2023-02-23 15:42:40 +05:30
Mayur Singal
cd4461397d
Add impyla as scheme for hive connector (#10270) 2023-02-22 16:54:56 +05:30
Parth Panchal
9bd22369d7
Fixed#10211: Enable LDAP configuration to be configured via environment variable (#10252)
* Fixed#10211: Enable LDAP configuration to be configured via environment variable

* Fixed#10211: Enable LDAP configuration to be configured via environment variable
2023-02-22 12:53:24 +05:30
Abhishek Pandey
c678b54b05
issue-9734: job-id-param-added-for-dbt-cloud-ingestion (#10248)
UI changes will be handled with https://github.com/open-metadata/OpenMetadata/issues/10269
2023-02-22 12:18:03 +05:30
Ayush Shah
785142d86a
Add policy tags from Bigquery (#10189) 2023-02-20 19:13:45 +00:00
Mohit Yadav
8939d81470
Update Docs (#10233) 2023-02-20 14:38:43 +05:30
Suresh Srinivas
afad0a4769
Fixes #10123 - Change entityReference in createRequests to fullyQualifiedName (#10124)
* Change entityReference to entity name or fullyQualifiedName

* Change backend code and tests to use FQN

* UI change for using fqns instead of EntityReference

* Ingestion framework changes for using fqns instead of EntityReference

* Fix test failures

* Fixed python tests and sample data new

* fix: minor ui changes for fqn

* Fixed python integration tests

* Fixed superset tests

* fix UI tests

* fix type issue

* fix cypress

* fix name for testcase

---------

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-02-13 13:38:55 +05:30
Suresh Srinivas
c4c660e25c
Relax data asset name restrictions to allow the special characters except "::" (#10169) 2023-02-09 21:27:08 -08:00
Parth Panchal
5c351781f5
Fixed#9427: LDAP-s support connection without MTLS (#10146)
* Fixed#9427: LDAP-s support connection without MTLS

* Fixed#9427: LDAP-s support connection without MTLS

* Fixed#9427: LDAP-s support connection without MTLS

* Fix Json Issues

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-02-09 17:42:38 +05:30
Mayur Singal
392107bc4a
Datalake Avro & Json Lines Support (#10129) 2023-02-08 17:31:25 +00:00
NiharDoshi99
34a0cc147e
Fix: Added changes for Pii sensitive (#10119)
* Fix: added changes for pii sensitive

* Fix: removed comments

* Fix: python checkstyle

* differtiate between sensitive and non sensitive tag

* fix: python test

* fix: added tests

* fix: maven CI
2023-02-08 16:00:47 +00:00
Pere Miquel Brull
1835de8bf2
Group airflowConfiguration into pipelineService config (#10120)
Group airflowConfiguration into pipelineService config (#10120)
2023-02-07 16:30:42 +01:00
ragul balaji ravichandran
97747d9803
add GChat as alert destination (#10109) 2023-02-04 17:04:31 -08:00
Suresh Srinivas
3d016c46f2
Fixes #10065 Allow Unicode characters and digits in the entityName (#10066)
* Fixes #10065 Allow Unicode characters and digits in the entityName

* Use Unicode modifier (?U) to JSON schema patterns

* fix(dataModel): convert unicode modifier from uppercase to lowercase

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2023-02-02 12:49:45 -08:00
Pere Miquel Brull
f0f3f0be6a
Add looker unit tests (#9691)
* Add looker tests

* Empty-Commit

* Install GE for tests

* Fix usage details python name

* Add missing test requirement
2023-02-01 09:20:26 +00:00
Teddy
0b3996d2fa
Remove profiler support for dynamodb (#10014)
* fix(DynamoDB): remove profiler support and cleaned up documentation

* fix(DynamoDB): remove supportProfiler filed in tests
2023-01-31 08:29:35 +01:00
Suresh Srinivas
a1b9d3fe65
Fixes 10021 - Add CSV import/export for users (#10022)
* Fixes 10021 - Add CSV import/export for users

* Fixes 10021 - Add CSV import/export for users
2023-01-30 21:34:34 -08:00
Sriharsha Chintalapani
28c8ce1386
Fix #9750: Refactor interfaces into openmetadata-spec module (#9751) 2023-01-30 11:47:30 -08:00
Suresh Srinivas
d81187817a
Fixes #9986 - Add documentation support for the CSV template for imports and exports (#9987) 2023-01-29 13:47:02 -08:00
Sriharsha Chintalapani
8b45b639fd
Add table entity name pattern (#9970)
* Added ()0 to table entityName for oracle metadata

* move # character to columnName pattern

* Test newly allowed characters in table name and columns

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
Co-authored-by: sureshms <srini30005@gmail.com>
2023-01-27 18:50:35 -08:00
Nahuel
da1f911581
Doc: Add missing source config types in connectors config examples (#9955) 2023-01-27 15:30:48 +01:00
Pere Miquel Brull
4650a453e1
Prepare Vertica Lineage and Usage (#9906)
* Prepare Vertica Lineage and Usage

* Simplify db usage

* Linting

* Revert postgres changes

* Revert postgres changes

* Add vertica flags
2023-01-26 14:34:33 +01:00
Pere Miquel Brull
c368116697
Add dbt cloud URL as an input parameter (#9911)
* Add dbt cloud URL as an input parameter

* support dbt cloud url from UI

* Flag dbtCloudUrl as required

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-01-26 09:36:03 +01:00
Suresh Srinivas
89bd68cdf6
Fixes #9829 Add owner field to GlossaryTerm (#9833) 2023-01-20 12:20:25 -08:00
Mayur Singal
939adf887b
Fix #8973: Superset fetch metadata from DB (#9645)
* Docs #8973: Superset with SSO docs

* Superset support for mysql & postgres db

* remove unwanted field from yaml

* Fix pylint

* Fix superset tests

* Fix sample data connection

* ci fix & review comments

* cypress & defualt provider fix
2023-01-19 16:39:43 +01:00
Pere Miquel Brull
b0058bdb65
Fix #9707 - Install instantclient and enable thick mode (#9761)
* Install instantclient

* Prep instant client

* Update docs

* Amend comments
2023-01-19 08:08:56 +01:00
Ayush Shah
02147b6502
Fix: Quicksight Dashboard and chart Metadata Ingestion (#9786)
* Fix Quicksight json schema and errors

* Fix Quicksight lineage

* add pagination

* Generalize Pagination

* Fix Pylints

* update max results

* fix tests

* Add Pydantic Models for QuickSight resp

* Add fqn builder

* Fix Py lInt

* Fix Tests
2023-01-18 20:37:41 +05:30
NiharDoshi99
f2649041f2
Fix: atlas test connection (#9791)
* Fix: atlas test connection

* Fix: docs changes

* Fix: docs changes

* Fix: python checkstyle

* Fix: python test
2023-01-18 20:16:07 +05:30