618 Commits

Author SHA1 Message Date
Suresh Srinivas
c4c660e25c
Relax data asset name restrictions to allow the special characters except "::" (#10169) 2023-02-09 21:27:08 -08:00
Parth Panchal
5c351781f5
Fixed#9427: LDAP-s support connection without MTLS (#10146)
* Fixed#9427: LDAP-s support connection without MTLS

* Fixed#9427: LDAP-s support connection without MTLS

* Fixed#9427: LDAP-s support connection without MTLS

* Fix Json Issues

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-02-09 17:42:38 +05:30
Mayur Singal
392107bc4a
Datalake Avro & Json Lines Support (#10129) 2023-02-08 17:31:25 +00:00
NiharDoshi99
34a0cc147e
Fix: Added changes for Pii sensitive (#10119)
* Fix: added changes for pii sensitive

* Fix: removed comments

* Fix: python checkstyle

* differtiate between sensitive and non sensitive tag

* fix: python test

* fix: added tests

* fix: maven CI
2023-02-08 16:00:47 +00:00
Pere Miquel Brull
1835de8bf2
Group airflowConfiguration into pipelineService config (#10120)
Group airflowConfiguration into pipelineService config (#10120)
2023-02-07 16:30:42 +01:00
ragul balaji ravichandran
97747d9803
add GChat as alert destination (#10109) 2023-02-04 17:04:31 -08:00
Suresh Srinivas
3d016c46f2
Fixes #10065 Allow Unicode characters and digits in the entityName (#10066)
* Fixes #10065 Allow Unicode characters and digits in the entityName

* Use Unicode modifier (?U) to JSON schema patterns

* fix(dataModel): convert unicode modifier from uppercase to lowercase

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2023-02-02 12:49:45 -08:00
Pere Miquel Brull
f0f3f0be6a
Add looker unit tests (#9691)
* Add looker tests

* Empty-Commit

* Install GE for tests

* Fix usage details python name

* Add missing test requirement
2023-02-01 09:20:26 +00:00
Teddy
0b3996d2fa
Remove profiler support for dynamodb (#10014)
* fix(DynamoDB): remove profiler support and cleaned up documentation

* fix(DynamoDB): remove supportProfiler filed in tests
2023-01-31 08:29:35 +01:00
Suresh Srinivas
a1b9d3fe65
Fixes 10021 - Add CSV import/export for users (#10022)
* Fixes 10021 - Add CSV import/export for users

* Fixes 10021 - Add CSV import/export for users
2023-01-30 21:34:34 -08:00
Sriharsha Chintalapani
28c8ce1386
Fix #9750: Refactor interfaces into openmetadata-spec module (#9751) 2023-01-30 11:47:30 -08:00
Suresh Srinivas
d81187817a
Fixes #9986 - Add documentation support for the CSV template for imports and exports (#9987) 2023-01-29 13:47:02 -08:00
Sriharsha Chintalapani
8b45b639fd
Add table entity name pattern (#9970)
* Added ()0 to table entityName for oracle metadata

* move # character to columnName pattern

* Test newly allowed characters in table name and columns

---------

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
Co-authored-by: sureshms <srini30005@gmail.com>
2023-01-27 18:50:35 -08:00
Nahuel
da1f911581
Doc: Add missing source config types in connectors config examples (#9955) 2023-01-27 15:30:48 +01:00
Pere Miquel Brull
4650a453e1
Prepare Vertica Lineage and Usage (#9906)
* Prepare Vertica Lineage and Usage

* Simplify db usage

* Linting

* Revert postgres changes

* Revert postgres changes

* Add vertica flags
2023-01-26 14:34:33 +01:00
Pere Miquel Brull
c368116697
Add dbt cloud URL as an input parameter (#9911)
* Add dbt cloud URL as an input parameter

* support dbt cloud url from UI

* Flag dbtCloudUrl as required

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-01-26 09:36:03 +01:00
Suresh Srinivas
89bd68cdf6
Fixes #9829 Add owner field to GlossaryTerm (#9833) 2023-01-20 12:20:25 -08:00
Mayur Singal
939adf887b
Fix #8973: Superset fetch metadata from DB (#9645)
* Docs #8973: Superset with SSO docs

* Superset support for mysql & postgres db

* remove unwanted field from yaml

* Fix pylint

* Fix superset tests

* Fix sample data connection

* ci fix & review comments

* cypress & defualt provider fix
2023-01-19 16:39:43 +01:00
Pere Miquel Brull
b0058bdb65
Fix #9707 - Install instantclient and enable thick mode (#9761)
* Install instantclient

* Prep instant client

* Update docs

* Amend comments
2023-01-19 08:08:56 +01:00
Ayush Shah
02147b6502
Fix: Quicksight Dashboard and chart Metadata Ingestion (#9786)
* Fix Quicksight json schema and errors

* Fix Quicksight lineage

* add pagination

* Generalize Pagination

* Fix Pylints

* update max results

* fix tests

* Add Pydantic Models for QuickSight resp

* Add fqn builder

* Fix Py lInt

* Fix Tests
2023-01-18 20:37:41 +05:30
NiharDoshi99
f2649041f2
Fix: atlas test connection (#9791)
* Fix: atlas test connection

* Fix: docs changes

* Fix: docs changes

* Fix: python checkstyle

* Fix: python test
2023-01-18 20:16:07 +05:30
Abhishek Pandey
6d7acdc140
Issue 9542 - Replace the default name for MySQL db (#9687)
* database-field-added-for-custom-database-name

* database-field-added-for-custom-database-name

* database field description-added

* databaseName-field-updated-in-connection-json

* database-name-field-updated-in-salesforce

* databaseName-description-updated

* database-name-logic-updated-in-common-db-source
2023-01-18 14:28:40 +01:00
Onkar Ravgan
08a4d33c5f
Powerbi pagination fix (#9784)
* Fixed powerbi pagination logic

* Added yaml changes
2023-01-18 17:17:13 +05:30
Suresh Srinivas
ac157a8f0b
Fixes #9618 Allow apostrophe in entity names (#9642)
* Fixes #9618 Allow apostrophe in entity names

* Add support for space in entity names (with exception) and column names

* Add regex checks to entity names, table column names to allow only required characters

* fix: allowed characters in entityName

* Add all the allowed special characters in the entity name in tests

* Allow '&', '/', ':' only for the table

* Override basic entityName by defining entity specific entityName for Python code gen

* fix: dagster migration + json schema path

* fix: python style check

* Fix test failure

Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2023-01-17 11:34:29 -08:00
Suresh Srinivas
b6e7bcfee5
Fixes #9721 Add support for count of terms in Glossary and Classification (#9733) 2023-01-13 11:40:24 -08:00
Corey Zuares
c3197beaf4
Extra headers for metadata API (#9321)
* Configurable Extra Authorization Header

- Adds two new header that can be set via env-var
  (`OMETA_HEADER_EXTRA_AUTH_NAME` and `OMETA_HEADER_EXTRA_AUTH_VALUE`)
- By default, it copies the Authorization header value to the extra auth header
- If a value is specified in the corresponding env-var, that value is
  used instead of the existing token

The primary use case is for Google Cloud's Identity Aware Proxy (IAP) which allows secure access to private services but removes the Authorization header in the process. The `Proxy-Authorization` header gets forwarded along to the service as the `Authorization` header for the internal service.

* Use openMetadataConnection schema instead of env vars

* Disable too-many-locals for _request function

* Fix dict merge
2023-01-13 09:59:44 +01:00
Milan Bariya
bb41fec08f
Add: Function for Postgres Tags (#9683)
* Add: Function for Postgres Tags

* Update: Query for generate fqn
2023-01-12 09:46:08 +01:00
Teddy
41ce2406a4
fixes #9673 - Oracle casting error (#9682)
* fix (profiler): oracle profiler datatype error

* fix (profiler): linting

* fix: reorder connection type to default to db schema
2023-01-11 19:46:10 +01:00
Mohit Yadav
ea0e0e03e4
Alert Test Fix (#9640)
* Alert Test Fix

* Removed Db calls in case local publishers available

* Fix test

* Removed unrequired code

* Maintaing entityType to class map

* checkstyle
2023-01-10 12:56:10 +05:30
Sriharsha Chintalapani
3c9d39e9af
Fix #9158: OpenSearch/ElasticSearch - Language based mappings (#9163)
* Add language for Search mappings

* Fix #9158: OpenSearch/ElasticSearch - Language based mappings

* Add analyzer to search in Japanese

* Adding JP es analyzer for all the indexes

* Fix review comments

* rebase with main

Co-authored-by: Megumi AIKAWA <megumi.aikawa@gmail.com>
2023-01-09 21:42:01 -08:00
Pere Miquel Brull
84348d4748
Fix #8866 - bump datamodel-codegen (#9623)
* Fix #8866 - bump datamodel-codegen

* Update connection options and arguments structure

* Add builders test

* Format

* Allow Any values in componentConfig

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-01-09 13:20:32 +01:00
Onkar Ravgan
e9ac8ecb85
Fix 7972: Added dbt tag classification name (#9575)
* Added dbt tag classification name

* resolved merge conflicts

* handle ui side change for dbt schema change

* Fixed migration scripts

* fixed failing cypress for dbt

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2023-01-06 11:08:21 +05:30
Mayur Singal
9ef32eef99
Fix #6829: Improve usage filtering (#9613) 2023-01-05 15:07:03 +00:00
Steve Russo
3a3f248edd
Issue 5592: Add Databricks Pipeline Connector (#9554)
* feat: update databricks_client

* feat: create pipelineconnection & connection

* test: add unit test

* feat: add examples file

* fix: remove unused imports

* style: fix python style

* fix: code review adjustments

* fix: remove unneeded schema ref

* fix: format requirements

* fix: change context calls

* test: add pipeline_status test

* format: pylint format change

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-01-05 14:14:12 +01:00
Mayur Singal
0a34e18e26
Fix #9562: Add duration in usage ingestion (#9564) 2023-01-05 06:33:07 +00:00
Pere Miquel Brull
1f49ea46b5
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588)
Fix #9149 - Postgres foreign tables & reorganize queries for sources (#9588)
2023-01-04 08:50:53 +01:00
Suresh Srinivas
240920fa9f
Fixes #9118 - Glossary bulk upload support (#9524)
* Add CSV support for bulk edits

* Add CSV support for bulk edits - updates to the previous patch

* APIs for Glossary bulk export/import support
2022-12-29 09:58:06 -08:00
Pere Miquel Brull
a4521fd664
Fix #6562 - Sources have their own package (#9521)
Fix #6562 - Sources have their own package (#9521)
2022-12-27 15:00:22 +01:00
Suresh Srinivas
758c976cba
Fixes #9259 Change Tags APIs to conform with rest of the APIs (#9260) 2022-12-26 12:32:17 -08:00
Suresh Srinivas
c931ee3447
Fixes #9510 Enable authorization tests for all the entities (#9511) 2022-12-23 07:47:09 -08:00
Mohit Yadav
70d5536745
Added Test cases for Alerts + bug Fixes (#9421)
* Added Test cases for Alerts + bug Fixes

* removed commented code

* random bool

* added anno

* Disabling test for AlertAction and Alert + alertAction random name

* Bulk Create AlertAction Api
2022-12-21 15:34:18 +05:30
NiharDoshi99
9a3d599f30
Azure datalake minor changes (#9407) 2022-12-21 10:28:41 +05:30
Ayush Shah
2bf5eb9051
fix 7995: profileSample % and row number (#9104) 2022-12-20 14:55:11 +05:30
Mohit Yadav
fa65480864
Alert Fixes, Removed Alert and AlertAction from Entity List, Get latest status from the data base, removed lastSuccesfulAt from failuredetails (#9386) 2022-12-18 23:39:27 +05:30
Mayur Singal
c06dbfd22e
Fix caching issue for BQ (#9352) 2022-12-17 09:16:16 -08:00
Sriharsha Chintalapani
0055ca2d1c
Fix #2435: Add tagging support for services and database (#9330)
* Fix #2435: Add tagging support for services and database

* Fix check style

* Fix check style

* Fix check style
2022-12-16 08:34:26 -08:00
Mohit Yadav
08f6536558
Alert Fixes : removed redundant files, updated Triggwe (#9346)
* Removed Redundant Code + Removed Unnecessary Triggers, Activity Feed Filterin

* all should have intersecting filters

* updated by should have entities index
2022-12-16 21:59:09 +05:30
Teddy
3856d63f3a
Fixes #5522 by implementing timeout for the profiler (#9340)
* feat(Profiler): Added timeout logic in the profiler workflow

* feat(Profiler): Added additional input in workflow UI

* fix(Profiler): timeout text blob in UI

* fix(Profiler): Return profile_results outside the context manager

* Fix(Profiler): Added localizer for timeout label and message

* Update ingestion/src/metadata/interfaces/sqalchemy/sqa_profiler_interface.py

Co-authored-by: Nahuel <nahuel@getcollate.io>

Co-authored-by: Nahuel <nahuel@getcollate.io>
2022-12-16 17:01:12 +01:00
NiharDoshi99
3190ac2516
Fix: making token as optional field (#9314) 2022-12-16 14:17:47 +00:00
Ashish Gupta
7c181a5a89
Dbt worflow (#9302) 2022-12-16 17:00:28 +05:30