618 Commits

Author SHA1 Message Date
Sachin Chaurasiya
23b127aa23
chore: add type in json schema (#12780) 2023-08-08 10:46:38 +05:30
Pere Miquel Brull
a82c2f001c
Fix #9704 - Add inline TNS string (#12711)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-08-07 11:04:13 +02:00
Pere Miquel Brull
ac06f55bd0
[Docs] - host.docker.internal docs & postgres ingestAllDatabases below db input (#12744) 2023-08-04 08:35:33 +02:00
karanh37
521711940e
fix: ui fixes 1.1.1 (#12694)
* fix: long text ellipsis wrapping

* fix: node suggestions long text issue

* replace name with fqn for tags

* fix cypress issue

* change the header label ui of explore page

* fixed classificatin issue for .

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: 07Himank <himank07mehta@gmail.com>
2023-08-02 18:17:45 +05:30
Ashish Gupta
de06d50e7f
fix(ui): ui breaking as url was not encoded. (#12643)
* fix ui breaking because of url having percent in it

* added support for % in about

* fix test cases

* checkstyle

* url fix

* fix storage url

* encoded topic, mlmodel and pipeline entity

---------

Co-authored-by: 07Himank <himank07mehta@gmail.com>
2023-08-01 14:12:47 +05:30
Chirag Madlani
098625c549
feat(server): description support for team hierarchy (#12638) 2023-08-01 11:25:42 +05:30
Josh Liberty
5d53335d83
Update storageService.json (#12529)
Add support to using a custom storage connection
2023-07-31 09:01:37 -07:00
Mayur Singal
5f07c6281a
Clickhouse: Make https protocol as permanent field (#12303) 2023-07-31 19:26:03 +05:30
Mayur Singal
dacd13f6d0
Fix #11778: Add support for hive metastore ingestion (#12599) 2023-07-31 15:56:02 +05:30
Suresh Srinivas
8830d1707d
Initialize EntityResoruceTest support flags automatically (#12513) 2023-07-27 20:42:10 -07:00
Mohit Yadav
f192185151
add aws-cognito to service provider (#12614)
* add aws-incognito to service provider

* Update openMetadataConnection.json

* chore(ui): auth provider changes

* chore: remove type casting

* fix: unit test

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-07-27 19:29:50 +05:30
Mayur Singal
c6de61f6bc
Add Lineage Parsing timeout and filtering condition (#12560) 2023-07-27 12:28:53 +05:30
Pere Miquel Brull
6773541d15
[1.1.1] - Bump size for FQN (#12092)
* Bump size for FQN

* Bump table entityName size

* Bump table entityName size

* Fix table resource tests

* Remove pattern from fqn

* Remove pattern from fqn

* Remove pattern from fqn

* Generalize get_by_name in ometa client

* Generalize get_by_name in ometa client

* Format

* Fix test suite

* Remove limit from FQN max size

* Remove limit from FQN max size

* Add sample data

* Update lint names

* Add more sample data

* Bump column name size

* 1024 max FQN length

* 1024 max FQN length

* 1024 max FQN length

* Bump FQN
2023-07-26 12:36:42 -07:00
Sriharsha Chintalapani
67b68a70c9
Fix #7712: Align AuthProvider values in the JSON schema definition (#11508)
* Fix #7712: Align AuthProvider values in the JSON schema definition

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Handle new providers

* Lint

* Fix #7712: Align AuthProvider values in the JSON schema definition

* Handle new providers

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Fix ingestion bot initialization, create jwt token if there is no authMechanism pre-configured

* Lint

* fix rebase

* Fix test

* Fix test

* Fix Usage of Auth Provider in User Resource

* remove unused function

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
2023-07-26 11:34:58 -07:00
Onkar Ravgan
0f8ebf4afe
Fixed Tableau-Bigquery lineage issue (#12570) 2023-07-26 16:07:48 +05:30
Ayush Shah
307dcbf629
Add DateTimeRange support in postgres (#12583) 2023-07-26 14:31:48 +05:30
Ayush Shah
cdb0b2fd63
Add Special Character Support (#12600) 2023-07-26 14:31:07 +05:30
vanshika18
72e4d0070f
Support MSSLQ handle multiple databases with pyodbc connection (#12526) 2023-07-26 10:43:24 +05:30
Ayush Shah
246bf15476
Add Clickhouse profiler fix (#12531) 2023-07-21 10:19:56 +05:30
vanshika18
c1c61c16df
fix #2936: Support Usage connector for Trino (#12483) 2023-07-20 19:51:17 +05:30
Ayush Shah
ab1ec50c2c
Fixes Mssql Ntext, text and Image (#12490) 2023-07-20 13:34:35 +05:30
Mayur Singal
0c6435ae9b
Fix #2957: Add support for qlik sense enterprise connector (#12450) 2023-07-19 21:08:05 +05:30
Suresh Srinivas
35154615d4
Code cleanup.20230718 (#12493)
* Move setting common patch and updated fields to EntityRepository

* Cleanup entityReferenceList.json and references
2023-07-18 15:10:22 -07:00
07Himank
b6d5a438ec
Support '%' in entity Name (#12382)
* added support for the % in entity name

* added support for the % in glossary and glossary term  name

* Added U exclude

* Fixed pytest

* fix: support % in glossary name

* added suppport for % in entityName

* fix % issue in the glossary or term

* fix: glossary with % encoding

* fix: tags redirect issue for glossaries with %

* fix: glossary import path

* fixed CI

* fixed Py Test

* fixed PyTest

* fixed glossary resource test

---------

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
2023-07-18 17:58:16 +05:30
Sachin Chaurasiya
8734e77acc
fix: json schema ref and id path (#12471) 2023-07-18 07:52:23 +00:00
Mohit Yadav
ac9ccc3c2b
Add missing Expert Enum name (#12454) 2023-07-17 11:49:41 +05:30
Suresh Srinivas
0d119de96e
Fixes #12447 Use JSON schema constrain validation in CSV imports (#12448) 2023-07-15 15:02:17 -07:00
Pere Miquel Brull
3433e2b5dd
Ingestion Pipeline - Bulk deploy API & Status response updated (#12365)
* Redeploy in tables init

* Use Status class

* Update response Status

* Format

* Rename pipelineServiceClientResponse
2023-07-14 07:38:29 +02:00
vanshika18
bbe58b1b5f
Handle Table type for Snowflake Transient Tables (#12274)
* Handle Table type for Snowflake Transient Tables

* Clean the code by removing comments

* Optimize the code

* Optimize the code

* Clean the code

* Rename includeTempTables with includeTransientTables

* Rename includeTempTables to includeTransientTables

* Update Documentation and resolved migartion query error

* Resolved migartion query for postgres connection

---------

Co-authored-by: Vanshika Kabra <vanshikakabra@Vanshikas-MacBook-Pro.local>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-07-13 19:25:46 +02:00
Pere Miquel Brull
f0f74dbe59
Fix #12368 - Make sourcePythonClass not required to allow generic use of custom connectors (#12408) 2023-07-13 18:03:14 +02:00
Onkar Ravgan
e200a42926
Removed datamodels references from dashboard and datamodels (#12371) 2023-07-13 14:26:28 +05:30
Pere Miquel Brull
1b7c5e3233
Fix #12321 - Deprecate impala scheme from Hive (#12327)
* Deprecate impala scheme from Hive

* Add migrations
2023-07-11 17:38:32 +02:00
Sriharsha Chintalapani
155cf354e3
Fix OpenMetadata schema to TypeScript generation (#12335)
* initial fix to add relative path to json schemas

* initial fix to add relative path to json schemas

* fix schema generation

* fix github workflow

* add license

* add jq to developer prerequisites

* add jq to developer prerequisites
2023-07-09 21:12:46 -07:00
Megumi AIKAWA
9aede5e821
Fixes 11924 : Add pgspider support (#12159)
* Add PGSpider service connector

* add sqlalchemy-pgspider to setup.py

* Delete unnecessary code and move files.

* Change to work as a Postgres connector sub-module

* Rebuild PGSpiderLineage class into a set of funcs

---------

Co-authored-by: Pham Ngoc Son <son1.phamngoc@toshiba.co.jp>
2023-07-05 09:18:59 +02:00
Suresh Srinivas
4f2831f3bb
Upgrade dependencies for 1.2.0 release (#12280) 2023-07-04 20:03:40 -07:00
Suresh Srinivas
97140e13fd
12142.task2 - Add domain and data product properties to entities (#12255) 2023-07-03 15:42:35 -07:00
Suresh Srinivas
4d4993effc
12142.task1 - Add support for Domains (#12143) 2023-07-02 11:32:52 -07:00
07Himank
7244c3f530
changed enum from Elastic/Open Search to elastic/open search (#12237) 2023-06-29 20:13:49 -07:00
Onkar Ravgan
acf25f4555
Added pagination Tableau data sources graphql (#12187)
* Added pagination tableau graphql

* changed downstream workbook
2023-06-28 14:57:09 +02:00
Mayur Singal
a3fd6e9522
Fix #11659: Add support for filter patterns in dbt workflow (#12063) 2023-06-26 11:30:35 +05:30
Mayur Singal
258a2e33e2
Fix #12078: Add clientSessionKeepAlive Field for snowflake (#12107) 2023-06-23 10:00:53 +02:00
Pere Miquel Brull
b5c4d5a9d3
Fix #10175 - Add Oracle lineage and usage (#12090)
* Add Oracle lineage and usage

* Add docs

* Add examples
2023-06-23 06:52:34 +02:00
Teddy
712dcf8587
fix: added field to testCase entity to list all Test Suites (#12093)
* fix: added  field to testCase entity to list all the linked test suites (executable and logical)

* fix: resolved conflicts from main

* fix: removed unecessary comment
2023-06-22 18:00:19 +02:00
07Himank
9355c0677d
Open search elastic search client task (#11879)
* Added diffrent clients for elasticSearch and openSearch and did the interface implementation

* removed some comments and some changes

* Add Elastic Search Test Container

* upgarded es version

* added test cases for elastic search

* added test case for, when tag is deleted cehck its relationships also get deleted

* changed package of open,elastic to openSearch and elasticSearch

* added support for skip search test case if index is not present of that entity

* Refactor bunch of files

* unnecessary semicolon

* refactor test case indexing

* fix checkstyle

* checkstyle

* added hardcoded configs and changed package name

* fixed no response from hits

* stopped elasticsearch container and some fixes

* closed elasticsearch container

* revert

* closed client after used in test

* style

* edit es config in junit

* checking if using already es version release memory for maven ci

* revert docker file changes

* remove es container for testing the ci

* commented out all test cases for elasticsearch

* fix

* creating elastic search with reusable enabled

* trying to fix ci error

* start container only if it is not running

* revert docker changes and done changes for elastic search container

* changed var name of sort_order for es

* lower the version of es client

* fixes

* all latest version of es including docker

* Use Single sort order

* Fix Keyword issue

* Immediate refresh policy for entity created

* increase sleep time

* disable test

* added reindexing logic in run_local_docker.sh

* enabled test case

---------

Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-06-22 16:55:05 +05:30
Onkar Ravgan
78d7faf04a
Added Azure Blob Storage to dbt config (#11839) 2023-06-22 10:58:38 +05:30
Pere Miquel Brull
520d5e2657
Add Pipeline Service Status Job to publish to Prometheus (#12046)
* Add Pipeline Service Status Job to publish to Prometheus

* Add Pipeline Service Status Job to publish to Prometheus

* Add config for health check interval
2023-06-22 06:53:56 +02:00
Ayush Shah
9fa7b78d87
Issue 11686: Add IncludeOwner + scheduleInterval in Airflow (#12005) 2023-06-21 14:18:45 +05:30
Pere Miquel Brull
bfa37df298
Add platform in Pipeline Service status msg (#12041)
* Add platform in Pipeline Service status msg

* chore(ui): show message based on platform

* update the styling

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-21 07:21:19 +02:00
Mayur Singal
c3cec54be9
Fix for #11807 Part 2: Add SourceUrl for table entity (#12013)
* Fix #11807 Part 2: Add SourceUrl for table entity

* address review comments: centralize sourceurl

* remove qlick

* pytest fix

* fix typo
2023-06-20 11:46:45 +02:00
Pere Miquel Brull
8751427679
Add project to Data Models JSON Schema & Implement in Looker (#12030)
* Add project to dashboards and models

* Add project to dashboards and models

* Update openmetadata-spec/src/main/resources/json/schema/entity/data/dashboardDataModel.json

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

* Update openmetadata-spec/src/main/resources/json/schema/api/data/createDashboardDataModel.json

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>

* Fix tests

---------

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-06-20 11:45:29 +02:00