426 Commits

Author SHA1 Message Date
Suresh Srinivas
dc5de12668
13188 (#13331)
* WIP

* Fixes #13188 Non-admin user can't add term on glossary after policy assigned
2023-09-25 12:09:25 -07:00
Mohit Yadav
d71647f9fd
Adds Edited By as relationship (#13317) 2023-09-25 02:38:10 +05:30
Teddy
e9ef7b5e81
Issue-12857: Remove ES Dependency from DI Workflow (#13303)
* feat: move elasticsearch indexing to backend + introduced EntityTimeSeries interface for timeseries type object

* feat: make reportData.json inherit from EntityTimeSeriesInterface

* feat: updated type to Object

* feat: deleted elasticsearch dependencies

* feat: removed elasticsearch indexing from workflow

* feat: added data insight sample data

* feat: cleaned up tests
2023-09-21 16:17:47 -07:00
Suresh Srinivas
f45d82484d
Fixes #3090 Glossary Term approval workflow (#13269) 2023-09-19 18:30:20 -07:00
Ashish Gupta
d7c212e893
fix(ui) : supported storedProcedure and dashboardDataModel search index (#13194)
* supported storedProcedure and dashboardDataModel search index

* added multi language support in lineage drawer and added stored procedure and data model in it

* added fix in explore side panel

* made id required in schema and ui side

* remove stored procedure from linages sidebar

* ui: supported storedProcedure as edge in lineage (#13225)

* supported storedProcedure as edge in lineage

* fix unit test

---------

Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
2023-09-19 19:42:39 +05:30
Pere Miquel Brull
18a4513ccc
Fix #13237 - Rename to instanceDomain and test DomoDashboard charts (#13247)
* Rename sandboxDomain to instanceDomain

* Test Get Charts in DomoDashboard

* Fix schemas

* Fix test

* Fix test

* Rename to Auto Tag PII

* Fix query test

* Fix query test

* Fix query test
2023-09-19 14:14:04 +02:00
Sriharsha Chintalapani
c2ed4f422f
Fix LifeCycle inconsistencies in Schema, make it common entity field (#13252)
* Fix LifeCycle inconsistencies in Schema; Add DELETE api

* set autocommit to true for non transactional

* make lifecycle common field for entities

* Add LifeCycle as common entity field

* Fix python life cycle code

* Fix search indexes

* remove unnecessary constant

* Add test back to entity resource test

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Add missing schema

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-09-19 14:03:57 +02:00
Pere Miquel Brull
22b0f44e38
Part of #12998 - Query Service & Lineage filter processed queries (#13215)
* Update mapping

* Prep

* prep

* Prep query migration

* Add query index deletion fix

* Docs and Maven CI fix

* Fix tests

* Add service filter

* Add query entity FQN col migration

* Fix lint

* supported serviceFQN in query api

* Prep repo

* Prep ES query search

* Do not recompute lineage

* Format

* Fix test

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2023-09-19 07:37:47 +02:00
Onkar Ravgan
cc47f5618f
Added SSL support to Superset (#13130) 2023-09-18 10:45:21 +05:30
Sriharsha Chintalapani
8cb73340ff
Fix glossary test (#13235) 2023-09-17 21:56:30 -07:00
Sriharsha Chintalapani
734629f846
Fix #9972: Add Style with color or icon for Tags,GlossaryTerm, Domain… (#13233)
* Fix #9972: Add Badge with color or icon for Tags,GlossaryTerm, Domain and DataProducts

* Fix #9972: Add Style with color or icon for Tags,GlossaryTerm, Domain and DataProducts

* Add style to TagLabel
2023-09-17 20:01:55 -07:00
Sriharsha Chintalapani
02094179e6
Fix #12899: UI to use Tier TAG displayName if provided (#13232)
* Fix #12899: UI to use Tier TAG displayName if provided

* fix python test
2023-09-17 14:28:10 -07:00
Onkar Ravgan
1e4d48a034
Added Life Cycle sample data and changed datetime to timestamp (#13141) 2023-09-13 10:59:19 +05:30
Sriharsha Chintalapani
90d1e87cd2
Fix #12898: Support custom properties for Glossary and Glossary Term (#13125) 2023-09-13 00:15:55 +05:30
Mayur Singal
e59304ed31
Fix hive metastore testconnection (#13157) 2023-09-12 22:49:54 +05:30
Pere Miquel Brull
f0995cbddc
Part of #12998 - Prep Stored Procedures Skeleton for Snowflake (#13121)
* Prep Stored Procedures Skeleton for Snowflake

* Update pylint and add migrations

* Fix test

* Reuse source url computation
2023-09-12 14:25:42 +02:00
Mayur Singal
81e40b7f59
Fix Ingesion Pipeline Configuration Titles (#13139) 2023-09-12 12:13:24 +05:30
Mayur Singal
2e3e50f0cb
Add used by field in query entity (#13096) 2023-09-11 17:30:34 +05:30
07Himank
e98cb244a4
Refactoring search clients (#13016)
* working on refactoring the search clients

* indention

* working on refactoring

* working on refactoring

* working on the refotring the whole search

* working on the refotring the whole search

* working on update indexes from repository

* Refactoring the clients

* cleanup code

* added index removed comments

* ..

* checkstyle

* working on adding new indexes

* style

* incorrect changes

* missed a catch statement

* error handling at one place and async call to clients

* added new indexes to jp,zh async call , many small handlings

* task type fix

* fixing CI

* checkStyle

* fixed CI testes

* fixed CI glossary test issue

* remove reindexing from the inportcsv

---------

Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
2023-09-09 12:24:46 +05:30
Mohit Yadav
e70f7afe4b
Add Voting to Entities (#13101)
* Add Voting to Entities

* Fix Failing test and maven build failure in main

* revert files
2023-09-08 16:23:32 +05:30
Mayur Singal
4e633877b3
Fix ElasticSearch Test Connection & Deploy (#13061) 2023-09-08 12:40:48 +05:30
karanh37
b6bab6c7dd
feat(ui): Domains functionality (#12839) 2023-09-07 12:26:25 +05:30
Onkar Ravgan
d8e78a32d8
Added Life Cycle fields, API and ingestion integration (#13014)
* Added initial code

* temp commit

* Added snowflake changes

* Added postgres migration

* removed non req changes

* review comments one

* Added ometa tests

* lint fix

* fix lint naming

* fixed merge changes
2023-09-06 12:37:55 +02:00
Onkar Ravgan
a3ca8b6e66
Added project filter pattern to dashboard entity (#12925) 2023-09-05 11:01:28 +05:30
Mayur Singal
029786d773
Fix #9207: Add source in lineage details (#13046) 2023-09-04 11:05:56 +05:30
gauthk6
ba2201f4ea
draft: implementation of lightdash connector. (#12957)
* Fixes a bug while patching the description of a TestCase

* Update docker-compose.yml

* Update docker-compose.yml

* Ran pre-commit checks and linter

* Added some clarifying points and fixed some grammatical errors in the documentation for installation instructions.

* revert changes made to docs

* implementation of lightdash connector.

* ui: add icon for lightdash service

* chore: update lightdash image

* chore: update the icon

* Cleaned up code (took out debug statements, etc.). Still TODO: yield_dashboard_lineage_details() not being called just yet.

* fix: ran linting

* added null checks

* Delete openmetadata-server.md

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2023-08-31 17:28:07 +02:00
vanshika18
3cc15e6d0b
Fixed #8418 added new connector couchbase (#12836) 2023-08-29 11:16:32 +05:30
Sriharsha Chintalapani
19b5c946a7
Fix #12167: Support for Stored Procedures as another entity under Database Schema (#12999)
* Add Stored Procedure Entity

* Stored Procedure repository

* Stored Procedure repository

* Fix #12998: Support for Stored Procedures as another entity under Database Schema

* Fix #12998: Support for Stored Procedures as another entity under Database Schema
2023-08-25 08:14:30 +02:00
vanshika18
52153dba98
fixed #10691 support multiDatabase, usage & lineage support for Azure SQL (#12982) 2023-08-24 17:47:41 +05:30
Ayush Shah
4464df81ff
Add include views to profiler (#12974) 2023-08-23 17:30:44 +05:30
Pere Miquel Brull
4862b0df28
Fix #10344 - Use ingestionPipeline retries when deploying (#12926) 2023-08-23 14:52:54 +05:30
Mayur Singal
f6d5c7413f
Fix #6700: Add support for table properties: file format for datalake (#12920)
* Fix #6700: Add support for table properties: file format for datalake & storage

* pylint fix

* resolve review comments
2023-08-22 09:46:22 +02:00
Suresh Srinivas
28b5e00c0c
Clean up documentation typos and grammar issues (#12930) 2023-08-20 20:08:30 -07:00
Yu Ishikawa
6b6d4bba76
Support to impersonate a service account in the Bigquery source (#10127) 2023-08-20 19:20:02 +05:30
Sachin Chaurasiya
6e2a8d0e4f
feat(ui): generate ingestion workflow form from schema (#12774) 2023-08-20 12:01:06 +05:30
Teddy
a45c644bc4
Issue-12884: Fix Test Case Summary Performance Issues (#12907) 2023-08-18 20:47:47 +02:00
Mayur Singal
2f7a2193e0
Fix #12302: Add ElasticSearch Ingestion Source (#12892) 2023-08-18 15:10:31 +05:30
Sriharsha Chintalapani
524e611edc
Deprecate Flyway and use Native migrations (#12865)
* Add ascii_bin for fqnhash, Add Constraints on timeseries

* Attemp of moving migrations to files

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Deprecate Flyway and use Native migrations

* Move upstream migrations to new path
2023-08-17 14:39:05 -07:00
Onkar Ravgan
3114ef9b5c
Add support for JWT auth in Trino (#12823)
* Added jwt field in trino

* fixed postgres migration

* fixed pytests
2023-08-17 07:22:58 +02:00
07Himank
ad3871b3bc
Added source url to search indexes and missing entites (#12773)
* added sourceUrl to missing entites and added sourceUrl to search  indexes

* edited desc in json files

* edited desc in json files

* working on adding api for getting entity from sourceUrl

* added API for getting Entity from sourceUrl

* added api to fetch entity based on sourceUrl

* remove desc

* working on adding alias
2023-08-16 15:56:27 +05:30
Teddy
8e4388c35e
Issue 11345 (#12859)
* feat: added serviceName dimension to entity report

* feat: fix python test
2023-08-14 08:05:14 +02:00
Suresh Srinivas
e3c0539301
12142 Task 4 - Add support for adding entities as part of Data Product (#12867) 2023-08-12 15:45:56 -07:00
Sriharsha Chintalapani
7aaf654f01
Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch (#12782)
* Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch

* Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch

* Fix #12779: Add support for SearchIndexes for ElasticSearch and OpenSearch

* Rebase fixes with main

* Add Sample Data

* lint fix

* remove unused import

* Fix service count test

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-08-10 16:47:37 -07:00
Sriharsha Chintalapani
da0c79cf78
Fix bulk soft-delete or restore from search indexes (#12807)
* Fix bulk soft-delete or restore from search indexes

* Fix bulk soft-delete or restore from search indexes

* Add entityRestored to list events api
2023-08-09 23:24:10 -07:00
Pere Miquel Brull
d8e288626c
Storage connection naming, ADLS service and namespace packaging (#12810)
* Update storage connection naming

* Update storage services and remove typing of generic service

* Remove inits for namespace package

* Add test connection

* Revert "Remove inits for namespace package"

This reverts commit 89f1ae9ea8a633593776f51caf2586d2110d4636.

* Add enum

* Remove init to allow namespace packaging in some modules

* Remove init to allow namespace packaging in some modules

* Move examples and fix test

* Format

* Linting
2023-08-10 07:32:58 +02:00
Suresh Srinivas
8acc1f61ec
12803.part2 - Display names are not updated in entity references (#12829) 2023-08-09 17:17:55 -07:00
Pere Miquel Brull
e97d4befb1
Fix #12770 - Cleanup DL structure & Readers & Python 3.8 (#12776) 2023-08-09 16:07:16 +05:30
Suresh Srinivas
340a029807
Fixes #12802 Add support for generic entity cache (#12803)
* Add entity cache in EntityRepository for all the entities

* Set entity fields only when it is not already set

* Clone entities from the cache

* Remove SubjectCache

* Remove RoleCache

* Remove PolicyCache

* Remove TagLabelCache

* Deleting lingering files

* Test failure fixes

* Remove getEntityReference methods from EntityDAO

* Move quoteName logic to EntityRepository

* Fix test failures
2023-08-08 16:42:54 -07:00
Sachin Chaurasiya
23b127aa23
chore: add type in json schema (#12780) 2023-08-08 10:46:38 +05:30
Pere Miquel Brull
a82c2f001c
Fix #9704 - Add inline TNS string (#12711)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-08-07 11:04:13 +02:00