984 Commits

Author SHA1 Message Date
Onkar Ravgan
8acebbb892
Update ingestion logic to use PATCH API for lifeCycle info (#13283) 2023-09-21 16:40:09 +05:30
Teddy
a424c37a34
[Proposal] Playwright CI (#13248)
* feat: playwright CI tests

* feat: ran python linting

* feat: updated workflow to run in PR

* feat: updated server start step

* feat: fixed playwright run command

* feat: fix matrix value

* feat: removed browser option in test run

* feat: updated typo in make file

* feat: move playwritgh installation

* feat: fail CI on error

* feat: save tracing on failure

* feat: enforce expect on .fill

* feat: change tracing to video

* feat: clean up code and update CI workflow for main branch
2023-09-19 14:44:36 +02:00
Pere Miquel Brull
18a4513ccc
Fix #13237 - Rename to instanceDomain and test DomoDashboard charts (#13247)
* Rename sandboxDomain to instanceDomain

* Test Get Charts in DomoDashboard

* Fix schemas

* Fix test

* Fix test

* Rename to Auto Tag PII

* Fix query test

* Fix query test

* Fix query test
2023-09-19 14:14:04 +02:00
Sriharsha Chintalapani
c2ed4f422f
Fix LifeCycle inconsistencies in Schema, make it common entity field (#13252)
* Fix LifeCycle inconsistencies in Schema; Add DELETE api

* set autocommit to true for non transactional

* make lifecycle common field for entities

* Add LifeCycle as common entity field

* Fix python life cycle code

* Fix search indexes

* remove unnecessary constant

* Add test back to entity resource test

* Fix lint

* Fix lint

* Fix lint

* Fix lint

* Add missing schema

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-09-19 14:03:57 +02:00
Teddy
b07dac8c1d
Issue-12305 - Add logic to sort test cases result by latest execution (#13143)
* fix: added logic to store test case result state with test case entity

* fix: added relative path logic in python unit test

* fix: added logic to sort testCases based on last execution date of test case result

* fix: ran python linting

* fix: added test case listing order param in frontend call

* fix: added tests around test suite result summary state

* fix: remove isDeleted repeated check

* fix: isolated test suite for maven tests + fix some UI failing tests

* fix: run java linting
2023-09-19 08:21:38 +02:00
Pere Miquel Brull
22b0f44e38
Part of #12998 - Query Service & Lineage filter processed queries (#13215)
* Update mapping

* Prep

* prep

* Prep query migration

* Add query index deletion fix

* Docs and Maven CI fix

* Fix tests

* Add service filter

* Add query entity FQN col migration

* Fix lint

* supported serviceFQN in query api

* Prep repo

* Prep ES query search

* Do not recompute lineage

* Format

* Fix test

---------

Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2023-09-19 07:37:47 +02:00
Sriharsha Chintalapani
02094179e6
Fix #12899: UI to use Tier TAG displayName if provided (#13232)
* Fix #12899: UI to use Tier TAG displayName if provided

* fix python test
2023-09-17 14:28:10 -07:00
Pere Miquel Brull
442528267c
Simplify topology & update context management (#13196) 2023-09-15 09:44:42 +02:00
Ayush Shah
5fea08cd33
Datalake: Add manifest file support, fix profiler metrics, add array and json column type support (#13017) 2023-09-13 15:15:49 +05:30
Onkar Ravgan
1e4d48a034
Added Life Cycle sample data and changed datetime to timestamp (#13141) 2023-09-13 10:59:19 +05:30
Pere Miquel Brull
f0995cbddc
Part of #12998 - Prep Stored Procedures Skeleton for Snowflake (#13121)
* Prep Stored Procedures Skeleton for Snowflake

* Update pylint and add migrations

* Fix test

* Reuse source url computation
2023-09-12 14:25:42 +02:00
Ayush Shah
3d4a3cbb95
add mysql e2e changes (#13145) 2023-09-12 11:55:19 +05:30
Pere Miquel Brull
66a89ffe61
Fix e2e profiler status (#13127) 2023-09-10 18:46:28 +02:00
Mayur Singal
4e633877b3
Fix ElasticSearch Test Connection & Deploy (#13061) 2023-09-08 12:40:48 +05:30
Onkar Ravgan
d8e78a32d8
Added Life Cycle fields, API and ingestion integration (#13014)
* Added initial code

* temp commit

* Added snowflake changes

* Added postgres migration

* removed non req changes

* review comments one

* Added ometa tests

* lint fix

* fix lint naming

* fixed merge changes
2023-09-06 12:37:55 +02:00
Pere Miquel Brull
de7e06d024
Update structure for PII processing (#13079)
* Update structure for PII processing

* Fix tests

* Fix tests

* Lint

* Remove typo
2023-09-06 11:30:46 +02:00
Pere Miquel Brull
a9b13de893
Move workflows back into src (#13100)
* Move workflows back into src

* Fix test
2023-09-06 11:30:33 +02:00
Mayur Singal
011eaf8ad5
Fix #13001: Fix query not populating to all tables (#13004) 2023-09-05 14:16:37 +05:30
Onkar Ravgan
a3ca8b6e66
Added project filter pattern to dashboard entity (#12925) 2023-09-05 11:01:28 +05:30
Ayush Shah
dc8e59eba8
Fix E2E - dashboards (#13082) 2023-09-04 23:25:31 +05:30
vanshika18
db592a157e
fix #12518 support get column lineage in spline (#13030) 2023-09-04 17:16:04 +05:30
Pere Miquel Brull
a3bfd4e696
Part of #11968 - Restructure Profiler Workflow and PII Processor (#13059)
* Structure PII

* Restructure Profiler Workflow

* Update signature for abc

* remove profiler sink

* Fix tests

* Fix lint

* Fix test

* Fix test
2023-09-04 11:02:57 +02:00
Mayur Singal
029786d773
Fix #9207: Add source in lineage details (#13046) 2023-09-04 11:05:56 +05:30
Teddy
ab3042e8ee
Issue 12297 bis -- Delete Insert logic in the DI workflow (#13058)
* fix: updated ingestion to delete existing data for the ingestion day

* fix: added delete endpoint for the report data

* fix: added migration to delete duplicate data + added tables creation & data deltion logic in a transaction

* fix: made deletion SQL engine aware

* fix: added ES deletion logic back while we work on DI event publisher
2023-09-01 18:01:20 +02:00
gauthk6
ba2201f4ea
draft: implementation of lightdash connector. (#12957)
* Fixes a bug while patching the description of a TestCase

* Update docker-compose.yml

* Update docker-compose.yml

* Ran pre-commit checks and linter

* Added some clarifying points and fixed some grammatical errors in the documentation for installation instructions.

* revert changes made to docs

* implementation of lightdash connector.

* ui: add icon for lightdash service

* chore: update lightdash image

* chore: update the icon

* Cleaned up code (took out debug statements, etc.). Still TODO: yield_dashboard_lineage_details() not being called just yet.

* fix: ran linting

* added null checks

* Delete openmetadata-server.md

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
2023-08-31 17:28:07 +02:00
Pere Miquel Brull
1188831cd5
Fix e2e for tableau, metabase and powerbi (#13050)
* Fix e2e

* Fix e2e
2023-08-31 15:45:41 +02:00
Pere Miquel Brull
a5667bb585
Fix import (#13044) 2023-08-31 08:30:18 +02:00
Pere Miquel Brull
6c0e9f5061
Part of #7272 - Centralize Workflows, Status, and Exception Management (#13029)
* Prep changes

* Prep changes

* prep changes

* Update imports

* Format

* Prep delete

* Prep delete

* Fix sink

* Prep test

* Commit

* passing either

* passing either

* Prep Either

* Metadata source with Either

* Update status

* Merge remote-tracking branch 'upstream/main' into issue-7272

* Format

* Linting

* Linting

* Linting

* Linting

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Comments
2023-08-30 15:49:42 +02:00
vanshika18
3cc15e6d0b
Fixed #8418 added new connector couchbase (#12836) 2023-08-29 11:16:32 +05:30
Ayush Shah
407900e5b4
Fix e2e according to latest changes (#13018) 2023-08-29 10:23:23 +05:30
Ayush Shah
300347fdd3
Fix mysql e2e (#13010) 2023-08-28 23:14:22 +05:30
Teddy
101cd0ebac
Issue 8930 - Update profiler timestamp from seconds to milliseconds (#12948) 2023-08-25 08:47:16 +02:00
Pere Miquel Brull
2f3a50e95f
Fix #8885 - Misleading 404 logs & suffix cleanup (#12916)
* Fix #8885 - Misleading 404 debug logs

* Clean get_suffix complexity

* Prepare airflow config

* Add bootstrap comment

* Revert "Add bootstrap comment"

This reverts commit cc053513df6e25bcc5e742786adb91339461fdd2.

* Revert "Prepare airflow config"

This reverts commit f57645fe953e17d6153652daa1f42c14b97c942d.

* add removed method

* Add icons for docs

* Add missing routes
2023-08-21 11:35:36 +02:00
Suresh Srinivas
28b5e00c0c
Clean up documentation typos and grammar issues (#12930) 2023-08-20 20:08:30 -07:00
Teddy
bfa0cc7598
fix: python tests failure after PR #12865 (#12927)
* fix: python tests failure after https://github.com/open-metadata/OpenMetadata/pull/12865

* fix: test in ometa_table_api

* fix: skip is None test temporarly
2023-08-18 18:11:47 +02:00
Pere Miquel Brull
191754ff20
Set Test Suite FQN (#12890)
* Set Test Suite FQN

* quoteName

* quoteFqn is false for TestSuites

* Fix tests
2023-08-18 18:56:26 +05:30
Mayur Singal
2f7a2193e0
Fix #12302: Add ElasticSearch Ingestion Source (#12892) 2023-08-18 15:10:31 +05:30
Onkar Ravgan
5b47fd4acf
Added source url to entities (#12901)
* Added source url to entites

* added support to create and update sourceUrl

* fixed pytests

---------

Co-authored-by: 07Himank <himank07mehta@gmail.com>
2023-08-18 10:17:38 +02:00
Onkar Ravgan
3114ef9b5c
Add support for JWT auth in Trino (#12823)
* Added jwt field in trino

* fixed postgres migration

* fixed pytests
2023-08-17 07:22:58 +02:00
Pere Miquel Brull
a183fc67e2
Fix ADLS parquet reads (#12840)
* Fix ADLS parquet reads

* Generalize service methods

* Fix tests
2023-08-14 19:57:06 -07:00
Ayush Shah
1e2d93a604
Fix #12479: MariaDB Mysql Time Col profiler issues + fix e2e tests (#12868) 2023-08-14 17:28:09 +05:30
Teddy
8e4388c35e
Issue 11345 (#12859)
* feat: added serviceName dimension to entity report

* feat: fix python test
2023-08-14 08:05:14 +02:00
Ayush Shah
5ea4de646c
Fix Postgres Profiler for incompatible versions (#12795) 2023-08-11 15:32:43 +05:30
Pere Miquel Brull
d8e288626c
Storage connection naming, ADLS service and namespace packaging (#12810)
* Update storage connection naming

* Update storage services and remove typing of generic service

* Remove inits for namespace package

* Add test connection

* Revert "Remove inits for namespace package"

This reverts commit 89f1ae9ea8a633593776f51caf2586d2110d4636.

* Add enum

* Remove init to allow namespace packaging in some modules

* Remove init to allow namespace packaging in some modules

* Move examples and fix test

* Format

* Linting
2023-08-10 07:32:58 +02:00
Pere Miquel Brull
e97d4befb1
Fix #12770 - Cleanup DL structure & Readers & Python 3.8 (#12776) 2023-08-09 16:07:16 +05:30
Pere Miquel Brull
a82c2f001c
Fix #9704 - Add inline TNS string (#12711)
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-08-07 11:04:13 +02:00
Onkar Ravgan
bf44c7996a
fixed nested records avro (#12633)
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-07-31 11:54:29 +05:30
vanshika18
72e4d0070f
Support MSSLQ handle multiple databases with pyodbc connection (#12526) 2023-07-26 10:43:24 +05:30
Ayush Shah
ab1ec50c2c
Fixes Mssql Ntext, text and Image (#12490) 2023-07-20 13:34:35 +05:30
Mayur Singal
0c6435ae9b
Fix #2957: Add support for qlik sense enterprise connector (#12450) 2023-07-19 21:08:05 +05:30