yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-08-27 18:36:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
10,268 Commits 764 Branches 156 Tags
Commit Graph

939 Commits

Author SHA1 Message Date
Nahuel
1583adee29
Fix lineage to support 'merge into' queries starting with comments (#8355) 2022-10-25 10:29:59 -07:00
Nahuel
36b12bd6f1
Fix lineage issues with merge_into and copy grants queries (#8335) 2022-10-24 17:22:22 +02:00
Suresh Srinivas
229b56e7b3
Fixes #8236 - Initialize bots from JSON data files (#8304) 2022-10-21 08:38:52 -07:00
Milan Bariya
1565aa7733
FIX: Trino&Presto catalogs as databases (#8189) 
* FIX: Trino&Presto catalogs as databases

* FIX: Trino&Presto catalogs as databases

* Change based on comments
 2022-10-18 20:00:17 +05:30
NiharDoshi99
84a89d7181
Added DOMO Pipeline (#8133) 2022-10-18 17:52:28 +05:30
Milan Bariya
5b3c877dd1
FIX:connection failed because of having '@' character in uname and pass (#8172) 2022-10-17 16:57:55 +05:30
Pere Miquel Brull
d48fd468d4
Tag PATCH + Cleanup of helpers methods (#8150) 
* cleanup

* lint

* Add tag patch

* Fix rename

* Dont kill tests
 2022-10-15 14:56:30 +02:00
Milan Bariya
e7a3770240
Add: databricks usage and lineage added (#7386) 2022-10-14 07:06:28 +05:30
NiharDoshi99
0e44f7d35d
Fixed: Intergrated DOMO Connector APIs (#7989) 2022-10-13 10:55:48 +05:30
Teddy
f883863b8a
Fixes #7490 - Split Profiler and TestSuite Interface (#8032) 
* Clean up test suite workflow and interface

* Fixed tests

* Split profiler and testSuite interfaces

* Cleaned up workflows and runners

* Fixed code formatting

* - remove old code
- remove `table` attribute used for testing and used mock instead

* Fixed execution bugs from refactor

* Fixed static type checking for profiler/api/workflow.py

* Fixed linting

* Added __init__ files
 2022-10-11 15:57:25 +02:00
Ashish Gupta
8ddab79dc1
UI :- Remove rc-tree and react-table dependencies (#8049) 
* Remove rc-tree and react-table dependencies

* fix unit test issue

* File Formatted

Co-authored-by: Ayush Shah <ayush@getcollate.io>
 2022-10-11 15:42:58 +05:30
Teddy
3b7f576d04
Fixed linting for Profiler (#7922) 
* - Fixed linting
- Added logic to skip partition check when engine is not BQ
- Added ingestion partition logic to testSuite

* Fixed python formating

* Fixed test for BQ partition
 2022-10-11 09:36:36 +02:00
Pere Miquel Brull
f031293af0
Fix Pylint ingestion/bulksink and ingestion/stage (#8030) 
* Pylint ingestion bulksink

* Pylint stage

* Remove test

* Remove stage
 2022-10-10 14:06:43 +02:00
Nahuel
5c499d2a7a
Add CLI E2E tests for MySQL (#8041) 
* Add CLI E2E tests for MySQL

* Fix setup.py and pylint

* Add missing doc and update code after pylint refactor
 2022-10-10 11:36:20 +02:00
Teddy
e244a241c3
added logic to clean up quotes from table constraints (#7959) 2022-10-05 16:09:33 +02:00
Milan Bariya
cd43f1fdcf
Fix: connector for hive has double dash (#7953) 2022-10-05 12:01:13 +02:00
Teddy
15f7c4aa41
Fix param name for median test (#7942) 
* Fixed param name for median test

* Fixed unite test for median DQ
 2022-10-05 06:32:28 +02:00
Teddy
63b5233fd3
Added util command to overwrite dag_generated_config path in DAG file (#7831) 
* Added util command to overwrite dag_generated_config path in DAG file

* Added logic to only write file on change
 2022-10-03 09:24:47 +02:00
Nahuel
593ca3a4a0
Fix bot creation and edition logic (#7796) 
* Fix bot creation and edition logic

* Fix minor error creating user

* Fix failing test

* Minor fixes

* Add missing tests for new flows

* Fix put_failIfBotHasARelationshipToAnotherUser test

* Changes after manual testing

* Move where auth_provider is retrieved in the secret manager
 2022-10-01 11:25:56 +02:00
Teddy
962866a30e
Fixes #7280 - Implements support for BYTEA DType (#7749) 
* Added support for postgres bytea data type

* Added BYTEA support for profiler and converts bytes to hex on results read
 2022-09-28 13:28:35 +02:00
Nahuel
df909703eb
Fix#6615: Fix failing test still in CI (#7725) 
* Fix failing test still some tests are failing

* Fix bug in service entity repository

* Fix all tests except Ingestion Pipeline integration tests

* Fix Ingestion Pipeline integration tests

* Fix failing Python tests
 2022-09-26 20:39:42 +02:00
Ayush Shah
56c71c4101
Fix Ingestion Time Partitioned Tables on Profiler (#7506) 2022-09-26 19:41:40 +05:30
Onkar Ravgan
35efd49256
Added control for DBT descriptions (#7653) 
* Added control for DBT descriptions

* Fixed tests

* Added UI changes

* fixed maven ci tests

* Java formatting changes

* ui review fixes

* Fixed pytests

* Fixed python integration tests

* fixed airflow tests

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
 2022-09-26 16:19:47 +05:30
NiharDoshi99
8adcf2ee3d
Fixed: Dagster to work with its API (#7579) 2022-09-23 17:46:40 +05:30
Teddy
f2bf5194bb
Fixes #7623 -- Added logic to encode and decode entityLink (#7670) 
* Encode entityLink string when processing request

* Added logic to decode column type from entityLink

* mvn code formating

* Extracted unquote step into its own function
 2022-09-23 09:42:33 +02:00
Teddy
c34281251f
Fixes #7509 - implement import path migration script (#7515) 
* Added script to migarte import

* Added tests

* Updated doc to include info about import migration

* renamed test resource files
 2022-09-22 14:43:25 +02:00
Ayush Shah
517e9a4aac
Remove unused imports (#7657) 2022-09-22 16:42:10 +05:30
Sriharsha Chintalapani
adaa8e7493
Fix #6917: startTs & endTs filters providing paginated data (#7617) 
* Fix #6917: startTs & endTs filters providing paginated data

* Updated ui side changes

* Fixed OM tests end to end test

* Fix #6917: startTs & endTs filters providing paginated data

* Fix #6917: startTs & endTs filters providing paginated data

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
 2022-09-21 13:33:47 -07:00
Pere Miquel Brull
1578fa7f1b
Fix #7121 - Support Spark metastore DB connection (#7520) 
* Fix #7121 - Support Spark metastore DB connection

* appname

* Update docs

* test validation

* Address PR comments

Co-authored-by: Nahuel <nahuel@getcollate.io>
 2022-09-20 16:47:57 +02:00
Nahuel
2a6c6134f4
Fix#7272: Improve logging when initializing workflow from CLI (#7522) 
* Improve logging when initializing workflow from CLI

* Fix broken tests
 2022-09-19 08:00:00 -07:00
Teddy
1ba6e284fe
Fixes #7118 by cleaning up test names (#7494) 
* Cleaned up tests names and add registry name tests

* Updated documentation for test types supported by OM
 2022-09-16 07:04:56 +02:00
Teddy
3fdbe6ecfb
Added logic to report column level metric computation failure for profiler (#7465) 2022-09-15 17:39:29 +02:00
NiharDoshi99
5eb45948bd
Fixed: PostGIS geometry type (#7382) 
Fixed: PostGIS geometry type (#7382)
 2022-09-15 10:35:15 +02:00
Sriharsha Chintalapani
656b50dd3a
Fix #7469: Refactor OpenMetadata code modules (#7474) 2022-09-14 23:14:02 -07:00
Teddy
71679137a0
Fixes #6009 - Adds SQA MetaData object instance per DB (#7388) 
* Added SQA MetaData object instance per DB to allow same schema/table name across multiple db

* Update ingestion/src/metadata/orm_profiler/orm/converter.py

updated type hint
 2022-09-12 18:05:36 +02:00
Teddy
0677b5258f
Fixes #7299 by updating GE submodule to new test API (#7332) 
* Update GE logic to the new test API

* Cleaned up part of the code from
update to TestSuiteMixin

* Added + fixed tests

* Deleted unused files for GE tests

* fixed python tests
 2022-09-11 20:07:19 -07:00
Nahuel
b829a2cbf3
Fix: Improvements on secret manager implementation (#7282) 
* Change local secret manager by noop

* Update openmetadata-secure-test.yaml
 2022-09-07 09:18:59 +02:00
Sriharsha Chintalapani
d29a230726
Fix #7104: Add support for get resource permission by FQN (#7270) 2022-09-06 12:30:52 -07:00
Onkar Ravgan
9be3df4a1e
Integrated DBT tests (#7122) 
* Integrated DBT tests

* Fixed py tests

* Review comments addressed

* Changed test suite name default value

* Added fixes

* update ui changes for new field

* send data along with api

* update name

* fixes for UI

* fixes for UI

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
 2022-09-05 22:25:31 +05:30
Teddy
9dbcb3911b
Fix minor column data quality test bugs (#7111) 
* Fixed test name issue + filtered out partition details for non BQ tables

* Exclude non BQ table from partition processing

* Fixed test + formating
 2022-09-01 13:47:00 +02:00
Teddy
ef41382cb1
Fixes #7094 by fixing minior bugs in table tests (#7095) 2022-08-31 21:35:33 +02:00
Teddy
a39c4db8e7
Add partial support for BQ partitioned table (#7066) 
* Added support for BQ time based partition (not ingestion)

* Fixed minor errors in test suite workflow
 2022-08-30 11:39:15 -07:00
Mayur Singal
791245c0e8
Redshift Mark Partitioned Tables (#7063) 2022-08-30 18:40:51 +02:00
Pere Miquel Brull
d2d3151cca
Simplify lineage logs (#7062) 2022-08-30 17:25:40 +02:00
Mayur Singal
5a3fcafb3b
Snowflake Mark Partitioned Tables (#7021) 
Snowflake Mark Partitioned Tables (#7021)
 2022-08-30 14:35:09 +02:00
Nahuel
4c6f0c7bda
Fix#6027: Fix logging in metadata package modules except ingestion and orm_profiler (#6970) 
* Fix logging in metadata package modules except ingestion and orm_profiler

* Reduce log level to _get_columns_with_constraints method
 2022-08-29 06:46:06 +02:00
Pere Miquel Brull
6cebf37b6c
Fix #6906 - Add Nifi connector (#6956) 
* Add Nifi connector

* Mock token

* Format

* pass verify param
 2022-08-26 21:50:01 +02:00
Teddy
ce578e73d4
Fixes #5831 by implenting testSuite workflow logic (#6911) 
* Added database filter in workflow

* Removed association between profiler and data quality

* fixed tests with removed association

* Fixed sonar code smells and bugs

* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample

* moved status to workflow

* Fixed tests

* removed test logic from profiler sink

* Added logic to return sample from workflow sample value

* Added profiler examples

* Updated documentation for profiler

* Fixed code smells

* commited changed to profiler

* initial commit of the revamp workflow

* Fixed python formating

* cleaned up profiler submodule by removing test related files and functions

* Added airflow DAG logic for testSuite workflow

* Fixed code smells + added airflow ingestion tests + fixed comments
 2022-08-25 10:01:28 +02:00
NiharDoshi99
f0f0476396
Added http & https scheme options in Hive connector (#6899) 
* Added other scheme options in Hive connector

* Added other scheme options in Hive connector
 2022-08-24 18:30:30 +05:30
Ayush Shah
322f6445e1
Antlr entitylink split fix (#6901) 
* Modified Antlr Split Listeners

* Modified EntityLink.g4

* Fix Grammar split
 2022-08-24 14:05:21 +02:00