13 Commits

Author SHA1 Message Date
Onkar Ravgan
b539b299ee
Integrated schema parsers (#9305)
* Integrated schema parsers

* Addressed review comments

* fixed pytests
2022-12-15 16:54:55 +05:30
Ayush Shah
a6ae9fd11a
Add Test Suite Implementation for Datalake (#9235) 2022-12-14 21:14:51 +05:30
Teddy
3cad959e44
Fixes #6760 -- Implements REGEX for regex test (#9033)
feat(testCase): impelemented regex logic for test suite
2022-11-29 13:00:28 +01:00
Teddy
989f2911c2
Fixes #7810 - Allow to only pass min or max (#8474)
* ISSUE-7810 Added default values for min and max
For all data validations on columns:-
min_bound is set to float("-inf"), if there is no next value
max_bound is set to float("inf"), if there is no next value

* Fixed PR errors by removing tuple + added tests

Co-authored-by: demi <deepak1212365@gmail.com>
2022-11-01 13:26:51 +01:00
Teddy
f883863b8a
Fixes #7490 - Split Profiler and TestSuite Interface (#8032)
* Clean up test suite workflow and interface

* Fixed tests

* Split profiler and testSuite interfaces

* Cleaned up workflows and runners

* Fixed code formatting

* - remove old code
- remove `table` attribute used for testing and used mock instead

* Fixed execution bugs from refactor

* Fixed static type checking for profiler/api/workflow.py

* Fixed linting

* Added __init__ files
2022-10-11 15:57:25 +02:00
Teddy
15f7c4aa41
Fix param name for median test (#7942)
* Fixed param name for median test

* Fixed unite test for median DQ
2022-10-05 06:32:28 +02:00
Teddy
f2bf5194bb
Fixes #7623 -- Added logic to encode and decode entityLink (#7670)
* Encode entityLink string when processing request

* Added logic to decode column type from entityLink

* mvn code formating

* Extracted unquote step into its own function
2022-09-23 09:42:33 +02:00
Teddy
1ba6e284fe
Fixes #7118 by cleaning up test names (#7494)
* Cleaned up tests names and add registry name tests

* Updated documentation for test types supported by OM
2022-09-16 07:04:56 +02:00
Sriharsha Chintalapani
656b50dd3a
Fix #7469: Refactor OpenMetadata code modules (#7474) 2022-09-14 23:14:02 -07:00
Teddy
9dbcb3911b
Fix minor column data quality test bugs (#7111)
* Fixed test name issue + filtered out partition details for non BQ tables

* Exclude non BQ table from partition processing

* Fixed test + formating
2022-09-01 13:47:00 +02:00
Teddy
ef41382cb1
Fixes #7094 by fixing minior bugs in table tests (#7095) 2022-08-31 21:35:33 +02:00
Teddy
a39c4db8e7
Add partial support for BQ partitioned table (#7066)
* Added support for BQ time based partition (not ingestion)

* Fixed minor errors in test suite workflow
2022-08-30 11:39:15 -07:00
Teddy
ce578e73d4
Fixes #5831 by implenting testSuite workflow logic (#6911)
* Added database filter in workflow

* Removed association between profiler and data quality

* fixed tests with removed association

* Fixed sonar code smells and bugs

* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample

* moved status to workflow

* Fixed tests

* removed test logic from profiler sink

* Added logic to return sample from workflow sample value

* Added profiler examples

* Updated documentation for profiler

* Fixed code smells

* commited changed to profiler

* initial commit of the revamp workflow

* Fixed python formating

* cleaned up profiler submodule by removing test related files and functions

* Added airflow DAG logic for testSuite workflow

* Fixed code smells + added airflow ingestion tests + fixed comments
2022-08-25 10:01:28 +02:00