Teddy ce578e73d4
Fixes #5831 by implenting testSuite workflow logic (#6911)
* Added database filter in workflow

* Removed association between profiler and data quality

* fixed tests with removed association

* Fixed sonar code smells and bugs

* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample

* moved status to workflow

* Fixed tests

* removed test logic from profiler sink

* Added logic to return sample from workflow sample value

* Added profiler examples

* Updated documentation for profiler

* Fixed code smells

* commited changed to profiler

* initial commit of the revamp workflow

* Fixed python formating

* cleaned up profiler submodule by removing test related files and functions

* Added airflow DAG logic for testSuite workflow

* Fixed code smells + added airflow ingestion tests + fixed comments
2022-08-25 10:01:28 +02:00

35 lines
874 B
YAML

source:
type: TestSuite
serviceName: <Workflow Name>
sourceConfig:
config:
type: TestSuite
processor:
type: "orm-test-runner"
config:
testSuites:
- name: <test suite name>
description: <this is a description>
testCases:
- name: <test case name>
description: <test case description>
testDefinitionName: <name of the test definition for this test case>
entityLink: "<#E::table::fqn> or <#E::table::fqn::columns::column_name>"
parameterValues:
- name: <parameter name>
value: <value>
- name: <parameter name>
value: <value>
...
...
sink:
type: metadata-rest
config: {}
workflowConfig:
openMetadataServerConfig:
hostPort: http://localhost:8585/api
authProvider: no-auth