OpenMetadata/openmetadata-docs/content/metadata-ui/data-quality/tests.md

title	slug
Tests	/metadata-ui/data-quality/tests

Tests

Here you can see all the supported tests and how to configure them.

A Test Case adds logic to the Metrics results. A Metric is neither good nor wrong, so we need the Test definitions to map results into Success or Failures.

In this section, you will learn what tests we currently support and how to configure them.

UI Configuration

You can navigate in the UI to a Table Entity and Add Tests from there. The form will help you select the type of test, the column, and its configurations.

From the Data Quality Tab you can create both Table and Column Tests:

<Image src={"/images/metadata-ui/data-quality/tests/write-your-first-tests.png"} alt="Write your first test" caption="Write your first test" />

<Image src={"/images/metadata-ui/data-quality/tests/sample-form-to-create-test.png"} alt="Sample form to create a column test" caption="Sample form to create a column test" />

Directly from the Profiler tab, you can create a Column Test in the column of your choice:

<Image src={"/images/metadata-ui/data-quality/tests/create-test-from-profiler-tab.png"} alt="Create a column test from the profiler tab" caption="Create a column test from the profiler tab" />

If you'd rather configure the tests directly in the Workflow JSON, we'll show examples for each of them below.

Table Tests

Tests applied on top of Table Metrics.

Table Row Count to Equal

Validate that the rowCount metric is equal to a given value.

Properties:

• value: Expected number of rows.

JSON Config

"testCase": {
    "config": {
        "value": 100
    },
    "tableTestType": "tableRowCountToEqual"
}

Table Row Count to be Between

Validate that the rowCount metric is within a given range of values.

Properties:

• minValue: Lower bound of the interval. If informed, the number of rows should be bigger than this number.
• maxValue: Upper bound of the interval. If informed, the number of rows should be lower than this number.

Any of those two need to be informed.

JSON Config

"testCase": {
    "config": {
        "minValue": 10,
        "maxValue": 100
    },
    "tableTestType": "tableRowCountToBeBetween"
}

Table Column Count to Equal

Validate that the number of columns in a table is equal to a given value.

Properties

• columnCount: Expected number of columns.

JSON Config

"testCase": {
    "config": {
        "columnCount": 7
    },
    "tableTestType": "tableColumnCountToEqual"
}

Column Tests

Tests applied on top of Column metrics.

Column Values to Be Unique

Makes sure that there are no duplicates in a given column.

Properties

• columnValuesToBeUnique: To be set as true. This is required for proper JSON parsing in the profiler module.

JSON Config

"testCase": {
    "config": {
        "columnValuesToBeUnique": true
    },
    "columnTestType": "columnValuesToBeUnique"
}

Column Values to Be Not Null

Validates that there are no null values in the column.

Properties

• columnValuesToBeNotNull: To be set as true. This is required for proper JSON parsing in the profiler module.

JSON Config

"testCase": {
    "config": {
        "columnValuesToBeNotNull": true
    },
    "columnTestType": "columnValuesToBeNotNull"
}

Column Values to Match Regex

This test allows us to specify how many values in a column we expect that will match a certain SQL LIKE expression.

Properties

• regex: SQL LIKE expression to match. E.g., %something%.

JSON Config

"testCase": {
    "config": {
        "regex": "%something%"
    },
    "columnTestType": "columnValuesToMatchRegex"
}

Column Values to Be Not In Set

Validate that there are no values in a column in a set of forbidden values.

Properties

• forbiddenValues: List of forbidden strings or numbers.

JSON Config

"testCase": {
    "config": {
        "forbiddenValues": ["forbidden1", "forbidden2"]
    },
    "columnTestType": "columnValuesToBeNotInSet"
}

Column Values to Be Between

Validate that the values of a column are within a given range.

Only supports numerical types.

Properties

• minValue: Lower bound of the interval. If informed, the column values should be bigger than this number.
• maxValue: Upper bound of the interval. If informed, the column values should be lower than this number.

Any of those two need to be informed.

JSON Config

"testCase": {
    "config": {
        "minValue": 10,
        "maxValue": 100
    },
    "columnTestType": "columnValuesToBeBetween"
}

Column Values Missing Count to Be Equal

Validates that the number of missing values matches a given number. Missing values are the sum of nulls, plus the sum of values in a given list which we need to consider as missing data. A clear example of that would be NA or N/A.

Properties

• missingCountValue: The number of missing values needs to be equal to this. This field is mandatory.
• missingValueMatch: A list of strings to consider as missing values. Optional.

JSON Config

"testCase": {
    "config": {
        "missingCountValue": 100,
        "missingValueMatch": ["NA", "N/A"]
    },
    "columnTestType": "columnValuesMissingCountToBeEqual"
}

Column Values Lengths to Be Between

Validates that the lengths of the strings in a column are within a given range.

Only supports concatenable types.

Properties

• minLength: Lower bound of the interval. If informed, the string length should be bigger than this number.
• maxLength: Upper bound of the interval. If informed, the string length should be lower than this number.

Any of those two need to be informed.

JSON Config

"testCase": {
    "config": {
        "minLength": 4,
        "maxLength": 18
    },
    "columnTestType": "columnValueLengthsToBeBetween"
}