Imri Paran be82086e25
MINOR: add column case sensitivity parameter (#18115)
* fix(data-quality): table diff

- added handling for case-insensitive columns
- added handling for different numeric types (int/float/Decimal)
- added handling of boolean test case parameters

* add migrations for table diff

* add migrations for table diff

* removed cross type diff for now. it appears to be flaky

* fixed migrations

* use casefold() instead of lower()

* - implemented utils.get_test_case_param_value
- fixed params for case sensitive column

* handle bool test case parameters

* format

* testing

* format

* list -> List

* list -> List

* - change caseSensitiveColumns default to fase
- added migration to stay backward compatible

* - removed migration files
- updated logging message for table diff migration

* changed bool test case parameters default to always be false

* format

* docs: data diff

- added the caseSensitiveColumns parameter

requires: https://github.com/open-metadata/OpenMetadata/pull/18115

* fixed test_get_bool_test_case_param
2024-10-15 16:29:43 +02:00

35 lines
1.0 KiB
Python

from ast import literal_eval
import pytest
from metadata.data_quality.validations.base_test_handler import BaseTestValidator
from metadata.generated.schema.tests.testCase import TestCaseParameterValue
@pytest.mark.parametrize(
"param_values, name, type_, default, expected",
[
([TestCaseParameterValue(name="str", value="test")], "str", str, None, "test"),
(
[TestCaseParameterValue(name="param", value="[1, 2, 3]")],
"param",
literal_eval,
None,
[1, 2, 3],
),
([TestCaseParameterValue(name="param", value="123")], "param", int, None, 123),
(
[TestCaseParameterValue(name="param", value=None)],
"param",
str,
"default",
"default",
),
],
)
def test_get_test_case_param_value(param_values, name, type_, default, expected):
result = BaseTestValidator.get_test_case_param_value(
param_values, name, type_, default
)
assert result == expected