Imri Paran b960b60965
Fix #16421: add tableDiff test case (#16554)
* feat: add tableDiff test case

This changed introduces a "table diff" test case which
compares two tables and fails if they are not identical.
The similarity is made based on a specific "key" (because the test only makes sense when performed on ordered collections).

1. Added the `tableDiff` test definition.
2. Implemented a "runtime" parameters feature which injects additional parameters for the test at runtime.
3. Integration tests (because of course).

This feature was not tested end-to-end yet because "array" data

* pydantic v2

* format

* format

* format and added data diff to setup.py

* format

* fixed param issue which has type ARRAY

* fixed runtime_parameter_setter

* moved models to parent directory

* handle errors in table diff

* fixed issue with edit test case

* format

* added more details to pytest skip

* format

* refactor: Improve createTestCaseParameters function in DataQualityUtils

* fixed unit test

* removed unused fixture

* removed validator.py

* fixed tests

* added validate kwarg to tests_mixin

* removed "postgres" data diff extra as they interfere with psycopg2-binary

* fixed tests

* pinned tenacity for tests

* reverted tenacity pinning

* added ui support for test diff

* fixed dq cypress and added edit flow

* organized the test case

* added dialect support

* fixed tests

* option style fix

* fixed calculation for passing/failing rows

* restrict the tableDiff test to limited services

* set where to None if blank string

* fixed where clause

* fixed tests for where clause

* use displayName in place of name in edit form

* added docs for RuntimeParameterSetter

* fixed cypress

---------

Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
2024-06-20 16:54:12 +02:00

42 lines
971 B
Python

import logging
import os
import sys
import pytest
from .integration_base import int_admin_ometa
if not sys.version_info >= (3, 9):
collect_ignore = ["trino"]
def pytest_configure():
helpers_path = os.path.abspath(os.path.dirname(__file__) + "/../helpers")
sys.path.insert(0, helpers_path)
@pytest.fixture(scope="session", autouse=True)
def configure_logging():
logging.getLogger("sqlfluff").setLevel(logging.CRITICAL)
logging.getLogger("pytds").setLevel(logging.CRITICAL)
@pytest.fixture(scope="module")
def metadata():
return int_admin_ometa()
def pytest_pycollect_makeitem(collector, name, obj):
try:
if obj.__base__.__name__ in ("BaseModel", "Enum"):
return []
except AttributeError:
pass
@pytest.fixture(scope="session", autouse=sys.version_info >= (3, 9))
def config_testcontatiners():
from testcontainers.core.config import testcontainers_config
testcontainers_config.max_tries = 10