mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-11-01 19:18:05 +00:00
ad9b5a0cb5
* updated metadata to work with the impala query engine. Uses the describe function to grab column names, data types, and comments. * added the ordinalPosition data point into the Column constructor. * renamed variable to better describe its usage. * updated profile errors. Hive connections now comment columns by default. * removed print statements * Cleaned up code by pulling check into its own function * Updated median function to return null when it is being used for first and third quartiles. * updated metadata to work with the impala query engine. Uses the describe function to grab column names, data types, and comments. * added the ordinalPosition data point into the Column constructor. * renamed variable to better describe its usage. * updated profile errors. Hive connections now comment columns by default. * removed print statements * Cleaned up code by pulling check into its own function * Updated median function to return null when it is being used for first and third quartiles. * removed print statements and ran make py_format * updated to fix some pylint errors. imported Dialects to remove string compare to "impala" engine * moved huge comment into function docstring. This comment shows us the sql to get quartiles in Impala * added cast to decimal for column when running average in mean.py * fixed lint error * fixed ui ordering of precision and scale. Precision should be ordred in front of scale since the precision is set first in decimal data types * Fixed overflow error when converting large numbers to bigint Fixed error for CHAR datatype missing. * Fixed NaN issues with Impala Profile * py formatting * Fixed warnings from SqlAlchemy The GenericFunction 'max' is already registered and is going to be overridden. The GenericFunction 'min' is already registered and is going to be overridden. Updated Min/Max to handle strings by getting they length. * Updated profiler to handle strings by using the string length as the parameter to compute the profile * py_format updates * fix: ran linting * fix: Mysql hardcoded table alias --------- Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com> Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>