* fix: optimize system metrics retrieval for memory
* fix: ran python linting
* fix: logic to retrieve unique system metrics operations
* fix: added logic to clean up query before parsing it
* fix: added E2E tests for rds, bq, snflk system metrics
* fix: ran python linting
* fix: fix postgres query + add default byte size to env var
* fix: ran python linting
This is regarding: https://github.com/psf/requests/issues/4664
The change allows errors to be handled and retried.
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* removed print statements and ran make py_format
* updated to fix some pylint errors.
imported Dialects to remove string compare to "impala" engine
* moved huge comment into function docstring.
This comment shows us the sql to get quartiles in Impala
* added cast to decimal for column when running average in mean.py
* fixed lint error
* fixed ui ordering of precision and scale.
Precision should be ordred in front of scale since the precision is set first in decimal data types
* Added get_view_definition to hive and impala connectors.
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* removed print statements and ran make py_format
* updated to fix some pylint errors.
imported Dialects to remove string compare to "impala" engine
* moved huge comment into function docstring.
This comment shows us the sql to get quartiles in Impala
* added cast to decimal for column when running average in mean.py
* fixed lint error
* fixed ui ordering of precision and scale.
Precision should be ordred in front of scale since the precision is set first in decimal data types
* Fixed overflow error when converting large numbers to bigint
Fixed error for CHAR datatype missing.
* Fixed NaN issues with Impala Profile
* py formatting
* Fixed warnings from SqlAlchemy
The GenericFunction 'max' is already registered and is going to be overridden.
The GenericFunction 'min' is already registered and is going to be overridden.
Updated Min/Max to handle strings by getting they length.
* Updated profiler to handle strings by using the string length as the parameter to compute the profile
* py_format updates
* fix: ran linting
* fix: Mysql hardcoded table alias
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
