* fix: optimize system metrics retrieval for memory
* fix: ran python linting
* fix: logic to retrieve unique system metrics operations
* fix: added logic to clean up query before parsing it
* fix: added E2E tests for rds, bq, snflk system metrics
* fix: ran python linting
* fix: fix postgres query + add default byte size to env var
* fix: ran python linting
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* removed print statements and ran make py_format
* updated to fix some pylint errors.
imported Dialects to remove string compare to "impala" engine
* moved huge comment into function docstring.
This comment shows us the sql to get quartiles in Impala
* added cast to decimal for column when running average in mean.py
* fixed lint error
* fixed ui ordering of precision and scale.
Precision should be ordred in front of scale since the precision is set first in decimal data types
* Fixed overflow error when converting large numbers to bigint
Fixed error for CHAR datatype missing.
* Fixed NaN issues with Impala Profile
* py formatting
* Fixed warnings from SqlAlchemy
The GenericFunction 'max' is already registered and is going to be overridden.
The GenericFunction 'min' is already registered and is going to be overridden.
Updated Min/Max to handle strings by getting they length.
* Updated profiler to handle strings by using the string length as the parameter to compute the profile
* py_format updates
* fix: ran linting
* fix: Mysql hardcoded table alias
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* updated metadata to work with the impala query engine.
Uses the describe function to grab column names, data types, and comments.
* added the ordinalPosition data point into the Column constructor.
* renamed variable to better describe its usage.
* updated profile errors.
Hive connections now comment columns by default.
* removed print statements
* Cleaned up code by pulling check into its own function
* Updated median function to return null when it is being used for first and third quartiles.
* removed print statements and ran make py_format
* updated to fix some pylint errors.
imported Dialects to remove string compare to "impala" engine
* moved huge comment into function docstring.
This comment shows us the sql to get quartiles in Impala
* added cast to decimal for column when running average in mean.py
* fixed lint error
* fixed ui ordering of precision and scale.
Precision should be ordred in front of scale since the precision is set first in decimal data types
* first pass for impala connector
* updated default auth_mechanism to be one of the enum values.
* updated UI documentation to match fields for the impalaconneciton.
refined impalaConnection to bring use_ssl to a boolean instead or relying on an extra connection option being manually added.
Removed reference to hive for type mapping
added impala to the pip setup
* py_format updates
* removed print statement
* Lints and fixes
* Updated database documentation to follow new style
* Flag as BETA
* Remove tests
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* feat: extracted builder logic into factory and cleaned up comments
* feat: added E3E logic for time based profiler partition + logic for testing detailed profiler results
* Fixed tableau url
* review comments and tests
* changes to remove host-port addition from the UI for dashboard and chart urls
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Deprecate location and old storage service
* Format
* Fix test
* Refactor
* Clean location
* Rename object store to storage
* Rename object store to storage
* Rename object store to storage
* Format
* Format
* Refactor object store for storage
* Refactor object store for storage
* Rename object store to storage
* Fix test
* Fix test
* Format
* chore(ui): change Objectstore to Storage
* Fixes
* Fix test
* Remove storage service from Glue cypress
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Organise calls
* Prepare skeleton
* Add looker model handling
* Parse files as sql
* Handle labels
* Linting
* Format
* Fix version
* Also check the API for explore lineage
