* refactor(profiler): integrated getter func.
Removed metric getter function from their own file.
Added metric getter to their own interface classs.
created dispatch by value methdo to dispatch metric getter func.
* feature(profiler): added systemProfiler schema
* feat(profiler): workflow fresh. & snflk impl.
* feat(profiler): freshness endpoint for put and get
* feat(profiler): added system met. for redshift
* feat(profiler): freshness met. for bigquery
* fix(profiler): keyword not found in func
* feat(profiler): Added sample data for freshness
* fix(profiler): fetch previous day for BQ
* fix(profiler): sonar + data fetching logic
* fix: typo in SystemMetric Class
* fix: linting
* fix: extracted out EntityList class into models.py
* Clean up test suite workflow and interface
* Fixed tests
* Split profiler and testSuite interfaces
* Cleaned up workflows and runners
* Fixed code formatting
* - remove old code
- remove `table` attribute used for testing and used mock instead
* Fixed execution bugs from refactor
* Fixed static type checking for profiler/api/workflow.py
* Fixed linting
* Added __init__ files
* Added database filter in workflow
* Removed association between profiler and data quality
* fixed tests with removed association
* Fixed sonar code smells and bugs
* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample
* moved status to workflow
* Fixed tests
* removed test logic from profiler sink
* Added logic to return sample from workflow sample value
* Added profiler examples
* Updated documentation for profiler
* Fixed code smells
* commited changed to profiler
* initial commit of the revamp workflow
* Fixed python formating
* cleaned up profiler submodule by removing test related files and functions
* Added airflow DAG logic for testSuite workflow
* Fixed code smells + added airflow ingestion tests + fixed comments
* Fix#6571: Add EntityLink for the testCase to ID columns
* Fix#6571: Add EntityLink for the testCase to ID columns
* Fix#6782: Separate TableProfile and ColumnProfile api calls
* Fix#6782: Separate TableProfile and ColumnProfile api calls - fix tests
* Fix#6782: Separate TableProfile and ColumnProfile api calls - fix tests
* Fix setFields
* Fix tests
* Update pipeline status endpoint
* updated ui side as per new schema for profiler tab
* updated profiler details with new API
* Fix Profiler tests and validation errors (#6827)
* add profilerSample field in TableProfile
* add profilerSample field in TableProfile
* get columnProfile with field profile
* get columnProfile with field profile
* Fixed sample data and python tests
* fixed date range filter change issue
* handled empty profiler case
* Added column level test case and results
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* Added database filter in workflow
* Removed association between profiler and data quality
* fixed tests with removed association
* Fixed sonar code smells and bugs
* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample
* moved status to workflow
* Fixed tests
* removed test logic from profiler sink
* Added logic to return sample from workflow sample value
* Added profiler examples
* Updated documentation for profiler
* Fixed code smells
* Added database filter in workflow
* Removed association between profiler and data quality
* fixed tests with removed association
* Fixed sonar code smells and bugs
* Added tests for multithreading SQA interface
* Added multithread support for metric computation
* Added thread ID to log debuger
* Cleaned up tests
* Fixed python formatting issues
* Added non blocking result processing + threadCount in config file to set numbers of threads
* Added frontend input field to set number of threads
* Fixed code smell, bug and comments from reviewer
