* refactor(profiler): integrated getter func.
Removed metric getter function from their own file.
Added metric getter to their own interface classs.
created dispatch by value methdo to dispatch metric getter func.
* feature(profiler): added systemProfiler schema
* feat(profiler): workflow fresh. & snflk impl.
* feat(profiler): freshness endpoint for put and get
* feat(profiler): added system met. for redshift
* feat(profiler): freshness met. for bigquery
* fix(profiler): keyword not found in func
* feat(profiler): Added sample data for freshness
* fix(profiler): fetch previous day for BQ
* fix(profiler): sonar + data fetching logic
* fix: typo in SystemMetric Class
* fix: linting
* fix: extracted out EntityList class into models.py
* Fix#6571: Add EntityLink for the testCase to ID columns
* Fix#6571: Add EntityLink for the testCase to ID columns
* Fix#6782: Separate TableProfile and ColumnProfile api calls
* Fix#6782: Separate TableProfile and ColumnProfile api calls - fix tests
* Fix#6782: Separate TableProfile and ColumnProfile api calls - fix tests
* Fix setFields
* Fix tests
* Update pipeline status endpoint
* updated ui side as per new schema for profiler tab
* updated profiler details with new API
* Fix Profiler tests and validation errors (#6827)
* add profilerSample field in TableProfile
* add profilerSample field in TableProfile
* get columnProfile with field profile
* get columnProfile with field profile
* Fixed sample data and python tests
* fixed date range filter change issue
* handled empty profiler case
* Added column level test case and results
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* Added database filter in workflow
* Removed association between profiler and data quality
* fixed tests with removed association
* Fixed sonar code smells and bugs
* Updated profiler workflow to:
- support only running profiler (removed test run)
- support column inclusion and exclusion
- added back support for partitioned table and sample
* moved status to workflow
* Fixed tests
* removed test logic from profiler sink
* Added logic to return sample from workflow sample value
* Added profiler examples
* Updated documentation for profiler
* Fixed code smells
* Added tests for multithreading SQA interface
* Added multithread support for metric computation
* Added thread ID to log debuger
* Cleaned up tests
* Fixed python formatting issues
* Added non blocking result processing + threadCount in config file to set numbers of threads
* Added frontend input field to set number of threads
* Fixed code smell, bug and comments from reviewer
* Fix sample data DAG
* Fix callback imports
* Use --set-default-enum-member in generate
* Format
* Add faulty merge hard_delete
* Fix airflow lineage, improve naming and fix lineage tests
* Add mysql url test
* Add mysql url test
* Update CI name
* Fix test ometa endpoint
* Format
* Fix metadata config
* Issue-3685: Variable based separator used for fullyQualifiedName instead of hardcoded . for Python and make the sepearator to :
* Fix failing test
* Use colon for run_local_docker validation
* Update tests FQDN
* Update tests FQDN
Co-authored-by: Sachin-chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
* Fix list typing
* Add sqlite service
* Add sqlite service
* Add sqlite service
* Refactor validation into class
* Refactor validation into class
* Prepare table simple profiler
* Add note
* test ORM conversion
* Prepare workflow config utilities
* Prepare workflow skeleton
* Use new core Validation
* Refactor workflow config parsing
* Add comment
* Simplify workflow validations
* Fix table metric check
* Add init for convenience, otherwise interpreter cries when trying to __call__ the get result
* Fix table metric check
* Format
* Format
* Fix table list and metrics init
* Prepare profiler workflow integration tests
* Bump version
* Fix pycharm imports
* format
