4 Commits

Author SHA1 Message Date
Teddy
ac77f33b08
Fixes #7447 -- Add freshness metrics to profiler (#9159)
* refactor(profiler): integrated getter func.

Removed metric getter function from their own file.
Added metric getter to their own interface classs.
created dispatch by value methdo to dispatch metric getter func.

* feature(profiler): added systemProfiler schema

* feat(profiler): workflow fresh. & snflk impl.

* feat(profiler): freshness endpoint for put and get

* feat(profiler): added system met. for redshift

* feat(profiler): freshness met. for bigquery

* fix(profiler): keyword not found in func

* feat(profiler): Added sample data for freshness

* fix(profiler): fetch previous day for BQ

* fix(profiler): sonar + data fetching logic

* fix: typo in SystemMetric Class

* fix: linting

* fix: extracted out EntityList class into models.py
2022-12-07 14:33:30 +01:00
Ayush Shah
5be0f8ee76
Dl Profiler (#8694)
* DQ commit

* Add DL Profiler

* Fix Ingestion and Profliing pylint checks

* Fix Tests

* PyFormat files

* Fix Tests

* Resolve Comments

* Fix Tests and Format Files

* Resolve Comments

* Fix Pylint and Code smells

* Resolve Comments

* Fix S3 parquet

* Fix Metrics Code Smell
2022-11-15 16:01:10 +01:00
Teddy
199b342288
Fixes #8135 - Implement partitioning config for profiler (#8623)
* Added logic to handle partitioning config in profiler

* extracted get_partition_details out of workflow classes
2022-11-10 10:54:31 +01:00
Teddy
f883863b8a
Fixes #7490 - Split Profiler and TestSuite Interface (#8032)
* Clean up test suite workflow and interface

* Fixed tests

* Split profiler and testSuite interfaces

* Cleaned up workflows and runners

* Fixed code formatting

* - remove old code
- remove `table` attribute used for testing and used mock instead

* Fixed execution bugs from refactor

* Fixed static type checking for profiler/api/workflow.py

* Fixed linting

* Added __init__ files
2022-10-11 15:57:25 +02:00