* refactor(profiler): integrated getter func.
Removed metric getter function from their own file.
Added metric getter to their own interface classs.
created dispatch by value methdo to dispatch metric getter func.
* feature(profiler): added systemProfiler schema
* feat(profiler): workflow fresh. & snflk impl.
* feat(profiler): freshness endpoint for put and get
* feat(profiler): added system met. for redshift
* feat(profiler): freshness met. for bigquery
* fix(profiler): keyword not found in func
* feat(profiler): Added sample data for freshness
* fix(profiler): fetch previous day for BQ
* fix(profiler): sonar + data fetching logic
* fix: typo in SystemMetric Class
* fix: linting
* fix: extracted out EntityList class into models.py
* Use ES to build Tables FQN when processing Lineage
* Fix pylent check
* Fix test failing
* Remove warning when calling ES with empty query
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* - made ES ingestion of data inisght idempotent for the daily execution
- added web analytic timedelta to 1 day to fecth data from the previous day
* Added time utils
* fixed timezone
* Removed part of the secrets manager implementation in the server side
* Removed part of the secrets manager implementation in the openmetadata python library side
* Remove deprecated test
* Address pylint checks
* Address new pylint checks
* Address PR comments
* Fix import on airflows apis
* Clear singleton instances for running Ometa secrets manager test
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* added domo to test subpackage
* cleaned up branch by removing commit from issue-8353 that were not merged in main
* Added web analytics data refinement
* Added get_status function
* Added from __futur__ for typing
* Fixed typos brought up during reviews
* Fixed: dagster to connect to cloud
* Fixed: changed test_cases
* Fixed: lineage between the assets and jobs
* Changes as per comments
* Fix python checkstyle
* Fix as per comments
* Fix added migration scripts
* minor fixes
* ISSUE-7810 Added default values for min and max
For all data validations on columns:-
min_bound is set to float("-inf"), if there is no next value
max_bound is set to float("inf"), if there is no next value
* Fixed PR errors by removing tuple + added tests
Co-authored-by: demi <deepak1212365@gmail.com>
* added data insight module to refine and ingest data
* Added json schema for data insight workflow
* Added mixin for data insight
* Added tests for data insight workflow and methods
* Fixed python style
* Fixed python linting
* Fixed unit test
* Added data insight worflow + workflow logging
* Fix python style
* Renamed analytic -> data_insight + added test to make + fixed style
* Fixed java style
* Fixed code smells + python style
* Ignore ES linting for now
* Fixed comments from review
* fix python formatting
* Fixe PipelineType attribute for data insight
* Clean up test suite workflow and interface
* Fixed tests
* Split profiler and testSuite interfaces
* Cleaned up workflows and runners
* Fixed code formatting
* - remove old code
- remove `table` attribute used for testing and used mock instead
* Fixed execution bugs from refactor
* Fixed static type checking for profiler/api/workflow.py
* Fixed linting
* Added __init__ files
* - Fixed linting
- Added logic to skip partition check when engine is not BQ
- Added ingestion partition logic to testSuite
* Fixed python formating
* Fixed test for BQ partition
