* Remove duplicates from entity_extension_time_series and add the constraing if missing
* Add sort buffer and work mem
* Revert "Add sort buffer and work mem"
This reverts commit fcfff5feb60c9212bb7c1cad34b524dc8c03bfc5.
* fix: removed legacy OS and ES aggregator classes
* fix: centralized aggregator business logic
* fix: implemented client specific aggregator
* fix: updated client instantiation to use client specific aggregator
* fix: clean up json schema
* fix: updated DI index names
* fix: added searchIndex + storedProcedure
* fix: ran linting
* fix: updated python test to include new entity types
* feat: updated DI workflow to inherit from BaseWorkflow + split processor and producer classes
* feat: __init__.py files creation
* feat: updated workflow import classes in code and doc
* feat: moved kpi runner from runner to processor folder
* fix: skip failure on list entities
* feat: deleted unused files
* feat: updated status reporter
* feat: ran linting
* feat: fix test error with typing and fqn
* feat: updated test dependencies
* feat: ran linting
* feat: move execution order up
* feat: updated cost analysis report to align with new workflow
* feat: fix entity already exists for pipeline entity status
* feat: ran python linting
* feat: move skip_on_failure to method
* feat: ran linting
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* feat: move elasticsearch indexing to backend + introduced EntityTimeSeries interface for timeseries type object
* feat: make reportData.json inherit from EntityTimeSeriesInterface
* feat: updated type to Object
* feat: deleted elasticsearch dependencies
* feat: removed elasticsearch indexing from workflow
* feat: added data insight sample data
* feat: cleaned up tests
* fix: updated ingestion to delete existing data for the ingestion day
* fix: added delete endpoint for the report data
* fix: added migration to delete duplicate data + added tables creation & data deltion logic in a transaction
* fix: made deletion SQL engine aware
* fix: added ES deletion logic back while we work on DI event publisher
* fix(dataInsight): handle cases where entity would be deleted from OM
* Update ingestion/src/metadata/data_insight/processor/web_analytic_report_data_processor.py
Added explanation in code comments
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* fix(dataInsight): tests failure
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* Change entityReference to entity name or fullyQualifiedName
* Change backend code and tests to use FQN
* UI change for using fqns instead of EntityReference
* Ingestion framework changes for using fqns instead of EntityReference
* Fix test failures
* Fixed python tests and sample data new
* fix: minor ui changes for fqn
* Fixed python integration tests
* Fixed superset tests
* fix UI tests
* fix type issue
* fix cypress
* fix name for testcase
---------
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* Mapping from kpi to dichart(One to one)
* Mapping from kpi to dichart(One to one)
* Removed second tests creating KPI
* Fix some ui improve as well
* review comment changed function name
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* added domo to test subpackage
* cleaned up branch by removing commit from issue-8353 that were not merged in main
* Added web analytics data refinement
* Added get_status function
* Added from __futur__ for typing
* Fixed typos brought up during reviews
* added data insight module to refine and ingest data
* Added json schema for data insight workflow
* Added mixin for data insight
* Added tests for data insight workflow and methods
* Fixed python style
* Fixed python linting
* Fixed unit test
* Added data insight worflow + workflow logging
* Fix python style
* Renamed analytic -> data_insight + added test to make + fixed style
* Fixed java style
* Fixed code smells + python style
* Ignore ES linting for now
* Fixed comments from review
* fix python formatting
* Fixe PipelineType attribute for data insight
