10 Commits

Author SHA1 Message Date
Teddy
e57849b732
Fixes #12298 - Update report data type to camel case (#13505)
* fix: updated DI to camelCase

* fix: ran linting

* fix: added migration

* fix: remove extra parenthesis in migration file

* fix: psql migration query

* fix: OS compose host

* fix: removed commented code block
2023-10-11 08:14:21 +02:00
Pere Miquel Brull
0282574bdd
Create ometa client once and pass it around & improve pycln config (#13310)
* Create ometa client once and pass it around & improve pycln config

* Fix

* Fix

* Fix tests

* Fix maven ci

* Fix tests

* Fix tests

* Fix tests

* Format

* Fix DI
2023-10-04 09:14:03 +02:00
Teddy
6ca71ae323
Issue 12679 - Handle Entity Object Instantiation Error + Refactor Workflow (#13384)
* feat: updated DI workflow to inherit from BaseWorkflow + split processor and producer classes

* feat: __init__.py files creation

* feat: updated workflow import classes in code and doc

* feat: moved kpi runner from runner to processor folder

* fix: skip failure on list entities

* feat: deleted unused files

* feat: updated status reporter

* feat: ran linting

* feat: fix test error with typing and fqn

* feat: updated test dependencies

* feat: ran linting

* feat: move execution order up

* feat: updated cost analysis report to align with new workflow

* feat: fix entity already exists for pipeline entity status

* feat: ran python linting

* feat: move skip_on_failure to method

* feat: ran linting

---------

Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-10-02 12:05:30 +02:00
Teddy
e9ef7b5e81
Issue-12857: Remove ES Dependency from DI Workflow (#13303)
* feat: move elasticsearch indexing to backend + introduced EntityTimeSeries interface for timeseries type object

* feat: make reportData.json inherit from EntityTimeSeriesInterface

* feat: updated type to Object

* feat: deleted elasticsearch dependencies

* feat: removed elasticsearch indexing from workflow

* feat: added data insight sample data

* feat: cleaned up tests
2023-09-21 16:17:47 -07:00
Teddy
8e4388c35e
Issue 11345 (#12859)
* feat: added serviceName dimension to entity report

* feat: fix python test
2023-08-14 08:05:14 +02:00
Chirag Madlani
7adc291364
fix(ui): circular deps for entityReference.json (#11760)
* fix(ui): circular deps for entityReference.json

* Fix circular Dependency python

* Cap Delta Spark version

---------

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2023-05-26 18:02:21 +05:30
Pere Miquel Brull
93381a7343
Add Pylint print checker and py-checkstyle as required (#8849)
* Always run the checkstyle

* Add print checker lint plugin

* Typos

* Add print checker

* Fix e2e
2022-11-17 16:44:26 +00:00
Teddy
9d594efcb5
Fixes #8473 - Implements KPI Workflow (#8657)
* Added KPI workflow logic

* Update ingestion/src/metadata/data_insight/runner/run_result_registry.py

* ran py format

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-11-11 06:35:33 +01:00
Teddy
b44972ef60
Fixes #8470 - Implements refinement functions for web analytics events (#8528)
* Moved webanalytics type in its own folder

* Added data insight chart api endpoint

* Jave formatting

* Added resource descriptor

* Added metadata entity endpoint

* Added aggregation endpoint for dataInsight

* Fix tag name

* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight

* added domo to test subpackage

* cleaned up branch by removing commit from issue-8353 that were not merged in main

* Added web analytics data refinement

* Added get_status function

* Added from __futur__ for typing

* Fixed typos brought up during reviews
2022-11-07 17:08:20 +01:00
Teddy
83b048c07c
Fixes #8206 -- Implement data insight refinement for entity report data (#8351)
* added data insight module to refine and ingest data

* Added json schema for data insight workflow

* Added mixin for data insight

* Added tests for data insight workflow and methods

* Fixed python style

* Fixed python linting

* Fixed unit test

* Added data insight worflow + workflow logging

* Fix python style

* Renamed analytic -> data_insight + added test to make + fixed style

* Fixed java style

* Fixed code smells + python style

* Ignore ES linting for now

* Fixed comments from review

* fix python formatting

* Fixe PipelineType attribute for data insight
2022-10-26 11:18:08 +02:00