* feat: updated DI workflow to inherit from BaseWorkflow + split processor and producer classes
* feat: __init__.py files creation
* feat: updated workflow import classes in code and doc
* feat: moved kpi runner from runner to processor folder
* fix: skip failure on list entities
* feat: deleted unused files
* feat: updated status reporter
* feat: ran linting
* feat: fix test error with typing and fqn
* feat: updated test dependencies
* feat: ran linting
* feat: move execution order up
* feat: updated cost analysis report to align with new workflow
* feat: fix entity already exists for pipeline entity status
* feat: ran python linting
* feat: move skip_on_failure to method
* feat: ran linting
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* feat: move elasticsearch indexing to backend + introduced EntityTimeSeries interface for timeseries type object
* feat: make reportData.json inherit from EntityTimeSeriesInterface
* feat: updated type to Object
* feat: deleted elasticsearch dependencies
* feat: removed elasticsearch indexing from workflow
* feat: added data insight sample data
* feat: cleaned up tests
* Fix LifeCycle inconsistencies in Schema; Add DELETE api
* set autocommit to true for non transactional
* make lifecycle common field for entities
* Add LifeCycle as common entity field
* Fix python life cycle code
* Fix search indexes
* remove unnecessary constant
* Add test back to entity resource test
* Fix lint
* Fix lint
* Fix lint
* Fix lint
* Add missing schema
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* fix: updated ingestion to delete existing data for the ingestion day
* fix: added delete endpoint for the report data
* fix: added migration to delete duplicate data + added tables creation & data deltion logic in a transaction
* fix: made deletion SQL engine aware
* fix: added ES deletion logic back while we work on DI event publisher
* Fixes a bug while patching the description of a TestCase
* Update docker-compose.yml
* Update docker-compose.yml
* Ran pre-commit checks and linter
* Added some clarifying points and fixed some grammatical errors in the documentation for installation instructions.
* revert changes made to docs
* Added diffrent clients for elasticSearch and openSearch and did the interface implementation
* removed some comments and some changes
* Add Elastic Search Test Container
* upgarded es version
* added test cases for elastic search
* added test case for, when tag is deleted cehck its relationships also get deleted
* changed package of open,elastic to openSearch and elasticSearch
* added support for skip search test case if index is not present of that entity
* Refactor bunch of files
* unnecessary semicolon
* refactor test case indexing
* fix checkstyle
* checkstyle
* added hardcoded configs and changed package name
* fixed no response from hits
* stopped elasticsearch container and some fixes
* closed elasticsearch container
* revert
* closed client after used in test
* style
* edit es config in junit
* checking if using already es version release memory for maven ci
* revert docker file changes
* remove es container for testing the ci
* commented out all test cases for elasticsearch
* fix
* creating elastic search with reusable enabled
* trying to fix ci error
* start container only if it is not running
* revert docker changes and done changes for elastic search container
* changed var name of sort_order for es
* lower the version of es client
* fixes
* all latest version of es including docker
* Use Single sort order
* Fix Keyword issue
* Immediate refresh policy for entity created
* increase sleep time
* disable test
* added reindexing logic in run_local_docker.sh
* enabled test case
---------
Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
* fix: removed sqlparse dependency for system metrics
* fix: update sample query
* fix: move system test os retrieval to `.get()`
* fix: move os.environ to `get`
* feat: updated GE integration to match new test workflow
* feat: updated unit tests to match new signature
* feat: added GE integration tests
* feat: ran python linting
* feat: updated dbt ingestion to match new TestSuite workflow
* feat: ran python linting
* feat: remove testSuite from Elasticsearch event test case update
* feat: ran java linting
* Fix fqn hash
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* Add name & fqnHash and remove generated columns from top level
* merge commits
* merge commits
* merge commits
* Fix glossary import/export
* Fix BotResource Tests
* Fix BotResource Tests
* Fix Glossary Term tests
* Fix Glossary Term tests
* Fix Glossary Import/Export tests
* All backend tests are fixed.
* merge with main
* Fix tests and revert local changes
* Fix tests and revert local changes
* Fix NullPointerException for Glossary and Query
* Fix Tests
---------
Co-authored-by: Ayush Shah <ayush@getcollate.io>
* feat: added logic to delete logical test suite + added check to prevent adding existing testCases to executable test suite
* feat: added elasticsearch index logic for testCases
* feat: added deletion logic from index logic when deleting test suites
* feat: added test case index search to endpoint
* feat: add executable/logical filter in list testSuite + filterOut tables without tests in Table resource
* feat: added summary field to testSuite
* feat: added executionSummary endpoint for test cases
* feat: removed tick marks around timestamp
* feat: addressed test failures
* feat: ran python linting
* feat: add limit to fetch all tables in TableResource testSuite test
* feat: fix conflict
* feat: ran java checkstyle
* feat: fixed mongo linting + disabled mongo failing tests
* feat: removed mongo test skip
* feat: removed unsued pytest import
* fixed issue for lineage description
* fixed issue while ingesting
* fixed issue while ingesting
* added test case for Lingeage with description
* addressing comments .. enhancement
* addressing comments .. enhancement
* modified py test case and removed description from addLineage as we are not using it.
* add support for topic entity and description in lineage details
* fix pylint & test
* pytest fix
* fix column lineage null issue
---------
Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
* feat: added entityReference field in testSuite to link testSuite to an entity when the testSuite is executable.
* feat: added `executableEntityReference` as an entity reference for executable test suite to their entity
* feat: add status object to test case results
* feat: ran python linting
* feat: fixed update to
* feat: added entityReference field in testSuite to link testSuite to an entity when the testSuite is executable.
* feat: added `executableEntityReference` as an entity reference for executable test suite to their entity
* feat: add status object to test case results
* feat: ran python linting
* feat: refactor ingestion flow logic
* feat: ran python linting
* feat: update tests to match new workflow
* feat: ran python linting
* feat: update sample data test suite name
* feat: Added backend logic to support logical and executable test suites
* feat: clean up java and json code
* feat: added sample data for logical and executable test suites
* feat: remove executable from CreateTestSuite
* feat: ran python and java linting
* feat: added README info for data quality structure
* skipping cypress to keep main green
* fixed typescript type issue
---------
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
* feat: fetch metrics from system tables
* feat: add permission doc for fetching metrics from system tables
* feat: fix E2E tests to reflect full table row count after table metric update
* feat: ran linting
* feat: fix doc string engine name + function typing
* feat: ran python linting
* Deprecate location and old storage service
* Format
* Fix test
* Refactor
* Clean location
* Rename object store to storage
* Rename object store to storage
* Rename object store to storage
* Format
* Format
* Refactor object store for storage
* Refactor object store for storage
* Rename object store to storage
* Fix test
* Fix test
* Format
* chore(ui): change Objectstore to Storage
* Fixes
* Fix test
* Remove storage service from Glue cypress
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
