* Services sensitive passwords fields are stored in secrets store
* Update services on application startup in case of changes in the JSON Schema
* Minor changes after manual test
* Bots sensitive passwords fields are stored in secrets store
* Ingestion pipeline sensitive passwords fields are stored in secrets store
* Fix Java style
* Add update for ingestion pipelines on start up
* Add pylint exception
* Fix broken test
* Update UI references
* Added support for 'Metadata Service' connectors
* Fixed failing unit tests
* Removed delete button for 'Metadata Service' connector of type OpenMetadataServer
* Code optimization for SelectServiceType component
* Fixed errors arrived due to backend changes related to OpenMetadata Services
* init reindex
* Added additional step for adding Metadata to ES configs while creating ingestion pipeline for Metadata Service
* Add ES reindex
* Add ES reindex
* Format
* Format
* Rename service
* Pick service name from IngestionPipeline
* fix ui conflicts
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* Removed part of the secrets manager implementation in the server side
* Removed part of the secrets manager implementation in the openmetadata python library side
* Remove deprecated test
* Address pylint checks
* Address new pylint checks
* Address PR comments
* Fix import on airflows apis
* Clear singleton instances for running Ometa secrets manager test
* Added SystemServices for creating DataInsight and ElasticSearch Reindexing Pipeling with Sink config
* Add Amundsen config
* Changed SystemService to MetadataService
* Change to MetadataConnection
* Fixed failing test
* Fixed Failing test
* Added Metadata Service Test
* Bootstrapped Openmetadat service
* checkstyle fix
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* added domo to test subpackage
* cleaned up branch by removing commit from issue-8353 that were not merged in main
* Added web analytics data refinement
* Added get_status function
* Added from __futur__ for typing
* Added dailyActiveUsers aggregation
* Added page views entities aggregation and active users aggregation
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* added domo to test subpackage
* cleaned up branch by removing commit from issue-8353 that were not merged in main
* Added web analytics data refinement
* Added get_status function
* Added from __futur__ for typing
* Fixed typos brought up during reviews
* Fix Json Schema filename
* Remove formatting
* FIx required property and change data type from array to object
* added items to array type
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* Fixed: dagster to connect to cloud
* Fixed: changed test_cases
* Fixed: lineage between the assets and jobs
* Changes as per comments
* Fix python checkstyle
* Fix as per comments
* Fix added migration scripts
* minor fixes
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* Fix pageViewEvent import issue
* Changed dataType to PERCENTAGE and NUMBER
* Cleaned up PR by removing metadata resource and ingestionPipeline sink logic
* Update openmetadata-service/src/main/resources/json/data/dataInsight/totalEntitiesByTier.json
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Update openmetadata-service/src/main/resources/json/data/dataInsight/totalEntitiesByType.json
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Fix#6141: Ingestion Pipeline Status Updates
* List Pipeline Status API & Improvements
* Rename State field to PipelineState in UI
* Convert Pipeline Status array to single object
* fix braking UI
* Rebase Fixes
* Profiler, TestStuite & DataInsigts Pipeline
* py_format
* fix logs page not loading
add pipelineStatus endpoint
* fix recent run changes
* Fix Tests
* address review comments for ui
* fix failing checks
* fix unit tests
* fix cypress test
* remove loader test as not using it
* wait for API when we click on the tab
* fix cypress waitFor api
* fix failed cypress tests
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* added data insight module to refine and ingest data
* Added json schema for data insight workflow
* Added mixin for data insight
* Added tests for data insight workflow and methods
* Fixed python style
* Fixed python linting
* Fixed unit test
* Added data insight worflow + workflow logging
* Fix python style
* Renamed analytic -> data_insight + added test to make + fixed style
* Fixed java style
* Fixed code smells + python style
* Ignore ES linting for now
* Fixed comments from review
* fix python formatting
* Fixe PipelineType attribute for data insight
* Fixes#8326 Add roles to system bots during initialization and use policy based authorization for bots
* Remove var usage and instead use specific types for variables
* Fix bot can't create entities
* Initial implementation of report endpoint
* Modified model files
* Added migration
* Added tests for analytics/report endpoint
* Finalized implementation of the reports endpoint
* Fixed maven test failure
* Fixed java style
* Added policy for reportDefinition
* Fixed minor typos in code
* Added database migration for web analytic event type
* Added logic for collect endpoint
* Added JSON Schema for collect endpoint
* Added index on table columns + added resource description for web event entities
* Fixed policy tests
* Updated add web analytic function to return the body of the request + updated test
* Added referer URL to PageView event
* Changed collection name to align with API endpoint
* updated initSeed method to initializeEntity
* cleaned up PR by removing unused filed from branch
* Fixed requested changes from review
* Clean up names
* Fixed comma in SQL statement
