* refactor(profiler): integrated getter func.
Removed metric getter function from their own file.
Added metric getter to their own interface classs.
created dispatch by value methdo to dispatch metric getter func.
* feature(profiler): added systemProfiler schema
* feat(profiler): workflow fresh. & snflk impl.
* feat(profiler): freshness endpoint for put and get
* feat(profiler): added system met. for redshift
* feat(profiler): freshness met. for bigquery
* fix(profiler): keyword not found in func
* feat(profiler): Added sample data for freshness
* fix(profiler): fetch previous day for BQ
* fix(profiler): sonar + data fetching logic
* fix: typo in SystemMetric Class
* fix: linting
* fix: extracted out EntityList class into models.py
* Fixed issue with user created Metadata Services not able to be deleted
* Fixed default values for Amundsen connection configs
* Added missing break statement in switch block
* Mask the Jwt token in Metadata service
* minor fix
* fix the icon alignment in Add Ingestion button
* disable the test connection for metadata service OpenMetadata type
* change the css name
* fix unit test issue
* Fix Auth Provider
* Fix add ingestion dropdown icon alignment
* Fix formatting
* Do not encryt JWT auth mechanism with secrets manager
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Nahuel Verdugo Revigliono <nahuel@getcollate.io>
* Services sensitive passwords fields are stored in secrets store
* Update services on application startup in case of changes in the JSON Schema
* Minor changes after manual test
* Bots sensitive passwords fields are stored in secrets store
* Ingestion pipeline sensitive passwords fields are stored in secrets store
* Fix Java style
* Add update for ingestion pipelines on start up
* Add pylint exception
* Fix broken test
* Update UI references
* Added support for 'Metadata Service' connectors
* Fixed failing unit tests
* Removed delete button for 'Metadata Service' connector of type OpenMetadataServer
* Code optimization for SelectServiceType component
* Fixed errors arrived due to backend changes related to OpenMetadata Services
* init reindex
* Added additional step for adding Metadata to ES configs while creating ingestion pipeline for Metadata Service
* Add ES reindex
* Add ES reindex
* Format
* Format
* Rename service
* Pick service name from IngestionPipeline
* fix ui conflicts
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* Removed part of the secrets manager implementation in the server side
* Removed part of the secrets manager implementation in the openmetadata python library side
* Remove deprecated test
* Address pylint checks
* Address new pylint checks
* Address PR comments
* Fix import on airflows apis
* Clear singleton instances for running Ometa secrets manager test
* Added SystemServices for creating DataInsight and ElasticSearch Reindexing Pipeling with Sink config
* Add Amundsen config
* Changed SystemService to MetadataService
* Change to MetadataConnection
* Fixed failing test
* Fixed Failing test
* Added Metadata Service Test
* Bootstrapped Openmetadat service
* checkstyle fix
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* added domo to test subpackage
* cleaned up branch by removing commit from issue-8353 that were not merged in main
* Added web analytics data refinement
* Added get_status function
* Added from __futur__ for typing
* Added dailyActiveUsers aggregation
* Added page views entities aggregation and active users aggregation
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* added domo to test subpackage
* cleaned up branch by removing commit from issue-8353 that were not merged in main
* Added web analytics data refinement
* Added get_status function
* Added from __futur__ for typing
* Fixed typos brought up during reviews
* Fix Json Schema filename
* Remove formatting
* FIx required property and change data type from array to object
* added items to array type
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
* Fixed: dagster to connect to cloud
* Fixed: changed test_cases
* Fixed: lineage between the assets and jobs
* Changes as per comments
* Fix python checkstyle
* Fix as per comments
* Fix added migration scripts
* minor fixes
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* Fix pageViewEvent import issue
* Changed dataType to PERCENTAGE and NUMBER
* Cleaned up PR by removing metadata resource and ingestionPipeline sink logic
* Update openmetadata-service/src/main/resources/json/data/dataInsight/totalEntitiesByTier.json
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Update openmetadata-service/src/main/resources/json/data/dataInsight/totalEntitiesByType.json
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Fix#6141: Ingestion Pipeline Status Updates
* List Pipeline Status API & Improvements
* Rename State field to PipelineState in UI
* Convert Pipeline Status array to single object
* fix braking UI
* Rebase Fixes
* Profiler, TestStuite & DataInsigts Pipeline
* py_format
* fix logs page not loading
add pipelineStatus endpoint
* fix recent run changes
* Fix Tests
* address review comments for ui
* fix failing checks
* fix unit tests
* fix cypress test
* remove loader test as not using it
* wait for API when we click on the tab
* fix cypress waitFor api
* fix failed cypress tests
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
