* wip
* feat: trigger external apps with override config
- Added in openmetadata-airflow-apis functionality to trigger DAG with feature.
- Modified openmetadata-airflow-apis application runner to accept override config from params.
- Added overloaded runPipeline with `Map<String,Object> config` to allow triggering apps with configuration. We might want to expand this to all ingestion pipelines. For now its just for apps.
- Implemented an example external app that can be used to test functionality of external apps. The app can be enabled by setting the `ENABLE_APP_HelloPipelines=true` environment variable.
* fix class doc for application
* fixed README for airflow apis
* fixes
* set HelloPipelines to disabeld by default
* fixed basedpywright errros
* fixed app schema
* reduced airflow client runPipeline to an overload with null config
removed duplicate call to runPipeline in AppResource
* Update openmetadata-docs/content/v1.7.x-SNAPSHOT/developers/applications/index.md
Co-authored-by: Matias Puerta <matias@getcollate.io>
* deleted documentation file
---------
Co-authored-by: Matias Puerta <matias@getcollate.io>
* GEN-1493 - Fix opensearch pagination
* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
* GEN-1494 - Add CI for py-tests with Postgres and Opensearch
* Fix backend tests and have index alias with clusterAlias appended
* Fix backend tests and have index alias with clusterAlias appended
* Fix failing tests
* alias setup
* fix suggestion not working due to alias
* fix getIndexOrAliasName method for multiple indexes
* update openmetadata.yaml
* update childAliases with clusterAlias
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
* Split ExternalSecretsManagerTest to new ExternalSecretsManagerTest and AWSBasedSecretsManagerTest
* implement SecretsManagerFactory to create GCPSecretsManager
* implement GCPSecretsManager
* implements gcp secret manager.
* Fix it for the GCP's rule.
* create a template of GCP
* fix compile error
* implements to use project_id
* add library for the google cloud secret manager
* add test code for using google credential in the docker container
* modify docker-compose.yml for GCP
* add google_crc32c module
* modify ways to get project id
* create a new docker-compose.yml for Google Cloud
* create a new document
* create compose file for gcp secret manager
* fix invalid styles and formats
* downgrade google library to avoid conflicting protoc versions
* Revert "fix(CI): update `run_local_docker.sh` script with cluster alias ES endpoint (#16604)"
This reverts commit baab52076f71f76e8bc48d89564ba733ac77a1e6.
* Revert "chore(CI): Enable Elasticsearch cluster alias by default (#16599)"
This reverts commit a3524e14d3a1244db19199ab1eae3796ba15b499.
* Fix supported characters in SM
* Update SM
* Fixes
* Fixes
* Improve class conversion exceptions
* Comments
* Rename noop to db secrets manager providee
* Update sm
* Fix
* db SM
* db SM
* Fix test
* UI
* Update openmetadata-ui/src/main/resources/ui/src/mocks/IngestionListTable.mock.ts
* update default
* - Removed Logi Configuration from the openmetadata.yaml
- Migrated Login Configuration to the UI
* Removed ApplicationConfig from tests
* Added Custom Header Logo Url
* Update field to favicon
* support login conifguration
* chore(ui): update logo and login config md files
* fix validation issue
* add cypress
* restructure folders and set default fallback for favicon
* autofocus to first field of edit form
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* working on maven shading
* update es in docker
* add maven shade for elasticsearch/opensearch dependencies
* add submodule for es/os dependencies
* upgrade docker elasticsearch to latest 8.10.2
* upgrade docker elasticsearch to latest 8.10.2
* Update Postgres dev to use OpenSearch
* Fix container service dependency
* revert conf change
---------
Co-authored-by: 07Himank <himank07mehta@gmail.com>
Co-authored-by: 07Himank <112613760+07Himank@users.noreply.github.com>
* Updating Dockerfile with multistage
* Updating reviewed changes
* Docker Development changes
* Docker workflow changes
* Arguments update
* Script path update
* Arguments update
* Resolving the reviewed suggestions
* Preparing for next release and updating docker sort value changes
* Adding .dev0 in the python related release version
* Adding .dev0 in the python related release version
* Added diffrent clients for elasticSearch and openSearch and did the interface implementation
* removed some comments and some changes
* Add Elastic Search Test Container
* upgarded es version
* added test cases for elastic search
* added test case for, when tag is deleted cehck its relationships also get deleted
* changed package of open,elastic to openSearch and elasticSearch
* added support for skip search test case if index is not present of that entity
* Refactor bunch of files
* unnecessary semicolon
* refactor test case indexing
* fix checkstyle
* checkstyle
* added hardcoded configs and changed package name
* fixed no response from hits
* stopped elasticsearch container and some fixes
* closed elasticsearch container
* revert
* closed client after used in test
* style
* edit es config in junit
* checking if using already es version release memory for maven ci
* revert docker file changes
* remove es container for testing the ci
* commented out all test cases for elasticsearch
* fix
* creating elastic search with reusable enabled
* trying to fix ci error
* start container only if it is not running
* revert docker changes and done changes for elastic search container
* changed var name of sort_order for es
* lower the version of es client
* fixes
* all latest version of es including docker
* Use Single sort order
* Fix Keyword issue
* Immediate refresh policy for entity created
* increase sleep time
* disable test
* added reindexing logic in run_local_docker.sh
* enabled test case
---------
Co-authored-by: Himank Mehta <himankmehta@Himanks-MacBook-Air.local>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
* Add Pipeline Service Status Job to publish to Prometheus
* Add Pipeline Service Status Job to publish to Prometheus
* Add config for health check interval
