yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-10-31 10:39:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(docs): Update Helm Values; Azure Key Vault Docs for 1.4.X (#16014)

Browse Source 
* chore(docs): Update Helm Values; Azure Key Vault Docs for 1.4.X

* fix: use bare-metal config values and update example

* docs: Add note on PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER usage with Helm Values
This commit is contained in:
Akash Jain 2024-04-24 15:53:58 +05:30 committed by GitHub
parent cd73328f87
commit f90f1301d9
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 203 additions and 41 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

36
openmetadata-docs/content/v1.3.x/deployment/kubernetes/values.md
View File

@ -14,6 +14,7 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| Key | Type | Default | Environment Variable from openmetadata.yaml |
|-----|------|---------| ---------------------- |
| openmetadata.config.authentication.enabled | bool | `true` | |
| openmetadata.config.authentication.clientType | string | `public` | AUTHENTICATION_CLIENT_TYPE |
| openmetadata.config.authentication.provider | string | `basic` | AUTHENTICATION_PROVIDER |
| openmetadata.config.authentication.publicKeys | list | `[http://openmetadata:8585/api/v1/system/config/jwks]` | AUTHENTICATION_PUBLIC_KEYS |
| openmetadata.config.authentication.authority | string | `https://accounts.google.com` | AUTHENTICATION_AUTHORITY |
@ -50,6 +51,24 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.authentication.ldapConfiguration.trustStoreConfig.hostNameConfig.acceptableHostNames | string | `[Empty String]` | AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES |
| openmetadata.config.authentication.ldapConfiguration.trustStoreConfig.jvmDefaultConfig.verifyHostname | string | `Empty String` | AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST |
| openmetadata.config.authentication.ldapConfiguration.trustStoreConfig.trustAllConfig.examineValidityDates | bool | `true` | AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES |
| openmetadata.config.authentication.oidcConfiguration.callbackUrl | string | `http://openmetadata:8585/callback` | OIDC_CALLBACK |
| openmetadata.config.authentication.oidcConfiguration.clientAuthenticationMethod | string | `client_secret_post` | OIDC_CLIENT_AUTH_METHOD |
| openmetadata.config.authentication.oidcConfiguration.clientId.secretKey | string | `openmetadata-oidc-client-id` | OIDC_CLIENT_ID |
| openmetadata.config.authentication.oidcConfiguration.clientId.secretRef | string | `oidc-secrets` | OIDC_CLIENT_ID |
| openmetadata.config.authentication.oidcConfiguration.clientSecret.secretKey | string | `openmetadata-oidc-client-secret` | OIDC_CLIENT_SECRET |
| openmetadata.config.authentication.oidcConfiguration.clientSecret.secretRef | string | `oidc-secrets` | OIDC_CLIENT_SECRET |
| openmetadata.config.authentication.oidcConfiguration.customParams | string | `Empty` | OIDC_CUSTOM_PARAMS |
| openmetadata.config.authentication.oidcConfiguration.disablePkce | bool | true | OIDC_DISABLE_PKCE |
| openmetadata.config.authentication.oidcConfiguration.discoveryUri | string | `Empty` | OIDC_DISCOVERY_URI |
| openmetadata.config.authentication.oidcConfiguration.enabled | bool | false | |
| openmetadata.config.authentication.oidcConfiguration.maxClockSkew | string | `Empty` | OIDC_MAX_CLOCK_SKEW |
| openmetadata.config.authentication.oidcConfiguration.oidcType | string | `Empty` | OIDC_TYPE |
| openmetadata.config.authentication.oidcConfiguration.preferredJwsAlgorithm | string | `RS256` | OIDC_PREFERRED_JWS |
| openmetadata.config.authentication.oidcConfiguration.responseType | string | `code` | OIDC_RESPONSE_TYPE |
| openmetadata.config.authentication.oidcConfiguration.scope | string | `openid email profile` | OIDC_SCOPE |
| openmetadata.config.authentication.oidcConfiguration.serverUrl | string | `http://openmetadata:8585` | OIDC_SERVER_URL |
| openmetadata.config.authentication.oidcConfiguration.tenant | string | `Empty` | OIDC_TENANT |
| openmetadata.config.authentication.oidcConfiguration.useNonce | bool | `true` | OIDC_USE_NONCE |
| openmetadata.config.authentication.saml.debugMode | bool | false | SAML_DEBUG_MODE |
| openmetadata.config.authentication.saml.idp.entityId | string | `Empty` | SAML_IDP_ENTITY_ID |
| openmetadata.config.authentication.saml.idp.ssoLoginUrl | string | `Empty` | SAML_IDP_SSO_LOGIN_URL |
@ -88,6 +107,7 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.airflow.auth.password.secretKey | string | `openmetadata-airflow-password` | AIRFLOW_PASSWORD |
| openmetadata.config.airflow.auth.username | string | `admin` | AIRFLOW_USERNAME |
| openmetadata.config.airflow.enabled | bool | `true` | |
| openmetadata.config.airflow.host | string | `http://openmetadata-dependencies-web:8080` | PIPELINE_SERVICE_CLIENT_ENDPOINT |
| openmetadata.config.airflow.openmetadata.serverHostApiUrl | string | `http://openmetadata:8585/api` | SERVER_HOST_API_URL |
| openmetadata.config.airflow.sslCertificatePath | string | `/no/path` | PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH |
| openmetadata.config.airflow.verifySsl | string | `no-ssl` | PIPELINE_SERVICE_CLIENT_VERIFY_SSL |
@ -151,12 +171,20 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.pipelineServiceClientConfig.verifySsl | string | `no-ssl` | PIPELINE_SERVICE_CLIENT_VERIFY_SSL |
| openmetadata.config.pipelineServiceClientConfig.hostIp | string | `Empty` | PIPELINE_SERVICE_CLIENT_HOST_IP |
| openmetadata.config.secretsManager.enabled | bool | `true` | |
| openmetadata.config.secretsManager.provider | string | `db` | SECRET_MANAGER |
| openmetadata.config.secretsManager.provider | string | `Empty String` | SECRET_MANAGER |
| openmetadata.config.secretsManager.prefix | string | `Empty String` | SECRET_MANAGER_PREFIX |
| openmetadata.config.secretsManager.tags | list | `[]` | SECRET_MANAGER_TAGS |
| openmetadata.config.secretsManager.additionalParameters.enabled | bool | `false` | |
| openmetadata.config.secretsManager.additionalParameters.accessKeyId.secretRef | string | `aws-access-key-secret` | OM_SM_ACCESS_KEY_ID |
| openmetadata.config.secretsManager.additionalParameters.accessKeyId.secretKey | string | `aws-key-secret` | OM_SM_ACCESS_KEY_ID |
| openmetadata.config.secretsManager.additionalParameters.clientId.secretRef | string | `azure-client-id-secret` | OM_SM_CLIENT_ID |
| openmetadata.config.secretsManager.additionalParameters.clientId.secretKey | string | `azure-key-secret` | OM_SM_CLIENT_ID |
| openmetadata.config.secretsManager.additionalParameters.clientSecret.secretRef | string | `azure-client-secret` | OM_SM_CLIENT_SECRET |
| openmetadata.config.secretsManager.additionalParameters.clientSecret.secretKey | string | `azure-key-secret` | OM_SM_CLIENT_SECRET |
| openmetadata.config.secretsManager.additionalParameters.tenantId.secretRef | string | `azure-tenant-id-secret` | OM_SM_TENANT_ID |
| openmetadata.config.secretsManager.additionalParameters.tenantId.secretKey | string | `azure-key-secret` | OM_SM_TENANT_ID |
| openmetadata.config.secretsManager.additionalParameters.vaultName.secretRef | string | `azure-vault-name-secret` | OM_SM_VAULT_NAME |
| openmetadata.config.secretsManager.additionalParameters.vaultName.secretKey | string | `azure-key-secret` | OM_SM_VAULT_NAME |
| openmetadata.config.secretsManager.additionalParameters.region | string | `Empty String` | OM_SM_REGION |
| openmetadata.config.secretsManager.additionalParameters.secretAccessKey.secretRef | string | `aws-secret-access-key-secret` | OM_SM_ACCESS_KEY |
| openmetadata.config.secretsManager.additionalParameters.secretAccessKey.secretKey | string | `aws-key-secret` | OM_SM_ACCESS_KEY |
@ -170,8 +198,8 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.smtpConfig.supportUrl | string | `https://slack.open-metadata.org` | OM_SUPPORT_URL |
| openmetadata.config.smtpConfig.transportationStrategy | string | `SMTP_TLS` | SMTP_SERVER_STRATEGY |
| openmetadata.config.smtpConfig.username | string | `Empty String` | SMTP_SERVER_USERNAME |
| openmetadata.config.upgradeMigrationConfigs.force | bool | `false` | |
| openmetadata.config.upgradeMigrationConfigs.migrationLimitParam | int | `1200` | MIGRATION_LIMIT_PARAM |
| openmetadata.config.upgradeMigrationConfigs.debug | bool | `false` | |
| openmetadata.config.upgradeMigrationConfigs.additionalArgs | string | `Empty String` | |
| openmetadata.config.web.enabled | bool | `true` | |
| openmetadata.config.web.contentTypeOptions.enabled | bool | `false` | WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED |
| openmetadata.config.web.csp.enabled | bool | `false` | WEB_CONF_XSS_CSP_ENABLED |
@ -210,7 +238,7 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| fullnameOverride | string | `"openmetadata"` |
| image.pullPolicy | string | `"Always"` |
| image.repository | string | `"docker.getcollate.io/openmetadata/server"` |
| image.tag | string | `1.3.1` |
| image.tag | string | `1.3.3` |
| imagePullSecrets | list | `[]` |
| ingress.annotations | object | `{}` |
| ingress.className | string | `""` |

28
openmetadata-docs/content/v1.3.x/deployment/secrets-manager/supported-implementations/azure-key-vault/index.md
View File

@ -64,9 +64,19 @@ secretsManagerConfiguration:
parameters:
enabled: true
vaultName: ${OM_SM_VAULT_NAME:-""}
pipelineServiceClientConfiguration:
secretsManagerLoader: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-airflow}
```
For Helm Values, you will need to add `PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER` as part of `extraEnvs`. This will look like below -
```yaml
---
...
extraEnvs:
- name: PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER
value: airflow
- name: PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER
value: airflow
...
```
#### Client Secret Credential
@ -83,9 +93,19 @@ secretsManagerConfiguration:
clientSecret: ${OM_SM_CLIENT_SECRET:-""}
tenantId: ${OM_SM_TENANT_ID:-""}
vaultName: ${OM_SM_VAULT_NAME:-""}
pipelineServiceClientConfiguration:
secretsManagerLoader: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-airflow}
```
For Helm Values, you will need to add `PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER` as part of `extraEnvs`. This will look like below -
```yaml
---
...
extraEnvs:
- name: PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER
value: airflow
- name: PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER
value: airflow
...
```
The changes to be done in `airflow.yaml` file of the Airflow are:

35
openmetadata-docs/content/v1.4.x-SNAPSHOT/deployment/kubernetes/values.md
View File

@ -14,6 +14,7 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| Key | Type | Default | Environment Variable from openmetadata.yaml |
|-----|------|---------| ---------------------- |
| openmetadata.config.authentication.enabled | bool | `true` | |
| openmetadata.config.authentication.clientType | string | `public` | AUTHENTICATION_CLIENT_TYPE |
| openmetadata.config.authentication.provider | string | `basic` | AUTHENTICATION_PROVIDER |
| openmetadata.config.authentication.publicKeys | list | `[http://openmetadata:8585/api/v1/system/config/jwks]` | AUTHENTICATION_PUBLIC_KEYS |
| openmetadata.config.authentication.authority | string | `https://accounts.google.com` | AUTHENTICATION_AUTHORITY |
@ -50,6 +51,24 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.authentication.ldapConfiguration.trustStoreConfig.hostNameConfig.acceptableHostNames | string | `[Empty String]` | AUTHENTICATION_LDAP_ALLOWED_HOSTNAMES |
| openmetadata.config.authentication.ldapConfiguration.trustStoreConfig.jvmDefaultConfig.verifyHostname | string | `Empty String` | AUTHENTICATION_LDAP_SSL_VERIFY_CERT_HOST |
| openmetadata.config.authentication.ldapConfiguration.trustStoreConfig.trustAllConfig.examineValidityDates | bool | `true` | AUTHENTICATION_LDAP_EXAMINE_VALIDITY_DATES |
| openmetadata.config.authentication.oidcConfiguration.callbackUrl | string | `http://openmetadata:8585/callback` | OIDC_CALLBACK |
| openmetadata.config.authentication.oidcConfiguration.clientAuthenticationMethod | string | `client_secret_post` | OIDC_CLIENT_AUTH_METHOD |
| openmetadata.config.authentication.oidcConfiguration.clientId.secretKey | string | `openmetadata-oidc-client-id` | OIDC_CLIENT_ID |
| openmetadata.config.authentication.oidcConfiguration.clientId.secretRef | string | `oidc-secrets` | OIDC_CLIENT_ID |
| openmetadata.config.authentication.oidcConfiguration.clientSecret.secretKey | string | `openmetadata-oidc-client-secret` | OIDC_CLIENT_SECRET |
| openmetadata.config.authentication.oidcConfiguration.clientSecret.secretRef | string | `oidc-secrets` | OIDC_CLIENT_SECRET |
| openmetadata.config.authentication.oidcConfiguration.customParams | string | `Empty` | OIDC_CUSTOM_PARAMS |
| openmetadata.config.authentication.oidcConfiguration.disablePkce | bool | true | OIDC_DISABLE_PKCE |
| openmetadata.config.authentication.oidcConfiguration.discoveryUri | string | `Empty` | OIDC_DISCOVERY_URI |
| openmetadata.config.authentication.oidcConfiguration.enabled | bool | false | |
| openmetadata.config.authentication.oidcConfiguration.maxClockSkew | string | `Empty` | OIDC_MAX_CLOCK_SKEW |
| openmetadata.config.authentication.oidcConfiguration.oidcType | string | `Empty` | OIDC_TYPE |
| openmetadata.config.authentication.oidcConfiguration.preferredJwsAlgorithm | string | `RS256` | OIDC_PREFERRED_JWS |
| openmetadata.config.authentication.oidcConfiguration.responseType | string | `code` | OIDC_RESPONSE_TYPE |
| openmetadata.config.authentication.oidcConfiguration.scope | string | `openid email profile` | OIDC_SCOPE |
| openmetadata.config.authentication.oidcConfiguration.serverUrl | string | `http://openmetadata:8585` | OIDC_SERVER_URL |
| openmetadata.config.authentication.oidcConfiguration.tenant | string | `Empty` | OIDC_TENANT |
| openmetadata.config.authentication.oidcConfiguration.useNonce | bool | `true` | OIDC_USE_NONCE |
| openmetadata.config.authentication.saml.debugMode | bool | false | SAML_DEBUG_MODE |
| openmetadata.config.authentication.saml.idp.entityId | string | `Empty` | SAML_IDP_ENTITY_ID |
| openmetadata.config.authentication.saml.idp.ssoLoginUrl | string | `Empty` | SAML_IDP_SSO_LOGIN_URL |
@ -152,12 +171,20 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.pipelineServiceClientConfig.verifySsl | string | `no-ssl` | PIPELINE_SERVICE_CLIENT_VERIFY_SSL |
| openmetadata.config.pipelineServiceClientConfig.hostIp | string | `Empty` | PIPELINE_SERVICE_CLIENT_HOST_IP |
| openmetadata.config.secretsManager.enabled | bool | `true` | |
| openmetadata.config.secretsManager.provider | string | `db` | SECRET_MANAGER |
| openmetadata.config.secretsManager.provider | string | `Empty String` | SECRET_MANAGER |
| openmetadata.config.secretsManager.prefix | string | `Empty String` | SECRET_MANAGER_PREFIX |
| openmetadata.config.secretsManager.tags | list | `[]` | SECRET_MANAGER_TAGS |
| openmetadata.config.secretsManager.additionalParameters.enabled | bool | `false` | |
| openmetadata.config.secretsManager.additionalParameters.accessKeyId.secretRef | string | `aws-access-key-secret` | OM_SM_ACCESS_KEY_ID |
| openmetadata.config.secretsManager.additionalParameters.accessKeyId.secretKey | string | `aws-key-secret` | OM_SM_ACCESS_KEY_ID |
| openmetadata.config.secretsManager.additionalParameters.clientId.secretRef | string | `azure-client-id-secret` | OM_SM_CLIENT_ID |
| openmetadata.config.secretsManager.additionalParameters.clientId.secretKey | string | `azure-key-secret` | OM_SM_CLIENT_ID |
| openmetadata.config.secretsManager.additionalParameters.clientSecret.secretRef | string | `azure-client-secret` | OM_SM_CLIENT_SECRET |
| openmetadata.config.secretsManager.additionalParameters.clientSecret.secretKey | string | `azure-key-secret` | OM_SM_CLIENT_SECRET |
| openmetadata.config.secretsManager.additionalParameters.tenantId.secretRef | string | `azure-tenant-id-secret` | OM_SM_TENANT_ID |
| openmetadata.config.secretsManager.additionalParameters.tenantId.secretKey | string | `azure-key-secret` | OM_SM_TENANT_ID |
| openmetadata.config.secretsManager.additionalParameters.vaultName.secretRef | string | `azure-vault-name-secret` | OM_SM_VAULT_NAME |
| openmetadata.config.secretsManager.additionalParameters.vaultName.secretKey | string | `azure-key-secret` | OM_SM_VAULT_NAME |
| openmetadata.config.secretsManager.additionalParameters.region | string | `Empty String` | OM_SM_REGION |
| openmetadata.config.secretsManager.additionalParameters.secretAccessKey.secretRef | string | `aws-secret-access-key-secret` | OM_SM_ACCESS_KEY |
| openmetadata.config.secretsManager.additionalParameters.secretAccessKey.secretKey | string | `aws-key-secret` | OM_SM_ACCESS_KEY |
@ -171,8 +198,8 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.smtpConfig.supportUrl | string | `https://slack.open-metadata.org` | OM_SUPPORT_URL |
| openmetadata.config.smtpConfig.transportationStrategy | string | `SMTP_TLS` | SMTP_SERVER_STRATEGY |
| openmetadata.config.smtpConfig.username | string | `Empty String` | SMTP_SERVER_USERNAME |
| openmetadata.config.upgradeMigrationConfigs.force | bool | `false` | |
| openmetadata.config.upgradeMigrationConfigs.migrationLimitParam | int | `1200` | MIGRATION_LIMIT_PARAM |
| openmetadata.config.upgradeMigrationConfigs.debug | bool | `false` | |
| openmetadata.config.upgradeMigrationConfigs.additionalArgs | string | `Empty String` | |
| openmetadata.config.web.enabled | bool | `true` | |
| openmetadata.config.web.contentTypeOptions.enabled | bool | `false` | WEB_CONF_CONTENT_TYPE_OPTIONS_ENABLED |
| openmetadata.config.web.csp.enabled | bool | `false` | WEB_CONF_XSS_CSP_ENABLED |
@ -211,7 +238,7 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| fullnameOverride | string | `"openmetadata"` |
| image.pullPolicy | string | `"Always"` |
| image.repository | string | `"docker.getcollate.io/openmetadata/server"` |
| image.tag | string | `1.3.1` |
| image.tag | string | `1.3.3` |
| imagePullSecrets | list | `[]` |
| ingress.annotations | object | `{}` |
| ingress.className | string | `""` |

133
openmetadata-docs/content/v1.4.x-SNAPSHOT/deployment/secrets-manager/supported-implementations/azure-key-vault/index.md
View File

@ -10,42 +10,106 @@ for the non-managed follow only the steps related to the Airflow server and CLI.
## Setup
### 1. Permissions needed
### 1. Create Principal
#### Service Principal
1. Go to `Microsoft Entra ID` and create an [App Registration](https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app).
2. Inside the App Registration, go to `Certificates & Secrets` and create a `Client secret`. Note down the `Value`, it will be our `clientSecret` configuration.
3. From the App Registration overview page, note down the `Application (client) ID` and the `Directory (tenant) ID`.
4. In your Key Vault overview page, note down the `Vault URI`.
5. Go to `Access Control (IAM)` and click on `Add Role Assignment`.
6. Give the permission `Key Vault Secrets Officer` to your App Registration.
### 2. Update configuration
#### Managed Identity (recommnded)
We have to set up the secret manager provider we want to use, that in our case is `azure-kv`, and the credentials for our
App Registration.
1. In your Azure subscription create [Manged Identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview)
2. Use this created identity - for AKS users this means you need to use [Pod Identity](https://learn.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity) or [Workload Identity (recommnded)](https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet).
{% note %}
Note that the using Managed Identity require using [default Authentication Credential](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme?view=azure-python#defaultazurecredential).
{% /note %}
### 2. Add RBAC roles
It if possible to use different Principals for OpenMetadata Server and the Ingestion. In that case the server needs higher privileges - `Key Vault Secrets Officer` - to be able to create/read/update secrets in the Vault.
While the Airflow part only needs to read the secrets hence the role `Key Vault Secrets Officer`.
#### Open Metadata server
1. In your Key Vault overview page, note down the `Vault URI`.
2. Go to `Access Control (IAM)` and click on `Add Role Assignment`.
3. Give the permission `Key Vault Secrets Officer` to your Principal.
#### Airflow
1. In your Key Vault overview page, note down the `Vault URI`.
2. Go to `Access Control (IAM)` and click on `Add Role Assignment`.
3. Give the permission `Key Vault Secrets Users` to your Principal.
### 3. Update configuration
We have to set up the secret manager provider we want to use, that in our case is `azure-kv`, and the credentials.
The changes to be done in `openmetadata.yaml` file of the OpenMetadata server are:
```yaml
...
secretsManagerConfiguration:
secretsManager: managed-azure-kv # or env var SECRET_MANAGER. For non-managed use 'azure-kv'.
prefix: ${SECRET_MANAGER_PREFIX:-""} # Define the secret key ID as <prefix>-<clusterName>-<key>
tags: ${SECRET_MANAGER_TAGS:-[]} # Add tags to the created resource. Format is `[key1:value1,key2:value2,...]`
parameters:
clientId: ${OM_SM_CLIENT_ID:-""}
clientSecret: ${OM_SM_CLIENT_SECRET:-""}
tenantId: ${OM_SM_TENANT_ID:-""}
vaultName: ${OM_SM_VAULT_NAME:-""}
#### Default Azure Credential
```yaml
---
secretsManagerConfiguration:
secretsManager: managed-azure-kv # or env var SECRET_MANAGER. For non-managed use 'azure-kv'.
prefix: ${SECRET_MANAGER_PREFIX:-""} # Define the secret key ID as <prefix>-<clusterName>-<key>
tags: ${SECRET_MANAGER_TAGS:-[]} # Add tags to the created resource. Format is `[key1:value1,key2:value2,...]`
parameters:
enabled: true
vaultName: ${OM_SM_VAULT_NAME:-""}
pipelineServiceClientConfiguration:
# ...
# Secrets Manager Loader: specify to the Ingestion Framework how to load the SM credentials from its env
# Supported: noop, airflow, env
secretsManagerLoader: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-"noop"}
secretsManagerLoader: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-airflow}
```
For Helm Values, you will need to add `PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER` as part of `extraEnvs`. This will look like below -
```yaml
---
...
extraEnvs:
- name: PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER
value: airflow
...
```
#### Client Secret Credential
```yaml
---
secretsManagerConfiguration:
secretsManager: managed-azure-kv # or env var SECRET_MANAGER. For non-managed use 'azure-kv'.
prefix: ${SECRET_MANAGER_PREFIX:-""} # Define the secret key ID as <prefix>-<clusterName>-<key>
tags: ${SECRET_MANAGER_TAGS:-[]} # Add tags to the created resource. Format is `[key1:value1,key2:value2,...]`
parameters:
enabled: true
clientId: ${OM_SM_CLIENT_ID:-""}
clientSecret: ${OM_SM_CLIENT_SECRET:-""}
tenantId: ${OM_SM_TENANT_ID:-""}
vaultName: ${OM_SM_VAULT_NAME:-""}
pipelineServiceClientConfiguration:
secretsManagerLoader: ${PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER:-airflow}
```
For Helm Values, you will need to add `PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER` as part of `extraEnvs`. This will look like below -
```yaml
---
...
extraEnvs:
- name: PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER
value: airflow
...
```
The changes to be done in `airflow.yaml` file of the Airflow are:
{% note %}
Note that the **Key Vault Name** parameter is MANDATORY for the system to know where to store and retrieve the secrets.
@ -75,6 +139,30 @@ AIRFLOW__OPENMETADATA_SECRETS_MANAGER__AZURE_CLIENT_SECRET= <App Registration Se
If only the `<Key Vault Name>`, parameter is provided, we will use Azure's [default Authentication Credential](https://learn.microsoft.com/en-us/python/api/overview/azure/identity-readme?view=azure-python#defaultazurecredential).
{% note %}
Also if you are using [Microsoft Entra Workload ID](https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview) with [Service Account Token Volume Projection](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#serviceaccount-token-volume-projection) then you need also to use projected service account instead one created by Airflow and OpenMetadata:
airflow.yaml:
```yaml
---
serviceAccount:
create: false
name: "name-of-your-service-account"
```
openmetadata.yaml:
```yaml
---
serviceAccount:
create: false
name: "name-of-your-service-account"
```
{% /note %}
### 3. Restart both servers
After updating the configuration files, we are ready to restart both services. When the OM server starts, it will
@ -118,7 +206,6 @@ Key Vault service to point to.
You can specify as well the environment variables of your App Registration if you're running the ingestion
outside of Azure: [docs](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.environmentcredential?view=azure-python).
## Airflow
If you enabled the Secret Manager and you are using your own Airflow to run the ingestions, make sure to configure