fix(docs): kubernetes docs for breaking changes for 1.1.0 (#12215)

* update global.* to openmetadata.config.*

* update values

* fix typo

* Add more docs

* elastic search application version related docs update
This commit is contained in:
Akash Jain 2023-06-29 08:50:19 +05:30 committed by GitHub
parent 3e879136c0
commit 72bb254342
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
17 changed files with 385 additions and 95 deletions

View File

@ -160,16 +160,17 @@ If using Kubernetes, make sure that you are passing the correct values to Helm C
```yaml ```yaml
# Custom OpenMetadata Values.yaml # Custom OpenMetadata Values.yaml
global: openmetadata:
airflow: config:
enabled: true pipelineServiceClientConfig:
# endpoint url for airflow enabled: true
host: http://openmetadata-dependencies-web.default.svc.cluster.local:8080 # endpoint url for airflow
auth: apiEndpoint: http://openmetadata-dependencies-web.default.svc.cluster.local:8080
username: admin auth:
password: username: admin
secretRef: airflow-secrets password:
secretKey: openmetadata-airflow-password secretRef: airflow-secrets
secretKey: openmetadata-airflow-password
``` ```
#### Validating the installation #### Validating the installation

View File

@ -239,34 +239,35 @@ Once you have the RDS and OpenSearch Services Setup, you can update the environm
```yaml ```yaml
# openmetadata-values.prod.yaml # openmetadata-values.prod.yaml
... ...
global: openmetadata:
elasticsearch: config:
host: <AMAZON_OPENSEARCH_SERVICE_ENDPOINT_WITHOUT_HTTPS> elasticsearch:
port: 443 host: <AMAZON_OPENSEARCH_SERVICE_ENDPOINT_WITHOUT_HTTPS>
scheme: https port: 443
connectionTimeoutSecs: 5 scheme: https
socketTimeoutSecs: 60 connectionTimeoutSecs: 5
keepAliveTimeoutSecs: 600 socketTimeoutSecs: 60
batchSize: 10 keepAliveTimeoutSecs: 600
auth: batchSize: 10
enabled: false auth:
username: <AMAZON_OPENSEARCH_USERNAME> enabled: false
password: username: <AMAZON_OPENSEARCH_USERNAME>
secretRef: elasticsearch-secrets password:
secretKey: openmetadata-elasticsearch-password secretRef: elasticsearch-secrets
database: secretKey: openmetadata-elasticsearch-password
host: <AMAZON_RDS_ENDPOINT> database:
port: 3306 host: <AMAZON_RDS_ENDPOINT>
driverClass: com.mysql.cj.jdbc.Driver port: 3306
dbScheme: mysql driverClass: com.mysql.cj.jdbc.Driver
dbUseSSL: true dbScheme: mysql
databaseName: <RDS_DATABASE_NAME> dbUseSSL: true
auth: databaseName: <RDS_DATABASE_NAME>
username: <RDS_DATABASE_USERNAME> auth:
password: username: <RDS_DATABASE_USERNAME>
secretRef: mysql-secrets password:
secretKey: openmetadata-mysql-password secretRef: mysql-secrets
... secretKey: openmetadata-mysql-password
...
``` ```
Make sure to create RDS and OpenSearch credentials as Kubernetes Secrets mentioned [here](https://docs.open-metadata.org/deployment/kubernetes#quickstart). Make sure to create RDS and OpenSearch credentials as Kubernetes Secrets mentioned [here](https://docs.open-metadata.org/deployment/kubernetes#quickstart).

View File

@ -98,7 +98,9 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.elasticsearch.auth.password.secretRef | string | `elasticsearch-secrets` | ELASTICSEARCH_PASSWORD | | openmetadata.config.elasticsearch.auth.password.secretRef | string | `elasticsearch-secrets` | ELASTICSEARCH_PASSWORD |
| openmetadata.config.elasticsearch.auth.password.secretKey | string | `openmetadata-elasticsearch-password` | ELASTICSEARCH_PASSWORD | | openmetadata.config.elasticsearch.auth.password.secretKey | string | `openmetadata-elasticsearch-password` | ELASTICSEARCH_PASSWORD |
| openmetadata.config.elasticsearch.host | string | `elasticsearch` | ELASTICSEARCH_HOST | | openmetadata.config.elasticsearch.host | string | `elasticsearch` | ELASTICSEARCH_HOST |
| openmetadata.config.elasticsearch.keepAliveTimeoutSecs | int | `600` | ELASTICSEARCH_KEEP_ALIVE_TIMEOUT_SECS |
| openmetadata.config.elasticsearch.port | int | 9200 | ELASTICSEARCH_PORT | | openmetadata.config.elasticsearch.port | int | 9200 | ELASTICSEARCH_PORT |
| openmetadata.config.elasticsearch.searchType | string | `ElasticSearch` | SEARCH_TYPE |
| openmetadata.config.elasticsearch.scheme | string | `http` | ELASTICSEARCH_SCHEME | | openmetadata.config.elasticsearch.scheme | string | `http` | ELASTICSEARCH_SCHEME |
| openmetadata.config.elasticsearch.searchIndexMappingLanguage | string | `EN`| ELASTICSEARCH_INDEX_MAPPING_LANG | | openmetadata.config.elasticsearch.searchIndexMappingLanguage | string | `EN`| ELASTICSEARCH_INDEX_MAPPING_LANG |
| openmetadata.config.elasticsearch.trustStore.enabled | bool | `false` | | | openmetadata.config.elasticsearch.trustStore.enabled | bool | `false` | |
@ -128,6 +130,7 @@ This page list all the supported helm values for OpenMetadata Helm Charts.
| openmetadata.config.pipelineServiceClientConfig.apiEndpoint | string | `http://openmetadata-dependencies-web.default.svc.cluster.local:8080` | PIPELINE_SERVICE_CLIENT_ENDPOINT | | openmetadata.config.pipelineServiceClientConfig.apiEndpoint | string | `http://openmetadata-dependencies-web.default.svc.cluster.local:8080` | PIPELINE_SERVICE_CLIENT_ENDPOINT |
| openmetadata.config.pipelineServiceClientConfig.className | string | `org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient` | PIPELINE_SERVICE_CLIENT_CLASS_NAME | | openmetadata.config.pipelineServiceClientConfig.className | string | `org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient` | PIPELINE_SERVICE_CLIENT_CLASS_NAME |
| openmetadata.config.pipelineServiceClientConfig.enabled | bool | `true` | | | openmetadata.config.pipelineServiceClientConfig.enabled | bool | `true` | |
| openmetadata.config.pipelineServiceClientConfig.healthCheckInterval | int | `300` | PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL |
| openmetadata.config.pipelineServiceClientConfig.ingestionIpInfoEnabled | bool | `false` | PIPELINE_SERVICE_IP_INFO_ENABLED | | openmetadata.config.pipelineServiceClientConfig.ingestionIpInfoEnabled | bool | `false` | PIPELINE_SERVICE_IP_INFO_ENABLED |
| openmetadata.config.pipelineServiceClientConfig.metadataApiEndpoint | string | `http://openmetadata.default.svc.cluster.local:8585/api` | SERVER_HOST_API_URL | | openmetadata.config.pipelineServiceClientConfig.metadataApiEndpoint | string | `http://openmetadata.default.svc.cluster.local:8585/api` | SERVER_HOST_API_URL |
| openmetadata.config.pipelineServiceClientConfig.sslCertificatePath | string | `/no/path` | PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH | | openmetadata.config.pipelineServiceClientConfig.sslCertificatePath | string | `/no/path` | PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH |

View File

@ -11,21 +11,22 @@ Once the `Client Id` and `Client Secret` are generated, see the snippet below fo
place the client id value and update the authorizer configurations in the `values.yaml`. place the client id value and update the authorizer configurations in the `values.yaml`.
```yaml ```yaml
global: openmetadata:
authorizer: config:
className: "org.openmetadata.service.security.DefaultAuthorizer" authorizer:
containerRequestFilter: "org.openmetadata.service.security.JwtFilter" className: "org.openmetadata.service.security.DefaultAuthorizer"
initialAdmins: containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
- "admin" initialAdmins:
principalDomain: "open-metadata.org" - "admin"
authentication: principalDomain: "open-metadata.org"
provider: "aws-cognito" authentication:
publicKeys: provider: "aws-cognito"
- "http://openmetadata:8585/api/v1/config/jwks" publicKeys:
- "{Cognito Domain}/{User Pool ID}/.well-known/jwks.json" # Update with your Cognito Domain and User Pool ID - "http://openmetadata:8585/api/v1/config/jwks"
authority: "{Cognito Domain}/{User Pool ID}" # Update with your Cognito Domain and User Pool ID as follows - https://cognito-idp.us-west-1.amazonaws.com/us-west-1_DL8xfTzj8 - "{Cognito Domain}/{User Pool ID}/.well-known/jwks.json" # Update with your Cognito Domain and User Pool ID
clientId: "{Client ID}" # Update with your Client ID authority: "{Cognito Domain}/{User Pool ID}" # Update with your Cognito Domain and User Pool ID as follows - https://cognito-idp.us-west-1.amazonaws.com/us-west-1_DL8xfTzj8
callbackUrl: "http://localhost:8585/callback" clientId: "{Client ID}" # Update with your Client ID
callbackUrl: "http://localhost:8585/callback"
``` ```
Finally, update the Airflow information with the JWT token Finally, update the Airflow information with the JWT token

View File

@ -80,5 +80,26 @@ global:
callbackUrl: "http://localhost:8585/callback" callbackUrl: "http://localhost:8585/callback"
``` ```
### After 1.1.0
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "suresh"
principalDomain: "open-metadata.org"
authentication:
provider: "auth0"
publicKeys:
- "http://openmetadata:8585/api/v1/config/jwks"
- "{Auth0 Domain Name}/.well-known/jwks.json"
authority: "https://parth-panchal.us.auth0.com/"
clientId: "{Client ID}"
callbackUrl: "http://localhost:8585/callback"
```
**Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for **Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for
ingesting data from Airflow. ingesting data from Airflow.

View File

@ -71,6 +71,28 @@ global:
callbackUrl: "http://localhost:8585/callback" callbackUrl: "http://localhost:8585/callback"
``` ```
### After 1.1.0
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "user1"
- "user2"
principalDomain: "open-metadata.org"
authentication:
provider: "azure"
publicKeys:
- "http://openmetadata:8585/api/v1/config/jwks"
- "https://login.microsoftonline.com/common/discovery/keys"
authority: "https://login.microsoftonline.com/{Tenant ID}"
clientId: "{Client ID}" # Azure Application
callbackUrl: "http://localhost:8585/callback"
```
{% note %} {% note %}
Follow [this](/deployment/security/azure#step-10-update-ingestion-bot-with-azure-sso-service-application) guide to configure the `ingestion-bot` credentials for ingesting data from Airflow. Follow [this](/deployment/security/azure#step-10-update-ingestion-bot-with-azure-sso-service-application) guide to configure the `ingestion-bot` credentials for ingesting data from Airflow.

View File

@ -203,14 +203,15 @@ Update your helm values to mount Kubernetes Secrets as Volumes and update the Jw
```yaml ```yaml
# openmetadata.prod.values.yml # openmetadata.prod.values.yml
global: openmetadata:
... config:
jwtTokenConfiguration: ...
rsapublicKeyFilePath: "/etc/openmetadata/jwtkeys/public_key.der" jwtTokenConfiguration:
rsaprivateKeyFilePath: "/etc/openmetadata/jwtkeys/private_key.der" rsapublicKeyFilePath: "/etc/openmetadata/jwtkeys/public_key.der"
jwtissuer: "open-metadata.org" # update this as per your environment rsaprivateKeyFilePath: "/etc/openmetadata/jwtkeys/private_key.der"
keyId: "c8ec220c-be7d-4e47-97c7-098bf6a57ce1" # update this to a unique uuid4 jwtissuer: "open-metadata.org" # update this as per your environment
... keyId: "c8ec220c-be7d-4e47-97c7-098bf6a57ce1" # update this to a unique uuid4
...
extraVolumes: extraVolumes:
- name: openmetadata-jwt-vol - name: openmetadata-jwt-vol
secret: secret:

View File

@ -59,8 +59,9 @@ Update your helm `maskPasswordsApi` value:
```yaml ```yaml
# openmetadata.prod.values.yml # openmetadata.prod.values.yml
global: openmetadata:
... config:
maskPasswordsApi: true ...
... maskPasswordsApi: true
...
``` ```

View File

@ -78,10 +78,11 @@ airflowConfiguration:
We have to update in the `values.yaml` file with: We have to update in the `values.yaml` file with:
```yaml ```yaml
global: openmetadata:
airflow: config:
verifySsl: "validate" pipelineServiceClientConfig:
sslCertificatePath: "/path/to/certificate/in/airflow" verifySsl: "validate"
sslCertificatePath: "/path/to/certificate/in/airflow"
``` ```
#### 2. Ignore the certification validation (not recommended for production): #### 2. Ignore the certification validation (not recommended for production):
@ -108,9 +109,10 @@ airflowConfiguration:
We have to update in the `values.yaml` file with: We have to update in the `values.yaml` file with:
```yaml ```yaml
global: openmetadata:
airflow: config:
verifySsl: "ignore" pipelineServiceClientConfig:
verifySsl: "ignore"
``` ```
Once one of the configurations is set, we can restart our OM server and deploy or redeploy without any issues. Once one of the configurations is set, we can restart our OM server and deploy or redeploy without any issues.

View File

@ -60,5 +60,28 @@ global:
callbackUrl: "http://localhost:8585/callback" callbackUrl: "http://localhost:8585/callback"
``` ```
### After 1.1.0
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "user1"
- "user2"
botPrincipals:
- "<service_application_client_id>"
principalDomain: "open-metadata.org"
authentication:
provider: "google"
publicKeys:
- "https://www.googleapis.com/oauth2/v3/certs"
authority: "https://accounts.google.com"
clientId: "{client id}"
callbackUrl: "http://localhost:8585/callback"
```
**Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for **Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for
ingesting data from Airflow. ingesting data from Airflow.

View File

@ -65,5 +65,26 @@ global:
callbackUrl: "http://localhost:8585/callback" callbackUrl: "http://localhost:8585/callback"
``` ```
### After 1.1.0
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "admin-user"
principalDomain: "open-metadata.org"
authentication:
provider: "custom-oidc"
publicKeys:
- "http://openmetadata:8585/api/v1/config/jwks"
- "http://localhost:8081/auth/realms/data-sec/protocol/openid-connect/certs"
authority: "http://localhost:8081/auth/realms/data-sec"
clientId: "{Client ID}"
callbackUrl: "http://localhost:8585/callback"
```
**Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for **Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for
ingesting data from Airflow. ingesting data from Airflow.

View File

@ -87,5 +87,27 @@ global:
callbackUrl: "http://localhost:8585/callback" callbackUrl: "http://localhost:8585/callback"
``` ```
### After 1.1.0
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "user1"
- "user2"
principalDomain: "open-metadata.org"
authentication:
provider: "okta"
publicKeys:
- "http://openmetadata:8585/api/v1/config/jwks"
- "{ISSUER_URL}/v1/keys"
authority: "{ISSUER_URL}"
clientId: "{CLIENT_ID - SPA APP}"
callbackUrl: "http://localhost:8585/callback"
```
**Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for **Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for
ingesting data from Airflow. ingesting data from Airflow.

View File

@ -59,5 +59,49 @@ global:
callbackUrl: "http://localhost:8585/callback" callbackUrl: "http://localhost:8585/callback"
``` ```
### After 1.1.0
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
# JWT Filter
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "suresh"
principalDomain: "open-metadata.org"
authentication:
provider: "custom-oidc"
publicKeys:
- "http://openmetadata:8585/api/v1/config/jwks"
- "{IssuerUrl}/certs"
authority: "{IssuerUrl}"
clientId: "{client id}"
callbackUrl: "http://localhost:8585/callback"
```
### After 1.1.0
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
# JWT Filter
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "suresh"
principalDomain: "open-metadata.org"
authentication:
provider: "custom-oidc"
publicKeys:
- "http://openmetadata:8585/api/v1/config/jwks"
- "{IssuerUrl}/certs"
authority: "{IssuerUrl}"
clientId: "{client id}"
callbackUrl: "http://localhost:8585/callback"
```
**Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for **Note:** Follow [this](/how-to-guides/feature-configurations/bots) guide to configure the `ingestion-bot` credentials for
ingesting data from Airflow. ingesting data from Airflow.

View File

@ -76,6 +76,66 @@ If you are upgrading production this is the recommended version to upgrade to.
## Breaking Changes for 1.1 Stable Release ## Breaking Changes for 1.1 Stable Release
### OpenMetadata Helm Chart Values
With `1.1.0` we are moving away from `global.*` helm values under openmetadata helm charts to `openmetadata.config.*`. This change is introduce as helm reserves global chart values across all the helm charts. This conflicted the use of OpenMetadata helm charts along with other helm charts for organizations using common helm values yaml files.
For example, with `1.0.X` Application version Releases, helm values would look like below -
```yaml
global:
...
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "user1"
botPrincipals:
- "<service_application_client_id>"
principalDomain: "open-metadata.org"
authentication:
provider: "google"
publicKeys:
- "https://www.googleapis.com/oauth2/v3/certs"
- "http://openmetadata:8585/api/v1/system/config/jwks"
authority: "https://accounts.google.com"
clientId: "{client id}"
callbackUrl: "http://localhost:8585/callback"
...
```
With OpenMetadata Application version `1.1.0` and above, the above config will need to be updated as
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "user1"
- "user2"
botPrincipals:
- "<service_application_client_id>"
principalDomain: "open-metadata.org"
authentication:
provider: "google"
publicKeys:
- "https://www.googleapis.com/oauth2/v3/certs"
- "http://openmetadata:8585/api/v1/system/config/jwks"
authority: "https://accounts.google.com"
clientId: "{client id}"
callbackUrl: "http://localhost:8585/callback"
```
A quick and easy way to update the config is to use [yq](https://mikefarah.gitbook.io/yq/) utility to manipulate YAML files.
```bash
yq -i -e '{"openmetadata": {"config": .global}}' openmetadata.values.yml
```
The above command will update `global.*` with `openmetadata.config.*` yaml config. Please note, the above command is only recommended for users with custom helm values file explicit for OpenMetadata Helm Charts.
For more information, visit the official helm docs for [global chart values](https://helm.sh/docs/chart_template_guide/subcharts_and_globals/#global-chart-values).
### Pipeline Service Client Configuration ### Pipeline Service Client Configuration
If reusing an old YAML configuration file, make sure to add the following inside `pipelineServiceClientConfiguration`: If reusing an old YAML configuration file, make sure to add the following inside `pipelineServiceClientConfiguration`:
@ -224,3 +284,5 @@ impacts BigQuery, Datalake and any other source where you are directly passing t
- Glue now supports custom database names via `databaseName`. - Glue now supports custom database names via `databaseName`.
- Snowflake supports the `clientSessionKeepAlive` parameter to keep the session open for long processes. - Snowflake supports the `clientSessionKeepAlive` parameter to keep the session open for long processes.
- Kafka and Redpanda now have the `saslMechanism` based on enum values `["PLAIN", "GSSAPI", "SCRAM-SHA-256", "SCRAM-SHA-512", "OAUTHBEARER"]`. - Kafka and Redpanda now have the `saslMechanism` based on enum values `["PLAIN", "GSSAPI", "SCRAM-SHA-256", "SCRAM-SHA-512", "OAUTHBEARER"]`.
- OpenMetadata Server Docker Image now installs the OpenMetadata Libraries under `/opt/openmetadata` directory
- Bumped up ElasticSearch version for Docker and Kubernetes OpenMetadata Dependencies Helm Chart to `7.16.3`

View File

@ -180,25 +180,27 @@ global:
✅ After 1.0.0 Helm Chart Release, the `global.pipelineServiceClient` section will replace the above `airflow` section - ✅ After 1.0.0 Helm Chart Release, the `global.pipelineServiceClient` section will replace the above `airflow` section -
```yaml ```yaml
global: openmetadata:
config:
...
pipelineServiceClientConfig:
enabled: true
className: "org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"
# endpoint url for airflow
apiEndpoint: http://openmetadata-dependencies-web.default.svc.cluster.local:8080
# this will be the api endpoint url of OpenMetadata Server
metadataApiEndpoint: http://openmetadata.default.svc.cluster.local:8585/api
# possible values are "no-ssl", "ignore", "validate"
verifySsl: "no-ssl"
ingestionIpInfoEnabled: false
# local path in Airflow Pod
sslCertificatePath: "/no/path"
auth:
username: admin
password:
secretRef: airflow-secrets
secretKey: openmetadata-airflow-password
... ...
pipelineServiceClientConfig:
enabled: true
className: "org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"
# endpoint url for airflow
apiEndpoint: http://openmetadata-dependencies-web.default.svc.cluster.local:8080
# this will be the api endpoint url of OpenMetadata Server
metadataApiEndpoint: http://openmetadata.default.svc.cluster.local:8585/api
# possible values are "no-ssl", "ignore", "validate"
verifySsl: "no-ssl"
ingestionIpInfoEnabled: false
# local path in Airflow Pod
sslCertificatePath: "/no/path"
auth:
username: admin
password:
secretRef: airflow-secrets
secretKey: openmetadata-airflow-password
... ...
``` ```

View File

@ -18,6 +18,66 @@ Upgrading from 1.0 to 1.1 can be done directly on your instances. This page will
## Breaking Changes for 1.1 Stable Release ## Breaking Changes for 1.1 Stable Release
### OpenMetadata Helm Chart Values
With `1.1.0` we are moving away from `global.*` helm values under openmetadata helm charts to `openmetadata.config.*`. This change is introduce as helm reserves global chart values across all the helm charts. This conflicted the use of OpenMetadata helm charts along with other helm charts for organizations using common helm values yaml files.
For example, with `1.0.X` Application version Releases, helm values would look like below -
```yaml
global:
...
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "user1"
botPrincipals:
- "<service_application_client_id>"
principalDomain: "open-metadata.org"
authentication:
provider: "google"
publicKeys:
- "https://www.googleapis.com/oauth2/v3/certs"
- "http://openmetadata:8585/api/v1/system/config/jwks"
authority: "https://accounts.google.com"
clientId: "{client id}"
callbackUrl: "http://localhost:8585/callback"
...
```
With OpenMetadata Application version `1.1.0` and above, the above config will need to be updated as
```yaml
openmetadata:
config:
authorizer:
className: "org.openmetadata.service.security.DefaultAuthorizer"
containerRequestFilter: "org.openmetadata.service.security.JwtFilter"
initialAdmins:
- "user1"
- "user2"
botPrincipals:
- "<service_application_client_id>"
principalDomain: "open-metadata.org"
authentication:
provider: "google"
publicKeys:
- "https://www.googleapis.com/oauth2/v3/certs"
- "http://openmetadata:8585/api/v1/system/config/jwks"
authority: "https://accounts.google.com"
clientId: "{client id}"
callbackUrl: "http://localhost:8585/callback"
```
A quick and easy way to update the config is to use [yq](https://mikefarah.gitbook.io/yq/) utility to manipulate YAML files.
```bash
yq -i -e '{"openmetadata": {"config": .global}}' openmetadata.values.yml
```
The above command will update `global.*` with `openmetadata.config.*` yaml config. Please note, the above command is only recommended for users with custom helm values file explicit for OpenMetadata Helm Charts.
For more information, visit the official helm docs for [global chart values](https://helm.sh/docs/chart_template_guide/subcharts_and_globals/#global-chart-values).
### Pipeline Service Client Configuration ### Pipeline Service Client Configuration
If reusing an old YAML configuration file, make sure to add the following inside `pipelineServiceClientConfiguration`: If reusing an old YAML configuration file, make sure to add the following inside `pipelineServiceClientConfiguration`:
@ -174,3 +234,5 @@ impacts BigQuery, Datalake and any other source where you are directly passing t
- Snowflake supports the `clientSessionKeepAlive` parameter to keep the session open for long processes. - Snowflake supports the `clientSessionKeepAlive` parameter to keep the session open for long processes.
- Databricks now supports the `useUnityCatalog` parameter to extract the metadata from unity catalog instead of hive metastore. - Databricks now supports the `useUnityCatalog` parameter to extract the metadata from unity catalog instead of hive metastore.
- Kafka and Redpanda now have the `saslMechanism` based on enum values `["PLAIN", "GSSAPI", "SCRAM-SHA-256", "SCRAM-SHA-512", "OAUTHBEARER"]`. - Kafka and Redpanda now have the `saslMechanism` based on enum values `["PLAIN", "GSSAPI", "SCRAM-SHA-256", "SCRAM-SHA-512", "OAUTHBEARER"]`.
- OpenMetadata Server Docker Image now installs the OpenMetadata Libraries under `/opt/openmetadata` directory
- Bumped up ElasticSearch version for Docker and Kubernetes OpenMetadata Dependencies Helm Chart to `7.16.3`

View File

@ -84,11 +84,12 @@ AUTHENTICATION_PUBLIC_KEYS=[https://www.googleapis.com/oauth2/v3/certs, http://l
- In the case of **kubernetes**, you have to update `publicKeys` values: - In the case of **kubernetes**, you have to update `publicKeys` values:
```yaml ```yaml
global: openmetadata:
authentication: config:
publicKeys: authentication:
- "https://www.googleapis.com/oauth2/v3/certs" publicKeys:
- "http://localhost:8585/api/v1/system/config/jwks" - "https://www.googleapis.com/oauth2/v3/certs"
- "http://localhost:8585/api/v1/system/config/jwks"
``` ```
**3. Redeploying ingestion pipelines** **3. Redeploying ingestion pipelines**