Doc: Update security docs related to SSO and JWT configuration (#10002)

* Doc: Update security docs related to SSO and JWT configuration

* Minor typo fix
This commit is contained in:
Nahuel 2023-01-30 14:13:23 +01:00 committed by GitHub
parent f4043f23b3
commit e152e734c2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 104 additions and 231 deletions

View File

@ -86,14 +86,3 @@ After the applying these steps, you can update the configuration of your deploym
## Configure Ingestion
The ingestion can be configured by [Enabling JWT Tokens](/deployment/security/enable-jwt-tokens).
When setting up the YAML config for the connector, update the `workflowConfig` as follows:
```yaml
workflowConfig:
openMetadataServerConfig:
hostPort: http://localhost:8585/api
authProvider: openmetadata
securityConfig:
jwtToken: jwt_token
```

View File

@ -47,9 +47,10 @@ Follow the sections in this guide to set up Auth0 SSO.
<Image src="/images/deployment/security/auth0/credentials.png" alt="credentials"/>
## Create Service Account
## Create Service Account (optional)
This section will guide to create the Ingestion Bot service account.
This is a guide to create ingestion bot service account. This step is optional if you configure the ingestion-bot with
the JWT Token, you can follow the documentation of [Enable JWT Tokens](/deployment/security/enable-jwt-tokens).
### Step 1: Enable Client-Credential

View File

@ -49,7 +49,10 @@ Admin permissions are required to register the application on the Azure portal.
"authority": "https://login.microsoftonline.com/c11234b7c-b1b2-9854-0mn1-56abh3dea295"
```
## Create Service Application
## Create Service Application (optional)
This is a guide to create ingestion bot service account. This step is optional if you configure the ingestion-bot with
the JWT Token, you can follow the documentation of [Enable JWT Tokens](/deployment/security/enable-jwt-tokens).
### Step 1: Access Tokens and ID Tokens

View File

@ -22,7 +22,7 @@ Below are the required steps to set up the Basic Login:
### Authentication Configuration
- The following configuration controls the auth mechanism for OpenMetadata. Update the mentioned fields as required.
The following configuration controls the auth mechanism for OpenMetadata. Update the mentioned fields as required.
```yaml
authenticationConfiguration:
@ -34,17 +34,14 @@ authenticationConfiguration:
For the Basic auth we need to set:
- `provider -> basic`
- `publicKeyUrls -> {http|https}://{your_domain}:{port}}/api/v1/config/jwks`
- `authority -> {your_domain}`
- `enableSelfSignup -> This flag indicates if users can come and signup by themselves on the OM`
- `provider`: basic
- `publicKeyUrls`: {http|https}://{your_domain}:{port}}/api/v1/config/jwks
- `authority`: {your_domain}
- `enableSelfSignup`: This flag indicates if users can come and signup by themselves on the OM
### Authorizer Configuration
- This configuration controls the authorizer for OpenMetadata:
This configuration controls the authorizer for OpenMetadata:
```yaml
authorizerConfiguration:
@ -55,104 +52,48 @@ authorizerConfiguration:
For the Basic auth we need to set:
- `adminPrincipals -> admin usernames to bootstrap the server with, comma-separated values`
- `allowedEmailRegistrationDomains -> This controls what all domain are allowed for email registration can be your {princialDomain} as well, for example gmail.com, outlook.comm etc.`
- `principalDomain -> This controls what all domain are allowed for email registration, for example gmail.com, outlook.comm etc.`
- `adminPrincipals`: admin usernames to bootstrap the server with, comma-separated values.
- `allowedEmailRegistrationDomains`: This controls what all domain are allowed for email registration can be your {princialDomain} as well, for example gmail.com, outlook.comm etc.
- `principalDomain`: This controls what all domain are allowed for email registration, for example gmail.com, outlook.comm etc.
<Note>
Please note the following are the formats to bootstrap admins on server startup:
Please note the following are the formats to bootstrap admins on server startup: `[admin1,admin2,admin3]`
`[admin1,admin2,admin3]`
This works for SMTP-enabled servers, Login Password for these are generated randomly and sent to the mail `adminName`@`principalDomain`.
- This works for SMTP-enabled servers, Login Password for these are generated randomly and sent to the mail {adminName}@{principalDomain}. If SMTP is not enabled for OpenMetadata, please use the below method to create admin users.
If SMTP is not enabled for OpenMetadata, please use the below method to create admin users: `[admin1:password1,admin2:password2,admin3:password3]`
`[admin1:password1,admin2:password2,admin3:[password3]]`
- This allows to bootstrap the server with given password, later on can be changed by specific users by visiting profile page.
This allows to bootstrap the server with given password, later on can be changed by specific users by visiting profile page.
</Note>
### Jwt Configuration
## Metadata Ingestion
- Please note that the JWT Configuration is mandatory to work with UserName/Password Login.
For ingesting metadata when Basic Auth is enabled, it is mandatory to configure the `ingestion-bot` account with the JWT
configuration. To know how to enable it, you can follow the documentation of [Enable JWT Tokens](/deployment/security/enable-jwt-tokens).
```yaml
jwtTokenConfiguration:
rsapublicKeyFilePath: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}
rsaprivateKeyFilePath: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}
jwtissuer: ${JWT_ISSUER:-"open-metadata.org"}
keyId: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}
```
<Note>
By default, the `jwtTokenConfiguration` is shipped with OM.
### For Local/Testing Deployment
- You can work with the existing configuration or generate private/public keys.
### For Production Deployment
- It is a **MUST** to update the JWT configuration. The following steps can be used.
- Generating Private/Public Keys
```commandline
openssl genrsa -out private_key.pem 2048
openssl pkcs8 -topk8 -inform PEM -outform DER -in private_key.pem -out private_key.der -nocrypt
openssl rsa -in private_key.pem -pubout -outform DER -out public_key.der
```
Update below with path of above generated private_key.der and public_key.der.
```yaml
rsapublicKeyFilePath: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}
rsaprivateKeyFilePath: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}
```
Jwt Issuer can be your `principalDomain`
```yaml
jwtissuer: ${JWT_ISSUER:-"open-metadata.org"}
```
The `KeyID` is a randomly generated UUID string. Use any UUID generator to get a new `KeyID`.
```yaml
keyId: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}
```
</Note>
### Setting up SMTP Server
- Basic Authentication is successfully set. For a better login experience, we can also set up the SMTP server to allow the users to
Reset Password, Account Status Updates etc. as well.
Basic Authentication is successfully set. For a better login experience, we can also set up the SMTP server to allow the
users to Reset Password, Account Status Updates, etc. as well.
```yaml
email:
emailingEntity: ${OM_EMAIL_ENTITY:-"OpenMetadata"} -> Company Name (Optional)
supportUrl: ${OM_SUPPORT_URL:-"https://slack.open-metadata.org"} -> SupportUrl (Optional)
enableSmtpServer : ${AUTHORIZER_ENABLE_SMTP:-false} -> True/False
openMetadataUrl: ${OPENMETADATA_SERVER_URL:-""} -> {http/https}://{your_domain}
serverEndpoint: ${SMTP_SERVER_ENDPOINT:-""} -> (Ex :- smtp.gmail.com)
serverPort: ${SMTP_SERVER_PORT:-""} -> (SSL/TLS port)
username: ${SMTP_SERVER_USERNAME:-""} -> (SMTP Server Username)
password: ${SMTP_SERVER_PWD:-""} -> (SMTP Server Password)
emailingEntity: ${OM_EMAIL_ENTITY:-"OpenMetadata"}`: Company Name (Optional)
supportUrl: ${OM_SUPPORT_URL:-"https://slack.open-metadata.org"}`: SupportUrl (Optional)
enableSmtpServer : ${AUTHORIZER_ENABLE_SMTP:-false}`: True/False
openMetadataUrl: ${OPENMETADATA_SERVER_URL:-""}`: {http/https}://{your_domain}
serverEndpoint: ${SMTP_SERVER_ENDPOINT:-""}`: (Ex :- smtp.gmail.com)
serverPort: ${SMTP_SERVER_PORT:-""}`: (SSL/TLS port)
username: ${SMTP_SERVER_USERNAME:-""}`: (SMTP Server Username)
password: ${SMTP_SERVER_PWD:-""}`: (SMTP Server Password)
transportationStrategy: ${SMTP_SERVER_STRATEGY:-"SMTP_TLS"}
```
<Note>
- Following are valid value for transportation Strategy
Following are valid value for transportation strategy:
`SMTP -> IF SMTP port is 25 use this`
`SMTPS -> IF SMTP port is 465 use this`
`SMTP_TLS -> IF SMTP port is 587 use this`
</Note>
- `SMTP`: If SMTP port is 25 use this
- `SMTPS`: If SMTP port is 465 use this
- `SMTP_TLS`: If SMTP port is 587 use this

View File

@ -18,7 +18,13 @@ this gap, we shipped JWT token generation and authentication within OpenMetadata
## Create Private / Public key
To create private/public key use the following commands
### For local/testing deployment
You can work with the existing configuration or generate private/public keys. By default, the `jwtTokenConfiguration` is shipped with OM.
### For production deployment
It is a **MUST** to update the JWT configuration. To create private/public key use the following commands can be used:
```commandline
openssl genrsa -out private_key.pem 2048
@ -56,32 +62,40 @@ authenticationConfiguration:
provider: ${AUTHENTICATION_PROVIDER:-no-auth}
# This will only be valid when provider type specified is customOidc
providerName: ${CUSTOM_OIDC_AUTHENTICATION_PROVIDER_NAME:-""}
publicKeyUrls: ${AUTHENTICATION_PUBLIC_KEYS:-[https://www.googleapis.com/oauth2/v3/certs]}
publicKeyUrls: ${AUTHENTICATION_PUBLIC_KEYS:-[{your SSO public keys URL}]}
authority: ${AUTHENTICATION_AUTHORITY:-https://accounts.google.com}
clientId: ${AUTHENTICATION_CLIENT_ID:-""}
callbackUrl: ${AUTHENTICATION_CALLBACK_URL:-""}
jwtPrincipalClaims: ${AUTHENTICATION_JWT_PRINCIPAL_CLAIMS:-[email,preferred_username,sub]}
```
add `http://localhost:8585/api/v1/config/jwks` to `publicKeyUrls`. You should append to the existing configuration such that
add `http://{your domain}:8585/api/v1/config/jwks` to `publicKeyUrls`. You should append to the existing configuration such that
your SSO and JWTToken auth verification will work.
```yaml
publicKeyUrls: ${AUTHENTICATION_PUBLIC_KEYS:-[{your SSO public keys URL}, http://{your domain}:8585/api/v1/config/jwks]}
```
Once you configure the above settings, restart OpenMetadata server .
## Generate Token
Once the above configuration is updated, the server is restarted. Admin can go to Settings -> Bots page.
<Image src="/images/deployment/security/enable-jwt/bot.png" alt="bots"/>
<Image src="/images/deployment/security/enable-jwt/bot.png" alt="Bot settings page" caption="Bot settings page"/>
Click on the generate token to create a token for the ingestion bot.
Click on the `ingestion-bot`. The current token can be revoked, or you can create a new one.
<Image src="/images/deployment/security/enable-jwt/bot-jwt-token.png" alt="Bot credentials edition" caption="Edit JWT Token for ingestion-bot"/>
## Configure Ingestion
The generated token from the above page should pass onto the ingestion framework so that the ingestion can make calls
securely to OpenMetadata. Make sure this token is not shared and stored securely.
### Using Airflow APIs
After `0.12.1` version, we don't need any other additional change in the configuration after configuring the `ingestion-bot`.
### Using Airflow APIs (only before 0.12.1)
If you are using OpenMetadata shipped Airflow container with our APIs to deploy ingestion workflows from the
OpenMetadata UIs. Configure the below section to enable JWT Token
@ -101,43 +115,18 @@ airflowConfiguration:
In the above configuration, you can see we configure `authProvider` to be "openmetadata" and `OM_AUTH_JWT_TOKEN` with the JWT token that was generated in the bots page.
### Using Ingestion Framework
### Running Ingestion from CLI
If you are running your own Airflow and using the ingestion framework from OpenMetadata APIs. Add the below
configuration to the workflow configuration you pass onto the ingestion framework
If you are running the ingestion from CLI. Add the below configuration to the workflow configuration you pass:
```yaml
source:
type: bigquery
serviceName: local_bigquery
serviceConnection:
config:
type: BigQuery
credentials:
gcsConfig:
type: service_account
projectId: project_id
privateKeyId: private_key_id
privateKey: private_key
clientEmail: gcpuser@project_id.iam.gserviceaccount.com
clientId: client_id
authUri: https://accounts.google.com/o/oauth2/auth
tokenUri: https://oauth2.googleapis.com/token
authProviderX509CertUrl: https://www.googleapis.com/oauth2/v1/certs
clientX509CertUrl: clientX509CertUrl
sourceConfig:
config:
type: DatabaseMetadata
sink:
type: metadata-rest
config: {}
workflowConfig:
openMetadataServerConfig:
hostPort: http://localhost:8585/api
authProvider: openmetadata
securityConfig:
jwtToken:
jwtToken: <jwt-token>
```
In the above section, under the `workflowConfig`, configure `authProvider` to be "openmetadata" and under `securityConfig`
section, add "jwtToken" and its value from the ingestion bot page.
section, add `jwtToken` and its value from the ingestion bot page.

View File

@ -72,9 +72,10 @@ After selecting the **Application Type**, name your project and give the authori
- You will find the **Client ID** and **Client Secret** in the top right corner
<Image src="/images/deployment/security/google/find-clientid-and-secret.png" alt="find-clientid-and-secret"/>
## Create Service Account
## Create Service Account (optional)
This is a guide to create ingestion bot service account.
This is a guide to create ingestion bot service account. This step is optional if you configure the ingestion-bot with
the JWT Token, you can follow the documentation of [Enable JWT Tokens](/deployment/security/enable-jwt-tokens).
### Step 1: Create Service-Account
- Navigate to your project dashboard

View File

@ -45,6 +45,14 @@ Follow the sections in this guide to set up Keycloak SSO.
- Click on `Save` button.
<Note>
Configuring a service account in Keycloak is optional if you configure the ingestion-bot with
the JWT Token, you can follow the documentation of [Enable JWT Tokens](/deployment/security/enable-jwt-tokens).
</Note>
### Step 5: Where to Find the Credentials
- Navigate to the `Credentials` tab.

View File

@ -3,20 +3,18 @@ title: Ldap Authentication
slug: /deployment/security/ldap
---
# Ldap Authentication
# Setting up Ldap Authentication
OpenMetadata allows using LDAP for validating email and password authentication.
Once setup successfully, the user should be able to sign in to OpenMetadata using the Ldap credentials.
# Setting up Ldap Authentication
Below are the required steps to set up the LDAP Authentication:
## Set up Configurations in openmetadata.yaml
### Authentication Configuration
- The following configuration controls the auth mechanism for OpenMetadata. Update the mentioned fields as required.
The following configuration controls the auth mechanism for OpenMetadata. Update the mentioned fields as required.
```yaml
authenticationConfiguration:
@ -31,7 +29,7 @@ authenticationConfiguration:
"dnAdminPassword": ${AUTHENTICATION_LOOKUP_ADMIN_PWD:-"secret"}
"userBaseDN": ${AUTHENTICATION_USER_LOOKUP_BASEDN:-"ou=people,dc=example,dc=com"}
"mailAttributeName": ${AUTHENTICATION_USER_MAIL_ATTR:-email}
#optional
# Optional
"maxPoolSize": ${AUTHENTICATION_LDAP_POOL_SIZE:-3}
"sslEnabled": ${AUTHENTICATION_LDAP_SSL_ENABLED:-true}
"keyStorePath": ${AUTHENTICATION_LDAP_KEYSTORE_PATH:-"/Users/mohityadav/sslTest/client/keystore.ks"}
@ -42,29 +40,22 @@ authenticationConfiguration:
For the LDAP auth we need to set:
OpenMetadata Specific Configuration :-
OpenMetadata Specific Configuration :
- `provider -> ldap`
- `publicKeyUrls -> {http|https}://{your_domain}:{port}}/api/v1/config/jwks`
- `authority -> {your_domain}`
- `enableSelfSignup -> This has to be false for Ldap.`
- `provider`: ldap
- `publicKeyUrls`: {http|https}://{your_domain}:{port}}/api/v1/config/jwks
- `authority`: {your_domain}
- `enableSelfSignup`: This has to be false for Ldap.
<Note>
Mandatory LDAP Specific Configuration:
- `host -> hostName for the Ldap Server (Ex - localhost)`
- `port -> port of the Ldap Server to connect to (Ex - 10636)`
- `dnAdminPrincipal -> This is the DN Admin Principal(Complete path Example :- cn=admin,dc=example,dc=com ) with a lookup access in the Directory`
- `dnAdminPassword -> Above Admin Principal Password`
- `userBaseDN -> User Base DN(Complete path Example :- ou=people,dc=example,dc=com)`
- `host`: hostName for the Ldap Server (Ex - localhost).
- `port`: port of the Ldap Server to connect to (Ex - 10636).
- `dnAdminPrincipal`: This is the DN Admin Principal(Complete path Example :- cn=admin,dc=example,dc=com ) with a lookup access in the Directory.
- `dnAdminPassword`: Above Admin Principal Password.
- `userBaseDN`: User Base DN(Complete path Example :- ou=people,dc=example,dc=com).
</Note>
@ -74,21 +65,16 @@ Please see the below image for a sample LDAP Configuration in ApacheDS.
Advanced LDAP Specific Configuration (Optional):
- `maxPoolSize -> Connection Pool Size to use to connect to LDAP Server.`
- `sslEnabled -> Set to true if the SSL is enable to connecto to LDAP Server.`
- `keyStorePath -> Path of Keystore in case the sslEnabled is set to true`
- `keyStorePassword -> Truststore Password`
- `truststoreFormat -> TrustStore Format (Example :- JKS)`
- `verifyCertificateHostname -> Controls using TrustAllSSLSocketVerifier vs HostNameSSLSocketVerifier. In case the certificate contains cn=hostname of the Ldap Server set it to true.`
- `maxPoolSize`: Connection Pool Size to use to connect to LDAP Server.
- `sslEnabled`: Set to true if the SSL is enable to connecto to LDAP Server.
- `keyStorePath`: Path of Keystore in case the sslEnabled is set to true.
- `keyStorePassword`: Truststore Password.
- `truststoreFormat`: TrustStore Format (Example :- JKS).
- `verifyCertificateHostname`: Controls using TrustAllSSLSocketVerifier vs HostNameSSLSocketVerifier. In case the certificate contains cn=hostname of the Ldap Server set it to true.
### Authorizer Configuration
- This configuration controls the authorizer for OpenMetadata:
This configuration controls the authorizer for OpenMetadata:
```yaml
authorizerConfiguration:
@ -98,59 +84,10 @@ authorizerConfiguration:
For the Ldap we need to set:
- `adminPrincipals -> This is the list of admin Principal for the OpenMetadata , if mail in ldap is example@openmetadata.org, then if we want this user to be admin in the OM, we should add 'example', in this list`
- `adminPrincipals`: This is the list of admin Principal for the OpenMetadata , if mail in ldap is example@openmetadata.org, then if we want this user to be admin in the OM, we should add 'example', in this list.
- `principalDomain`: Company Domain.
- `principalDomain -> Company Domain`
## Metadata Ingestion
### Jwt Configuration
- Please note that the JWT Configuration is mandatory to work with Ldap Authentication.
```yaml
jwtTokenConfiguration:
rsapublicKeyFilePath: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}
rsaprivateKeyFilePath: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}
jwtissuer: ${JWT_ISSUER:-"open-metadata.org"}
keyId: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}
```
<Note>
By default, the `jwtTokenConfiguration` is shipped with OM.
### For Local/Testing Deployment
- You can work with the existing configuration or generate private/public keys.
### For Production Deployment
- It is a **MUST** to update the JWT configuration. The following steps can be used.
- Generating Private/Public Keys
```commandline
openssl genrsa -out private_key.pem 2048
openssl pkcs8 -topk8 -inform PEM -outform DER -in private_key.pem -out private_key.der -nocrypt
openssl rsa -in private_key.pem -pubout -outform DER -out public_key.der
```
Update below with path of above generated private_key.der and public_key.der.
```yaml
rsapublicKeyFilePath: ${RSA_PUBLIC_KEY_FILE_PATH:-"./conf/public_key.der"}
rsaprivateKeyFilePath: ${RSA_PRIVATE_KEY_FILE_PATH:-"./conf/private_key.der"}
```
Jwt Issuer can be your `principalDomain`
```yaml
jwtissuer: ${JWT_ISSUER:-"open-metadata.org"}
```
The `KeyID` is a randomly generated UUID string. Use any UUID generator to get a new `KeyID`.
```yaml
keyId: ${JWT_KEY_ID:-"Gb389a-9f76-gdjs-a92j-0242bk94356"}
```
</Note>
For ingesting metadata when LDAP is enabled, it is mandatory to configure the `ingestion-bot` account with the JWT configuration.
To know how to enable it, you can follow the documentation of [Enable JWT Tokens](/deployment/security/enable-jwt-tokens).

View File

@ -115,9 +115,10 @@ The Issuer URL shows up as Dynamic by default. Change the Issuer URL to Okta URL
- The **Audience** is the same as the Client ID.
<Image src="/images/deployment/security/okta/click-edit-token.png" alt="click-edit-token"/>
## Create Service Application
## Create Service Application (optional)
This is a guide to create ingestion bot service app.
This is a guide to create ingestion bot service app. This step is optional if you configure the ingestion-bot with
the JWT Token, you can follow the documentation of [Enable JWT Tokens](/deployment/security/enable-jwt-tokens).
### Step 1: Generate Public/Private Key Pair
#### For a Test or Staging Instance:

View File

@ -44,7 +44,10 @@ Follow the sections in this guide to set up OneLogin SSO.
- Copy the Issuer URL
## Create Service Account
## Create Service Account (optional)
This step is optional if you configure the ingestion-bot with the JWT Token, you can follow the documentation of
[Enable JWT Tokens](/deployment/security/enable-jwt-tokens).
### Create Secret Key

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB