mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2025-08-18 14:06:59 +00:00
Modify Docs related to SSL (#7927)
This commit is contained in:
parent
f77f0c8130
commit
65525316d7
@ -449,7 +449,41 @@ metadata profile -c <path-to-yaml>
|
|||||||
```
|
```
|
||||||
|
|
||||||
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
||||||
|
## SSL Configuration
|
||||||
|
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
source:
|
||||||
|
type: mariadb
|
||||||
|
serviceName: "<service name>"
|
||||||
|
serviceConnection:
|
||||||
|
config:
|
||||||
|
type: MariaDB
|
||||||
|
username: <username>
|
||||||
|
password: <password>
|
||||||
|
hostPort: <hostPort>
|
||||||
|
...
|
||||||
|
...
|
||||||
|
connectionArguments:
|
||||||
|
ssl:
|
||||||
|
ssl_ca: /path/to/client-ssl/ca.pem,
|
||||||
|
ssl_cert: /path/to/client-ssl/client-cert.pem
|
||||||
|
ssl_key: /path/to/client-ssl/client-key.pem
|
||||||
|
#ssl_disabled: True #boolean
|
||||||
|
#ssl_verify_cert: True #boolean
|
||||||
|
#ssl_verify_identity: True #boolean
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
- **ssl**: A dict of arguments which contains:
|
||||||
|
- **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate.
|
||||||
|
- **ssl_cert**: Path to the file that contains a PEM-formatted client certificate.
|
||||||
|
- **ssl_disabled**: A boolean value that disables usage of TLS.
|
||||||
|
- **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate.
|
||||||
|
- **ssl_verify_cert**: Set to true to check the server certificate's validity.
|
||||||
|
- **ssl_verify_identity**: Set to true to check the server's identity.
|
||||||
## DBT Integration
|
## DBT Integration
|
||||||
|
|
||||||
You can learn more about how to ingest DBT models' definitions and their lineage [here](https://docs.open-metadata.org/openmetadata/ingestion/workflows/metadata/dbt).
|
You can learn more about how to ingest DBT models' definitions and their lineage [here](https://docs.open-metadata.org/openmetadata/ingestion/workflows/metadata/dbt).
|
||||||
|
@ -168,6 +168,17 @@ caption="Configure Metadata Ingestion Page"
|
|||||||
- **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system.
|
- **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system.
|
||||||
- **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema
|
- **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema
|
||||||
|
|
||||||
|
#### **SSL Configuration**
|
||||||
|
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source.
|
||||||
|
|
||||||
|
- **ssl**: A dict of arguments which contains:
|
||||||
|
- **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate.
|
||||||
|
- **ssl_cert**: Path to the file that contains a PEM-formatted client certificate.
|
||||||
|
- **ssl_disabled**: A boolean value that disables usage of TLS.
|
||||||
|
- **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate.
|
||||||
|
- **ssl_verify_cert**: Set to true to check the server certificate's validity.
|
||||||
|
- **ssl_verify_identity**: Set to true to check the server's identity.
|
||||||
### 7. Schedule the Ingestion and Deploy
|
### 7. Schedule the Ingestion and Deploy
|
||||||
|
|
||||||
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
||||||
|
@ -449,6 +449,41 @@ metadata profile -c <path-to-yaml>
|
|||||||
```
|
```
|
||||||
|
|
||||||
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
||||||
|
## SSL Configuration
|
||||||
|
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
source:
|
||||||
|
type: mysql
|
||||||
|
serviceName: "<service name>"
|
||||||
|
serviceConnection:
|
||||||
|
config:
|
||||||
|
type: Mysql
|
||||||
|
username: <username>
|
||||||
|
password: <password>
|
||||||
|
hostPort: <hostPort>
|
||||||
|
...
|
||||||
|
...
|
||||||
|
connectionArguments:
|
||||||
|
ssl:
|
||||||
|
ssl_ca: /path/to/client-ssl/ca.pem,
|
||||||
|
ssl_cert: /path/to/client-ssl/client-cert.pem
|
||||||
|
ssl_key: /path/to/client-ssl/client-key.pem
|
||||||
|
#ssl_disabled: True #boolean
|
||||||
|
#ssl_verify_cert: True #boolean
|
||||||
|
#ssl_verify_identity: True #boolean
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
- **ssl**: A dict of arguments which contains:
|
||||||
|
- **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate.
|
||||||
|
- **ssl_cert**: Path to the file that contains a PEM-formatted client certificate.
|
||||||
|
- **ssl_disabled**: A boolean value that disables usage of TLS.
|
||||||
|
- **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate.
|
||||||
|
- **ssl_verify_cert**: Set to true to check the server certificate's validity.
|
||||||
|
- **ssl_verify_identity**: Set to true to check the server's identity.
|
||||||
|
|
||||||
## DBT Integration
|
## DBT Integration
|
||||||
|
|
||||||
|
@ -170,6 +170,18 @@ caption="Configure Metadata Ingestion Page"
|
|||||||
- **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system.
|
- **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system.
|
||||||
- **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema
|
- **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema
|
||||||
|
|
||||||
|
#### **SSL Configuration**
|
||||||
|
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source.
|
||||||
|
|
||||||
|
- **ssl**: A dict of arguments which contains:
|
||||||
|
- **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate.
|
||||||
|
- **ssl_cert**: Path to the file that contains a PEM-formatted client certificate.
|
||||||
|
- **ssl_disabled**: A boolean value that disables usage of TLS.
|
||||||
|
- **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate.
|
||||||
|
- **ssl_verify_cert**: Set to true to check the server certificate's validity.
|
||||||
|
- **ssl_verify_identity**: Set to true to check the server's identity.
|
||||||
|
|
||||||
### 7. Schedule the Ingestion and Deploy
|
### 7. Schedule the Ingestion and Deploy
|
||||||
|
|
||||||
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
||||||
|
@ -553,6 +553,44 @@ metadata profile -c <path-to-yaml>
|
|||||||
|
|
||||||
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
||||||
|
|
||||||
|
## SSL Configuration
|
||||||
|
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
source:
|
||||||
|
type: redshift
|
||||||
|
serviceName: <service name>
|
||||||
|
serviceConnection:
|
||||||
|
config:
|
||||||
|
type: Redshift
|
||||||
|
hostPort: cluster.name.region.redshift.amazonaws.com:5439
|
||||||
|
username: username
|
||||||
|
...
|
||||||
|
...
|
||||||
|
...
|
||||||
|
connectionArguments:
|
||||||
|
sslmode: <ssl-mode>
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### SSL Modes
|
||||||
|
|
||||||
|
There are couple of types of SSL modes that Redshift supports which can be added to ConnectionArguments, they are as follows:
|
||||||
|
- **disable**: SSL is disabled and the connection is not encrypted.
|
||||||
|
|
||||||
|
- **allow**: SSL is used if the server requires it.
|
||||||
|
|
||||||
|
- **prefer**: SSL is used if the server supports it. Amazon Redshift supports SSL, so SSL is used when you set sslmode to prefer.
|
||||||
|
|
||||||
|
- **require**: SSL is required.
|
||||||
|
|
||||||
|
- **verify-ca**: SSL must be used and the server certificate must be verified.
|
||||||
|
|
||||||
|
- **verify-full**: SSL must be used. The server certificate must be verified and the server hostname must match the hostname attribute on the certificate.
|
||||||
|
|
||||||
|
For more information, you can visit [Redshift SSL documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html)
|
||||||
## DBT Integration
|
## DBT Integration
|
||||||
|
|
||||||
You can learn more about how to ingest DBT models' definitions and their lineage [here](https://docs.open-metadata.org/openmetadata/ingestion/workflows/metadata/dbt).
|
You can learn more about how to ingest DBT models' definitions and their lineage from [here](https://docs.open-metadata.org/openmetadata/ingestion/workflows/metadata/dbt).
|
||||||
|
@ -177,6 +177,26 @@ caption="Configure Metadata Ingestion Page"
|
|||||||
|
|
||||||
</Note>
|
</Note>
|
||||||
|
|
||||||
|
#### **SSL Configuration**
|
||||||
|
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source.
|
||||||
|
|
||||||
|
##### **SSL Modes**
|
||||||
|
|
||||||
|
There are couple of types of SSL modes that Redshift supports which can be added to ConnectionArguments, they are as follows:
|
||||||
|
- **disable**: SSL is disabled and the connection is not encrypted.
|
||||||
|
|
||||||
|
- **allow**: SSL is used if the server requires it.
|
||||||
|
|
||||||
|
- **prefer**: SSL is used if the server supports it. Amazon Redshift supports SSL, so SSL is used when you set sslmode to prefer.
|
||||||
|
|
||||||
|
- **require**: SSL is required.
|
||||||
|
|
||||||
|
- **verify-ca**: SSL must be used and the server certificate must be verified.
|
||||||
|
|
||||||
|
- **verify-full**: SSL must be used. The server certificate must be verified and the server hostname must match the hostname attribute on the certificate.
|
||||||
|
|
||||||
|
For more information, you can visit [Redshift SSL documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html)
|
||||||
### 7. Schedule the Ingestion and Deploy
|
### 7. Schedule the Ingestion and Deploy
|
||||||
|
|
||||||
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
||||||
|
@ -463,6 +463,34 @@ metadata profile -c <path-to-yaml>
|
|||||||
```
|
```
|
||||||
|
|
||||||
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow.
|
||||||
|
## SSL Configuration
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under **connectionArguments** which is placed in source.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
source:
|
||||||
|
type: trino
|
||||||
|
serviceName: <service name>
|
||||||
|
serviceConnection:
|
||||||
|
config:
|
||||||
|
type: Trino
|
||||||
|
hostPort: <hostPort>
|
||||||
|
username: <username>
|
||||||
|
catalog: <catalog>
|
||||||
|
...
|
||||||
|
...
|
||||||
|
connectionArguments:
|
||||||
|
verify: <verify-mode>
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### SSL Modes
|
||||||
|
There are couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows:
|
||||||
|
- **false**: In order to disable SSL verification, set the `verify` parameter to `False`.
|
||||||
|
- **\<path-to-crt\>**: To use self-signed certificates, specify a path to the certificate in `verify` parameter.
|
||||||
|
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification).
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## DBT Integration
|
## DBT Integration
|
||||||
|
|
||||||
|
@ -169,6 +169,17 @@ caption="Configure Metadata Ingestion Page"
|
|||||||
- **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system.
|
- **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system.
|
||||||
- **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema
|
- **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema
|
||||||
|
|
||||||
|
|
||||||
|
#### **SSL Configuration**
|
||||||
|
In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under **connectionArguments** which is placed in source.
|
||||||
|
|
||||||
|
##### **SSL Modes**
|
||||||
|
There are couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows:
|
||||||
|
- **false**: In order to disable SSL verification, set the `verify` parameter to `False`.
|
||||||
|
- **\<path-to-crt\>**: To use self-signed certificates, specify a path to the certificate in `verify` parameter.
|
||||||
|
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification).
|
||||||
|
|
||||||
|
|
||||||
### 7. Schedule the Ingestion and Deploy
|
### 7. Schedule the Ingestion and Deploy
|
||||||
|
|
||||||
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
Scheduling can be set up at an hourly, daily, or weekly cadence. The
|
||||||
|
Loading…
x
Reference in New Issue
Block a user