diff --git a/openmetadata-docs/content/openmetadata/connectors/database/mariadb/cli.md b/openmetadata-docs/content/openmetadata/connectors/database/mariadb/cli.md index cbc86ec2225..9d022df8508 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/mariadb/cli.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/mariadb/cli.md @@ -449,7 +449,41 @@ metadata profile -c ``` Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow. +## SSL Configuration +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source. + +```yaml +--- +source: + type: mariadb + serviceName: "" + serviceConnection: + config: + type: MariaDB + username: + password: + hostPort: + ... + ... + connectionArguments: + ssl: + ssl_ca: /path/to/client-ssl/ca.pem, + ssl_cert: /path/to/client-ssl/client-cert.pem + ssl_key: /path/to/client-ssl/client-key.pem + #ssl_disabled: True #boolean + #ssl_verify_cert: True #boolean + #ssl_verify_identity: True #boolean + +``` + + - **ssl**: A dict of arguments which contains: + - **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate. + - **ssl_cert**: Path to the file that contains a PEM-formatted client certificate. + - **ssl_disabled**: A boolean value that disables usage of TLS. + - **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate. + - **ssl_verify_cert**: Set to true to check the server certificate's validity. + - **ssl_verify_identity**: Set to true to check the server's identity. ## DBT Integration You can learn more about how to ingest DBT models' definitions and their lineage [here](https://docs.open-metadata.org/openmetadata/ingestion/workflows/metadata/dbt). diff --git a/openmetadata-docs/content/openmetadata/connectors/database/mariadb/index.md b/openmetadata-docs/content/openmetadata/connectors/database/mariadb/index.md index 3c43d0393f1..17cddc1ebf1 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/mariadb/index.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/mariadb/index.md @@ -168,6 +168,17 @@ caption="Configure Metadata Ingestion Page" - **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system. - **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema +#### **SSL Configuration** + +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source. + +- **ssl**: A dict of arguments which contains: + - **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate. + - **ssl_cert**: Path to the file that contains a PEM-formatted client certificate. + - **ssl_disabled**: A boolean value that disables usage of TLS. + - **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate. + - **ssl_verify_cert**: Set to true to check the server certificate's validity. + - **ssl_verify_identity**: Set to true to check the server's identity. ### 7. Schedule the Ingestion and Deploy Scheduling can be set up at an hourly, daily, or weekly cadence. The diff --git a/openmetadata-docs/content/openmetadata/connectors/database/mysql/cli.md b/openmetadata-docs/content/openmetadata/connectors/database/mysql/cli.md index c12966a0ce8..ac70253ff31 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/mysql/cli.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/mysql/cli.md @@ -449,6 +449,41 @@ metadata profile -c ``` Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow. +## SSL Configuration + +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source. + +```yaml +--- +source: + type: mysql + serviceName: "" + serviceConnection: + config: + type: Mysql + username: + password: + hostPort: + ... + ... + connectionArguments: + ssl: + ssl_ca: /path/to/client-ssl/ca.pem, + ssl_cert: /path/to/client-ssl/client-cert.pem + ssl_key: /path/to/client-ssl/client-key.pem + #ssl_disabled: True #boolean + #ssl_verify_cert: True #boolean + #ssl_verify_identity: True #boolean + +``` + + - **ssl**: A dict of arguments which contains: + - **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate. + - **ssl_cert**: Path to the file that contains a PEM-formatted client certificate. + - **ssl_disabled**: A boolean value that disables usage of TLS. + - **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate. + - **ssl_verify_cert**: Set to true to check the server certificate's validity. + - **ssl_verify_identity**: Set to true to check the server's identity. ## DBT Integration diff --git a/openmetadata-docs/content/openmetadata/connectors/database/mysql/index.md b/openmetadata-docs/content/openmetadata/connectors/database/mysql/index.md index 2d841fed33e..fa0f5d2e1a5 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/mysql/index.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/mysql/index.md @@ -170,6 +170,18 @@ caption="Configure Metadata Ingestion Page" - **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system. - **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema +#### **SSL Configuration** + +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source. + +- **ssl**: A dict of arguments which contains: + - **ssl_ca**: Path to the file that contains a PEM-formatted CA certificate. + - **ssl_cert**: Path to the file that contains a PEM-formatted client certificate. + - **ssl_disabled**: A boolean value that disables usage of TLS. + - **ssl_key**: Path to the file that contains a PEM-formatted private key for the client certificate. + - **ssl_verify_cert**: Set to true to check the server certificate's validity. + - **ssl_verify_identity**: Set to true to check the server's identity. + ### 7. Schedule the Ingestion and Deploy Scheduling can be set up at an hourly, daily, or weekly cadence. The diff --git a/openmetadata-docs/content/openmetadata/connectors/database/redshift/cli.md b/openmetadata-docs/content/openmetadata/connectors/database/redshift/cli.md index 3c8f8dd25e2..d801db88330 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/redshift/cli.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/redshift/cli.md @@ -553,6 +553,44 @@ metadata profile -c Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow. +## SSL Configuration + +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source. + +```yaml +--- +source: + type: redshift + serviceName: + serviceConnection: + config: + type: Redshift + hostPort: cluster.name.region.redshift.amazonaws.com:5439 + username: username + ... + ... + ... + connectionArguments: + sslmode: +``` + + +### SSL Modes + +There are couple of types of SSL modes that Redshift supports which can be added to ConnectionArguments, they are as follows: +- **disable**: SSL is disabled and the connection is not encrypted. + +- **allow**: SSL is used if the server requires it. + +- **prefer**: SSL is used if the server supports it. Amazon Redshift supports SSL, so SSL is used when you set sslmode to prefer. + +- **require**: SSL is required. + +- **verify-ca**: SSL must be used and the server certificate must be verified. + +- **verify-full**: SSL must be used. The server certificate must be verified and the server hostname must match the hostname attribute on the certificate. + +For more information, you can visit [Redshift SSL documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html) ## DBT Integration -You can learn more about how to ingest DBT models' definitions and their lineage [here](https://docs.open-metadata.org/openmetadata/ingestion/workflows/metadata/dbt). +You can learn more about how to ingest DBT models' definitions and their lineage from [here](https://docs.open-metadata.org/openmetadata/ingestion/workflows/metadata/dbt). diff --git a/openmetadata-docs/content/openmetadata/connectors/database/redshift/index.md b/openmetadata-docs/content/openmetadata/connectors/database/redshift/index.md index 3641a021f3f..0ea3f93b1e4 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/redshift/index.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/redshift/index.md @@ -177,6 +177,26 @@ caption="Configure Metadata Ingestion Page" +#### **SSL Configuration** + +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under connectionArguments which is placed in the source. + +##### **SSL Modes** + +There are couple of types of SSL modes that Redshift supports which can be added to ConnectionArguments, they are as follows: +- **disable**: SSL is disabled and the connection is not encrypted. + +- **allow**: SSL is used if the server requires it. + +- **prefer**: SSL is used if the server supports it. Amazon Redshift supports SSL, so SSL is used when you set sslmode to prefer. + +- **require**: SSL is required. + +- **verify-ca**: SSL must be used and the server certificate must be verified. + +- **verify-full**: SSL must be used. The server certificate must be verified and the server hostname must match the hostname attribute on the certificate. + +For more information, you can visit [Redshift SSL documentation](https://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html) ### 7. Schedule the Ingestion and Deploy Scheduling can be set up at an hourly, daily, or weekly cadence. The diff --git a/openmetadata-docs/content/openmetadata/connectors/database/trino/cli.md b/openmetadata-docs/content/openmetadata/connectors/database/trino/cli.md index 7033d2075c8..0eed0f4a1d0 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/trino/cli.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/trino/cli.md @@ -463,6 +463,34 @@ metadata profile -c ``` Note how instead of running `ingest`, we are using the `profile` command to select the Profiler workflow. +## SSL Configuration +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under **connectionArguments** which is placed in source. + +```yaml +--- +source: + type: trino + serviceName: + serviceConnection: + config: + type: Trino + hostPort: + username: + catalog: + ... + ... + connectionArguments: + verify: +``` + + +### SSL Modes +There are couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows: +- **false**: In order to disable SSL verification, set the `verify` parameter to `False`. +- **\**: To use self-signed certificates, specify a path to the certificate in `verify` parameter. +More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification). + + ## DBT Integration diff --git a/openmetadata-docs/content/openmetadata/connectors/database/trino/index.md b/openmetadata-docs/content/openmetadata/connectors/database/trino/index.md index 0541af380a2..f1e580ea2f2 100644 --- a/openmetadata-docs/content/openmetadata/connectors/database/trino/index.md +++ b/openmetadata-docs/content/openmetadata/connectors/database/trino/index.md @@ -169,6 +169,17 @@ caption="Configure Metadata Ingestion Page" - **Mark Deleted Tables (toggle)**: Set the Mark Deleted Tables toggle to flag tables as soft-deleted if they are not present anymore in the source system. - **Mark Deleted Tables from Filter Only (toggle)**: Set the Mark Deleted Tables from Filter Only toggle to flag tables as soft-deleted if they are not present anymore within the filtered schema or database only. This flag is useful when you have more than one ingestion pipelines. For example if you have a schema + +#### **SSL Configuration** +In order to integrate SSL in the Metadata Ingestion Config, the user will have to add the SSL config under **connectionArguments** which is placed in source. + +##### **SSL Modes** +There are couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows: +- **false**: In order to disable SSL verification, set the `verify` parameter to `False`. +- **\**: To use self-signed certificates, specify a path to the certificate in `verify` parameter. +More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification). + + ### 7. Schedule the Ingestion and Deploy Scheduling can be set up at an hourly, daily, or weekly cadence. The