yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-11-29 00:53:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

GitBook: [#62] Minor edits - Connectors

Browse Source
This commit is contained in:
Shilpa V 2022-01-04 07:28:23 +00:00 committed by gitbook-bot
parent 6639e28208
commit a94f6b45ec
No known key found for this signature in database
GPG Key ID: 07D2180C7B12D0FF
9 changed files with 33 additions and 21 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/connectors/glue-catalog.md
View File

@ -54,7 +54,7 @@ In a later step, you will install the Python module for this connector and its d
#### **1.1 Create a directory for openmetadata**
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata
@ -243,7 +243,9 @@ Use `source.config.table_filter_pattern.includes` to include all tables with nam
See the documentation for the[ Python re module](https://docs.python.org/3/library/re.html) for information on how to construct regular expressions.
{% hint style="info" %}
You may use either `excludes` or `includes` but not both in `table_filter_pattern`.
{% endhint %}
#### **schema\_filter\_pattern (optional)**

4
docs/connectors/mariadb.md
View File

@ -54,7 +54,7 @@ In a later step, you will install the Python module for this connector and its d
#### **1.1 Create a directory for openmetadata**
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata
@ -288,7 +288,9 @@ Use `source.config.table_filter_pattern.includes` to include all tables with nam
See the documentation for the[ Python re module](https://docs.python.org/3/library/re.html) for information on how to construct regular expressions.
{% hint style="info" %}
You may use either `excludes` or `includes` but not both in `table_filter_pattern`.
{% endhint %}
#### **schema\_filter\_pattern (optional)**

4
docs/openmetadata/connectors/athena.md
View File

@ -54,7 +54,7 @@ In a later step, you will install the Python module for this connector and its d
#### **1.1 Create a directory for openmetadata**
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata
@ -290,7 +290,9 @@ Use `source.config.table_filter_pattern.includes` to include all tables with nam
See the documentation for the[ Python re module](https://docs.python.org/3/library/re.html) for information on how to construct regular expressions.
{% hint style="info" %}
You may use either `excludes` or `includes` but not both in `table_filter_pattern`.
{% endhint %}
#### **schema\_filter\_pattern (optional)**

6
docs/openmetadata/connectors/bigquery-usage.md
View File

@ -54,7 +54,7 @@ In a later step, you will install the Python module for this connector and its d
#### **1.1 Create a directory for openmetadata**
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata
@ -212,7 +212,7 @@ OpenMetadata uniquely identifies services by their `service_name`. Edit the valu
#### duration
The `duration` is added in days.
Use duration to specify the window of time in which the profiler should capture usage data. Values should be integers and represent the number of days for which to capture usage information. For example, if you specify 2 as the value for duration, the data profiler will capture usage information for the 48 hours prior to when the ingestion workflow is run.
```javascript
"duration": 2
@ -326,7 +326,9 @@ Use `source.config.table_filter_pattern.includes` to include all tables with nam
See the documentation for the[ Python re module](https://docs.python.org/3/library/re.html) for information on how to construct regular expressions.
{% hint style="info" %}
You may use either `excludes` or `includes` but not both in `table_filter_pattern`.
{% endhint %}
#### **schema\_filter\_pattern (optional)**

4
docs/openmetadata/connectors/bigquery.md
View File

@ -54,7 +54,7 @@ In a later step, you will install the Python module for this connector and its d
#### **1.1 Create a directory for openmetadata**
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata
@ -321,7 +321,9 @@ Use `source.config.table_filter_pattern.includes` to include all tables with nam
See the documentation for the[ Python re module](https://docs.python.org/3/library/re.html) for information on how to construct regular expressions.
{% hint style="info" %}
You may use either `excludes` or `includes` but not both in `table_filter_pattern`.
{% endhint %}
#### **schema\_filter\_pattern (optional)**

4
docs/openmetadata/connectors/elastic-search.md
View File

@ -54,7 +54,7 @@ In a later step, you will install the Python module for this connector and its d
#### **1.1 Create a directory for openmetadata**
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata
@ -283,7 +283,9 @@ Use `source.config.table_filter_pattern.includes` to include all tables with nam
See the documentation for the[ Python re module](https://docs.python.org/3/library/re.html) for information on how to construct regular expressions.
{% hint style="info" %}
You may use either `excludes` or `includes` but not both in `table_filter_pattern`.
{% endhint %}
#### **schema\_filter\_pattern (optional)**

24
docs/openmetadata/connectors/mysql.md
View File

@ -48,13 +48,13 @@ Here’s an overview of the steps in this procedure. Please follow the steps rel
### 1. Prepare a Python virtual environment
In this step, we'll create a Python virtual environment. Using a virtual environment enables us to avoid conflicts with other Python installations and packages on your host system. 
In this step, we'll create a Python virtual environment. Using a virtual environment enables us to avoid conflicts with other Python installations and packages on your host system.
In a later step, you will install the Python module for this connector and its dependencies in this virtual environment.
#### 1.1 Create a directory for openmetadata
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata
@ -70,7 +70,7 @@ python3 -m venv env
#### 1.3 Activate the virtual environment
Run the following command to activate the virtual environment. 
Run the following command to activate the virtual environment.
```bash
source env/bin/activate
@ -96,12 +96,12 @@ pip3 install 'openmetadata-ingestion[mysql]'
### 3. Create a configuration file using template JSON
Create a new file called `mysql.json` in the current directory. Note that the current directory should be the `openmetadata` directory you created in Step 1. 
Create a new file called `mysql.json` in the current directory. Note that the current directory should be the `openmetadata` directory you created in Step 1.
Copy and paste the configuration template below into the `mysql.json` file you created.
{% hint style="info" %}
Note: The `source.config` field in the configuration JSON will include the majority of the settings for your connector. In the steps below we describe how to customize the key-value pairs in the `source.config` field to meet your needs. 
Note: The `source.config` field in the configuration JSON will include the majority of the settings for your connector. In the steps below we describe how to customize the key-value pairs in the `source.config` field to meet your needs.
{% endhint %}
{% code title="mysql.json" %}
@ -139,13 +139,13 @@ Note: The `source.config` field in the configuration JSON will include the major
```
{% endcode %}
### 4. Configure service settings 
### 4. Configure service settings
In this step we will configure the MySQL service settings required for this connector. Please follow the instructions below to ensure that you've configured the connector to read from your MySQL service as desired.
#### host\_port
Edit the value for `source.config.host_port` in `mysql.json` for your MySQL deployment. Use the `host:port` format illustrated in the example below. 
Edit the value for `source.config.host_port` in `mysql.json` for your MySQL deployment. Use the `host:port` format illustrated in the example below.
```json
"host_port": "hostname.domain.com:5439"
@ -183,7 +183,7 @@ OpenMetadata uniquely identifies services by their `service_name`. Edit the valu
#### database (optional)
If you want to limit metadata ingestion to a single database, include the `source.config.database` field in your configuration file. If this field is not included, the connector will ingest metadata from all databases that the specified user is authorized to read. 
If you want to limit metadata ingestion to a single database, include the `source.config.database` field in your configuration file. If this field is not included, the connector will ingest metadata from all databases that the specified user is authorized to read.
To specify a single database to ingest metadata from, provide the name of the database as the value for the `source.config.database` key as illustrated in the example below.
@ -193,7 +193,7 @@ To specify a single database to ingest metadata from, provide the name of the da
### 5. Enable/disable the data profiler
The data profiler ingests usage information for tables. This enables you to assess the frequency of use, reliability, and other details. 
The data profiler ingests usage information for tables. This enables you to assess the frequency of use, reliability, and other details.
#### data\_profiler\_enabled
@ -205,7 +205,7 @@ You may disable the data profiler by setting the value for the key `source.confi
"data_profiler_enabled": "false"
```
If you want to enable the data profiler, update your configuration file as follows. 
If you want to enable the data profiler, update your configuration file as follows.
```json
"data_profiler_enabled": "true"
@ -269,7 +269,7 @@ Note: `source.config.include_tables` is set to `true` by default.
#### table\_filter\_pattern (optional)
Use `source.config.table_filter_pattern` to select tables for metadata ingestion by name. 
Use `source.config.table_filter_pattern` to select tables for metadata ingestion by name.
Use `source.config.table_filter_pattern.excludes` to exclude all tables with names matching one or more of the supplied regular expressions. All other tables will be included. See below for an example. This example is also included in the configuration template provided.
@ -411,7 +411,7 @@ Then re-run the install command in [Step 2](mysql.md#install-from-pypi-or-source
### requests.exceptions.ConnectionError
If you encounter the following error when attempting to run the ingestion workflow in Step 12, this is probably because there is no OpenMetadata server running at http://localhost:8585. 
If you encounter the following error when attempting to run the ingestion workflow in Step 12, this is probably because there is no OpenMetadata server running at http://localhost:8585.
```
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8585):

2
docs/openmetadata/connectors/redshift.md
View File

@ -62,7 +62,7 @@ In a later step, you will install the Python module for this connector and its d
#### 1.1 Create a directory for openmetadata
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata

2
docs/openmetadata/connectors/snowflake.md
View File

@ -54,7 +54,7 @@ In a later step, you will install the Python module for this connector and its d
#### 1.1 Create a directory for openmetadata
Throughout the docs, we use a consistent directory structure, OpenMetadata server, and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
Throughout the docs, we use a consistent directory structure for OpenMetadata services and connector installation. If you have not already done so by following another guide, please create an openmetadata directory now and change into that directory in your command line environment.
```
mkdir openmetadata; cd openmetadata