mirror of
https://github.com/open-metadata/OpenMetadata.git
synced 2026-01-07 13:07:22 +00:00
5d6f385a75
* Added Filter Params for table and Schema * Bigquery Doc changes * Doc Changes for databases * Filter Pattern Changes * Table Filter Pattern Example Changes * Filter Pattern Example Changes
2.2 KiB
2.2 KiB
| description |
|---|
| This guide will help install Athena connector and run manually |
Athena
{% hint style="info" %} Prerequisites
OpenMetadata is built using Java, DropWizard, Jetty, and MySQL.
- Python 3.7 or above {% endhint %}
Install from PyPI
{% tabs %} {% tab title="Install Using PyPI" %}
pip install 'openmetadata-ingestion[athena]'
{% endtab %} {% endtabs %}
Configuration
{% code title="athena.json" %}
{
"source": {
"type": "athena",
"config": {
"host_port":"host_port",
"username": "openmetadata_user",
"password": "openmetadata_password",
"service_name": "local_athena",
"service_type": "Athena"
}
},
...
{% endcode %}
- username - pass the Athena username. We recommend creating a user with read-only permissions to all the databases in your Athena installation
- password - password for the username
- service_name - Service Name for this Athena cluster. If you added the Athena cluster through OpenMetadata UI, make sure the service name matches the same.
- table_filter_pattern - It contains includes, excludes options to choose which pattern of tables you want to ingest into OpenMetadata.
- schema_filter_pattern - It contains includes, excludes options to choose which pattern of schemas you want to ingest into OpenMetadata.
Publish to OpenMetadata
Below is the configuration to publish Athena data into the OpenMetadata service.
Add metadata-rest sink along with metadata-server config
{% code title="athena.json" %}
{
"source": {
"type": "athena",
"config": {
"host_port":"host_port",
"username": "openmetadata_user",
"password": "openmetadata_password",
"service_name": "local_athena",
"service_type": "Athena"
}
},
"processor": {
"type": "pii",
"config": {
"api_endpoint": "http://localhost:8585/api"
}
},
"sink": {
"type": "metadata-rest",
"config": {
}
},
"metadata_server": {
"type": "metadata-server",
"config": {
"api_endpoint": "http://localhost:8585/api",
"auth_provider_type": "no-auth"
}
}
}
{% endcode %}