* DOCS - Prepare 1.7 Release and 1.8 SNAPSHOT * DOCS - Prepare 1.7 Release and 1.8 SNAPSHOT
3.6 KiB
| title | slug |
|---|---|
| Run the Atlas Connector Externally | /connectors/metadata/atlas/yaml |
{% connectorDetailsHeader name="Atlas" stage="PROD" platform="OpenMetadata" availableFeatures=["Metadata"] unavailableFeatures=[] / %}
In this section, we provide guides and references to use the Atlas connector.
Configure and schedule Atlas metadata and profiler workflows from the OpenMetadata UI:
{% partial file="/v1.7/connectors/external-ingestion-deployment.md" /%}
Requirements
Before this, you must ingest the database / messaging service you want to get metadata for. For more details click here
Python Requirements
{% partial file="/v1.7/connectors/python-requirements.md" /%}
To run the Atlas ingestion, you will need to install:
pip3 install "openmetadata-ingestion[atlas]"
Metadata Ingestion
All connectors are defined as JSON Schemas. Here you can find the structure to create a connection to Atlas.
In order to create and run a Metadata Ingestion workflow, we will follow the steps to create a YAML configuration able to connect to the source, process the Entities if needed, and reach the OpenMetadata server.
The workflow is modeled around the following JSON Schema
1. Define the YAML Config
{% codePreview %}
{% codeInfoContainer %}
Source Configuration - Service Connection
{% codeInfo srNumber=12 %}
hostPort: Atlas Host of the data source.
{% /codeInfo %}
{% codeInfo srNumber=13 %}
username: Username to connect to the Atlas. This user should have privileges to read all the metadata in Atlas.
{% /codeInfo %}
{% codeInfo srNumber=14 %}
password: Password to connect to the Atlas.
{% /codeInfo %}
{% codeInfo srNumber=15 %}
databaseServiceName: source database of the data source(Database service that you created from UI. example- local_hive).
{% /codeInfo %}
{% codeInfo srNumber=16 %}
messagingServiceName: messaging service source of the data source.
{% /codeInfo %}
{% codeInfo srNumber=17 %}
entity_type: Name of the entity type in Atlas.
{% /codeInfo %}
Sink Configuration
{% codeInfo srNumber=18 %}
To send the metadata to OpenMetadata, it needs to be specified as type: metadata-rest.
{% /codeInfo %}
{% partial file="/v1.7/connectors/yaml/workflow-config-def.md" /%} {% /codeInfoContainer %}
{% codeBlock fileName="filename.yaml" %}
source:
type: Atlas
serviceName: local_atlas
serviceConnection:
config:
type: Atlas
hostPort: http://localhost:10000
username: username
password: password
databaseServiceName: ["local_hive"] # create database service and messaging service and pass `service name` here
messagingServiceName: []
entity_type: Table
sourceConfig:
config:
type: DatabaseMetadata
sink:
type: metadata-rest
config: {}
{% partial file="/v1.7/connectors/yaml/workflow-config.md" /%}
{% /codeBlock %}
{% /codePreview %}
{% partial file="/v1.7/connectors/yaml/ingestion-cli.md" /%}