Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

70 lines
2.8 KiB
Markdown
Raw Permalink Normal View History

2024-06-18 15:53:06 +02:00
---
title: Redpanda Connector | OpenMetadata Messaging Services
2024-06-18 15:53:06 +02:00
slug: /connectors/messaging/redpanda
---
{% connectorDetailsHeader
name="Redpanda"
stage="PROD"
platform="OpenMetadata"
availableFeatures=["Topics", "Sample Data"]
unavailableFeatures=[]
/ %}
In this section, we provide guides and references to use the Redpanda connector.
Configure and schedule Redpanda metadata and profiler workflows from the OpenMetadata UI:
- [Requirements](#requirements)
- [Metadata Ingestion](#metadata-ingestion)
- [Troubleshooting](/connectors/messaging/redpanda/troubleshooting)
2024-06-18 15:53:06 +02:00
{% partial file="/v1.7/connectors/ingestion-modes-tiles.md" variables={yamlPath: "/connectors/messaging/redpanda/yaml"} /%}
2024-06-18 15:53:06 +02:00
## Requirements
Connecting to Redpanda does not require any previous configuration.
The ingestion of the Kafka topics' schema is done separately by configuring the **Schema Registry URL**. However, only the **Bootstrap Servers** information is mandatory.
## Metadata Ingestion
{% partial
file="/v1.7/connectors/metadata-ingestion-ui.md"
2024-06-18 15:53:06 +02:00
variables={
connector: "Redpanda",
selectServicePath: "/images/v1.7/connectors/redpanda/select-service.png",
addNewServicePath: "/images/v1.7/connectors/redpanda/add-new-service.png",
serviceConnectionPath: "/images/v1.7/connectors/redpanda/service-connection.png",
2024-06-18 15:53:06 +02:00
}
/%}
{% stepsContainer %}
{% extraContent parentTagName="stepsContainer" %}
#### Connection Details
- **Bootstrap Servers**: List of brokers as comma separated values of broker `host` or `host:port`. Example: `host1:9092,host2:9092`
- **Schema Registry URL**: URL of the Schema Registry used to ingest the schemas of the topics.
- **SASL Username**: SASL username for use with the PLAIN and SASL-SCRAM mechanisms.
- **SASL Password**: SASL password for use with the PLAIN and SASL-SCRAM mechanisms.
- **SASL Mechanism**: SASL mechanism to use for authentication.
- **Basic Auth User Info**: Schema Registry Client HTTP credentials in the form of `username:password`. By default, user info is extracted from the URL if present.
- **Consumer Config**: The accepted additional values for the consumer configuration can be found in the following [link](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md).
- **Schema Registry Config**: The accepted additional values for the Schema Registry configuration can be found in the following [link](https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#schemaregistryclient).
2024-06-18 15:53:06 +02:00
{% note %}
To ingest the topic schema `Schema Registry URL` must be passed
{% /note %}
{% /extraContent %}
{% partial file="/v1.7/connectors/test-connection.md" /%}
2024-06-18 15:53:06 +02:00
{% partial file="/v1.7/connectors/messaging/configure-ingestion.md" /%}
2024-06-18 15:53:06 +02:00
{% partial file="/v1.7/connectors/ingestion-schedule-and-deploy.md" /%}
2024-06-18 15:53:06 +02:00
{% /stepsContainer %}