Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

74 lines
3.3 KiB
Markdown
Raw Permalink Normal View History

2024-06-18 15:53:06 +02:00
---
title: Nifi Connector | OpenMetadata Data Pipeline Integration
2024-06-18 15:53:06 +02:00
slug: /connectors/pipeline/nifi
---
{% connectorDetailsHeader
name="NiFi"
2024-06-18 15:53:06 +02:00
stage="PROD"
platform="OpenMetadata"
availableFeatures=["Pipelines", "Usage"]
2024-06-18 15:53:06 +02:00
unavailableFeatures=["Pipeline Status", "Owners", "Tags", "Lineage"]
/ %}
In this section, we provide guides and references to use the NiFi connector.
2024-06-18 15:53:06 +02:00
Configure and schedule NiFi metadata workflows from the OpenMetadata UI:
2024-06-18 15:53:06 +02:00
- [Requirements](#requirements)
- [Metadata Ingestion](#metadata-ingestion)
- [Troubleshooting](/connectors/pipeline/nifi/troubleshooting)
2024-06-18 15:53:06 +02:00
{% partial file="/v1.7/connectors/ingestion-modes-tiles.md" variables={yamlPath: "/connectors/pipeline/nifi/yaml"} /%}
2024-06-18 15:53:06 +02:00
## Requirements
### Metadata
OpenMetadata supports 2 types of connection for the NiFi connector:
- **basic authentication**: use username/password to authenticate to NiFi.
2024-06-18 15:53:06 +02:00
- **client certificate authentication**: use CA, client certificate and client key files to authenticate.
The user should be able to send request to the NiFi API and access the `Resources` endpoint.
2024-06-18 15:53:06 +02:00
## Metadata Ingestion
{% partial
file="/v1.7/connectors/metadata-ingestion-ui.md"
2024-06-18 15:53:06 +02:00
variables={
connector: "Nifi",
selectServicePath: "/images/v1.7/connectors/nifi/select-service.png",
addNewServicePath: "/images/v1.7/connectors/nifi/add-new-service.png",
serviceConnectionPath: "/images/v1.7/connectors/nifi/service-connection.png",
2024-06-18 15:53:06 +02:00
}
/%}
{% stepsContainer %}
{% extraContent parentTagName="stepsContainer" %}
#### Connection Details
- **Host and Port**: Pipeline Service Management/UI URI. This should be specified as a string in the format 'hostname:port'.
- **NiFi Config**: OpenMetadata supports username/password or client certificate authentication.
2024-06-18 15:53:06 +02:00
1. Basic Authentication
- Username: Username to connect to NiFi. This user should be able to send request to the Nifi API and access the `Resources` endpoint.
- Password: Password to connect to NiFi.
2024-06-18 15:53:06 +02:00
- Verify SSL: Whether SSL verification should be perform when authenticating.
2. Client Certificate Authentication
- Certificate Authority Path: Path to the certificate authority (CA) file. This is the certificate used to store and issue your digital certificate. This is an optional parameter. If omitted SSL verification will be skipped; this can present some sever security issue.
**important**: This file should be accessible from where the ingestion workflow is running. For example, if you are using OpenMetadata Ingestion Docker container, this file should be in this container.
- Client Certificate Path: Path to the certificate client file.
**important**: This file should be accessible from where the ingestion workflow is running. For example, if you are using OpenMetadata Ingestion Docker container, this file should be in this container.
- Client Key Path: Path to the client key file.
**important**: This file should be accessible from where the ingestion workflow is running. For example, if you are using OpenMetadata Ingestion Docker container, this file should be in this container.
{% /extraContent %}
{% partial file="/v1.7/connectors/test-connection.md" /%}
2024-06-18 15:53:06 +02:00
{% partial file="/v1.7/connectors/pipeline/configure-ingestion.md" /%}
2024-06-18 15:53:06 +02:00
{% partial file="/v1.7/connectors/ingestion-schedule-and-deploy.md" /%}
2024-06-18 15:53:06 +02:00
{% /stepsContainer %}