yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-06-27 05:03:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
datahub/docs/how/kafka-config.md
shakti-garg-saxo b80c14d543
Update kafka-config.md (#1763) 
Fix name of spring-kafka property to pass SASL_JAAS config
2020-07-31 04:10:21 -07:00

1.6 KiB
Raw Blame History

How to configure Kafka?

With the exception of KAFKA_BOOTSTRAP_SERVER and KAFKA_SCHEMAREGISTRY_URL, Kafka is configured via spring-boot, specifically with KafkaProperties. See Integration Properties prefixed with spring.kafka.

Below is an example of how SASL/GSSAPI properties can be configured via environment variables:

export KAFKA_BOOTSTRAP_SERVER=broker:29092
export KAFKA_SCHEMAREGISTRY_URL=http://schema-registry:8081
export SPRING_KAFKA_PROPERTIES_SASL_KERBEROS_SERVICE_NAME=kafka
export SPRING_KAFKA_PROPERTIES_SECURITY_PROTOCOL=SASL_PLAINTEXT
export SPRING_KAFKA_PROPERTIES_SASL_JAAS_CONFIG=com.sun.security.auth.module.Krb5LoginModule required principal='principal@REALM' useKeyTab=true storeKey=true keyTab='/keytab';

These properties can be specified via application.properties or application.yml files, or as command line switches, or as environment variables. See Spring's Externalized Configuration to see how this works.

See Kafka Connect Security for more ways to connect.

DataHub components that connect to Kafka are currently:

  • mce-consumer-job
  • mae-consumer-job
  • gms