Kafka Kerberos Authentication

Table of contents

Reading Time: 2 minutes

In this article we will start looking into Kerberos authentication and will focus on the client-side configuration required to authenticate with clusters configured to use Kerberos.

Kafka supports four different communication protocols between Consumers, Producers, and Brokers. Each protocol considers different security aspects, while PLAINTEXT is the old insecure communication protocol.

PLAINTEXT (non-authenticated, non-encrypted)
SSL (SSL authentication, encrypted)
PLAINTEXT+SASL (authentication, non-encrypted)
SSL+SASL (encrypted authentication, encrypted transport)

Kerberos is by far the most common option we see being used in the field to secure Kafka clusters. It enables users to use their corporate identities, stored in services like Active Directory, RedHat IPA, and FreeIPA, which simplifies identity management. A kerberized Kafka cluster also makes it easier to integrate with other services in a Big Data ecosystem, which typically use Kerberos for strong authentication.

The basic Kafka client properties that must be set to configure the Kafka client to authenticate via Kerberos are shown below:

# Uses SASL/GSSAPI over a TLS encrypted connection
security.protocol=SASL_SSL
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
# TLS truststore
ssl.truststore.location=/opt/cloudera/security/jks/truststore.jks

JAAS configuration

The properties above, though, don’t provide the client with the credentials it needs to authenticate with the Kafka cluster. We need some more information.

When using Kerberos, we can provide the credentials to the client application in two ways. Either in the form of a valid Kerberos ticket, stored in a ticket cache, or as a keytab file, which the application can use to obtain a Kerberos ticket

The handling of the Kerberos credentials in a Kafka client is done by the Java Authentication and Authorization Service library. So we need to configure the client with the necessary information so that JAAS knows where to get the credentials from.

There are two ways to set those properties for the Kafka client:

Create a JAAS configuration file and set the Java system property java.security.auth.login.config to point to it; OR
Set the Kafka client property sasl.jaas.config with the JAAS configuration inline.

If you are using a JAAS configuration file you need to tell the Kafka Java client where to find it. This is done by setting the following Java property in the command line:

To use a keytab, use the following instead:

KafkaClient {
  com.sun.security.auth.module.Krb5LoginModule required
  useKeyTab=true
  keyTab="/etc/security/keytabs/alice.keytab"
  principal="alice@EXAMPLE.COM";

Sample configuration:

kafka:
    bootstrap-servers: ${}
    ssl:
      trust-store-type: JKS
    properties:
      security:
        protocol: ${KAFKA_SECURITY_PROTOCOL:SASL_SSL}
      sasl:
        mechanism: ${KAFKA_SASL_MECHANISM:GSSAPI}
        kerberos:
          service:
            name: kafka
        jaas:
          config: com.sun.security.auth.module.Krb5LoginModule required debug=true useKeyTab=true keyTab="/tmp/kafka.keytab" storeKey=true useTicketCache=false principal="${KERBEROS_PRINCIPAL}";

In this blog we have covered the configuration for kafka kerberos authentication. In the next blog we will look at how we can configure the kafka client to use kerberos authetication. Check out more blogs at Knoldus Blogs.