Security & SSL Setup in Confluent Kafka

Reading Time: 2 minutes

What is SSL ?

Secure Socket Layer (SSL) is a security protocol for the transport layer. In SSL Protocol data is divided into fragments. The fragments are compressed and encrypted Message Authentication Code (MAC) generated by algorithms like Secure Hash Protocol(SHA) and MD5(Message Digest) is appended. SSL is the predecessor of Transport Layer Security(TLS) . After encryption of data, finally, the SSL header is appended to the data.

By default, Confluent Kafka communicates with an unsecured plaintext protocol over 9092 ports.

Confluent Kafka security supports SSL security protocol in intra broker and client communications. Each broker authenticates other brokers and the clients. Brokers and the clients both authenticate each other (2-way authentication). One can also use SSL with SASL security, hit the reference section for Confluent Kafka sasl.

Note: Enabling SSL (TLS) in Confluent Kafka security would override the zero-copy optimization in Kafka consumers. In the Zero-copy mechanism, data is copied into page cache only once and reused on each consumption instead of being stored in memory and copied out to user-space every time it is read. This allows messages to be consumed at a rate that approaches the limit of the network connection.

For a successful handshake:

  • Each broker should have its own private-key/certificate pair, and the client uses the certificate to authenticate the broker.
  • Each client should have a private-key/certificate pair if client authentication is enabled*, and the broker uses the certificate to authenticate the client.

*Client authentication is optional but recommended. The tradeoff of having it is performance implications but not having it can allow non-authenticated brokers (impersonation) to communicate to clients.

Steps to enable SSL protocol in Kafka Brokers to enable Confluent Kafka Security:

Note: Below steps do not include steps for creating SSL certificates & keys, refer to this link.

Update the with below corresponding configurations.
Set the below config in for enabling inter-broker SSL communication.
Now update the listeners and advertised listeners (If different) in the Generally we use 9093 port for SSL as 9092 is default for plaintext.
For continuing plaintext protocol as well along with SSL use below.

Once the SSL port works fine, we can remove/disable the 9092 plaintext port.

  •  For enabling client authentication (2 way authentication) the below configuration must be enabled.


  • On the client side, we are required to have both certificates and keys. Below configuration is to be mentioned in all clients communicating with SSL enabled Kafka cluster.
These configs can be put in a file and reused with regular client application files.

Note: ssl.truststore.password  value is optional in both broker and client configuration but is strongly recommended for integrity checking.

Examples of Kafka clients with SSL meta configuration in Confluent Kafka Security are below.

  • ${home}/bin/kafka-console-producer –broker-list kafkabroker1:9093 –topic test_topic –producer.config
  • ${home}/bin/kafka-console-consumer –bootstrap-server kafkabroker1:9093 –topic test_topic –consumer.config –from-beginning
  • For JVM based producers & consumers below export can be included in start scripts, in command line or set as environment variable for dedicated machines.
export KAFKA_OPTS=<password;password>


Written by 

Munander is a Software Consultant in Knoldus Software LLP. He has done from IMS Engineering college, Ghaziabad. He has decent knowledge of C,C++,Java,Angular and Lagom. He always tries to explore new technologies. His hobbies include playing cricket and adventure.