Generate Docker Image For Mesosphere Kafka Client

Have you ever tried to access Kafka running on mesos on top of DCOS, and figure out that you end up with no latest Kafka client image in the docker hub?

I have uploaded a new image with the Latest Kafka Stable Version 2.0.0, and one can get it easily –

docker pull piyushdocker/kafka-client-2.0.0-image

If you want to create your own image with any other required version then in this blog, you can learn how to create a Docker image for Kafka client so that is can be used with Kafka installed on DCOS.

You can find one image that works with Kafka version 0.9.0.1 – https://hub.docker.com/r/mesosphere/kafka-client/

Now, this is a stale Kafka client image, and you will not get all the new features available in the new Kafka version.we will learn how to create such images and push them on Docker Repo.

we will create one Kafka client image that works with the latest Stable Kafka version i.e 2.0.0, where you can use new features of the latest Kafka like fetching from particular offsets and so on.

So let’s begin –

Pre-requisites –  Docker should be installed on your machine, if not please refer this link

Step 1) Create a Docker file 

touch Dockerfile

vim Dockerfile

Now add the below content in the docker file –

FROM java:openjdk-8-jre

ENV SCALA_VERSION 2.12
ENV KAFKA_VERSION 2.0.0

RUN curl http://apache.mirrors.spacedump.net/kafka/"$KAFKA_VERSION"/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz | tar xvz --strip-components=1

WORKDIR /bin

Your Docker File is ready now let’s create Docker Image from it.

Step 2) Create your Docker Image 

docker build -t kafka-client-2.0.0-image .

The output should be something like this  –

piyush@piyush:~/workstation/KafkaClientDockerImage$ docker build -t kafka-client-2.0.0-image .
Sending build context to Docker daemon 2.048kB
Step 1/5 : FROM java:openjdk-8-jre
---> e44d62cf8862
Step 2/5 : ENV SCALA_VERSION 2.12
---> Using cache
---> 003c406828dd
Step 3/5 : ENV KAFKA_VERSION 2.0.0
---> Using cache
---> d064dfcac078
Step 4/5 : RUN curl http://apache.mirrors.spacedump.net/kafka/"$KAFKA_VERSION"/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz | tar xvz --strip-components=1
---> Running in f1be59903352

kafka_2.12-2.0.0/libs/kafka-streams-examples-2.0.0.jar
Removing intermediate container f1be59903352
---> 1c6eda990a0d
Step 5/5 : WORKDIR /bin
---> Running in 8345901371b7
Removing intermediate container 8345901371b7
---> 6589f721538b
Successfully built 6589f721538b
Successfully tagged kafka-client-2.0.0-image:latest

check the image being created using the command –

docker images

output - 

REPOSITORY                    TAG    IMAGE ID         CREATED         SIZE
kafka-client-2.0.0-image latest 6589f721538b 2 minutes ago 364MB

Step 3) Push your image to Docker Repo – 

Tag your image -

docker tag 6589f721538b piyushdocker/kafka-client-2.0.0-image:kafka-mesos-client

docker images
REPOSITORY                     TAG    IMAGE ID        CREATED     SIZE
kafka-client-2.0.0-image latest 6589f721538b 2 hours ago 364MB
piyushdocker/kafka-client-2.0.0-image kafka-mesos-client 6589f721538b 2 hours ago 364MB

push it - 

docker push piyushdocker/kafka-client-2.0.0-image:latest

Step 4) How to use it –

If you have followed the above steps successfully then you have your image in your local, and you can directly run it via –

~/workstation/KafkaClientDockerImage$ sudo docker run -it 6589f721538b /bin/bash
root@91148092d745:/bin# ./kafka-console-consumer.sh --help
help is not a recognized option
Option Description
------ -----------
--bootstrap-server
--consumer-property properties in the form key=value to
the consumer.......

.... and so on.

Else don’t worry you can directly download the image and ran it as described in above step from –

docker pull piyushdocker/kafka-client-2.0.0-image