Getting started with Apache Kafka and Quarkus

Table of contents

Reading Time: 4 minutes

Overview

The ideal framework for continuous delivery and greater resilience is provided by micro-service-oriented architecture. In addition to boosting developer efficiency and improving real-time scalability, they encourage quicker innovation to respond to shifting market conditions.

In order to build scalable architectures, we need event-driven and asynchronous integration between microservices. We have multiple options available for asynchronous integration. Kafka is the most popular platform offering characteristics like message retention, fault tolerance, and consumer groups.

In this article, you will learn how to get started with Apache Kafka and Quarkus framework. Here, we will be using reactive messaging to build the event-driven microservices, but you can also use Kafka APIs as well for your implementation.

Step 1: Generate Your Project

Head over to https://code.quarkus.io, and fill up the required project details like group id and artifact id. Select the following dependencies from the list:

SmallRye reactive messaging
Resteasy reactive jackson
Hibernate validator

Click on the Generate your application, download the project as a zip file, unzip it, and load it in your favorite IDE. The project structure will look like this in the IDE.

Step 2: Create the model object

In Kafka, we produce and consume records. A record contains a key and a value. We will create an Employee object, which will be produced and consumed.

Step 3: Create the producer

Add a new class EmployeeProducer as per the below code

Let’s decode this class. We have added annotation @ApplicationScoped on the class which means that an instance will be created only once for the duration of the whole application. Inside the class, we have injected an Emitter who is responsible for sending messages to a channel. The emitter is attached to the movies-out channel and will send messages to Kafka. We have a method createEmployee which will send Record objects of Employee as key-value pair.

Step 4: Create a consumer

Here, we will be creating a consumer EmployeeConsumer for the event we published above.

We are using the @Incoming annotation to ask Quarkus to call the receive method for every record received from channel employee-in.

Step 5: Configuring Kafka and channels

In reactive messaging, we send and receive messages from channels. These channels are mapped to the underlying messaging platform using configuration. In our application, we’ve used employee-in and employee-out channel for messaging. We will configure these channels in the application. properties file.

Since we have an object to send as the value we have created a custom serializer and deserializer to send and receive the message.

Step 6: REST endpoint to send the message

To call our producer and send the Employee object as a message, let’s create an endpoint that will call the createEmployee method.

Step 7: Setup Kafka

Create a docker-compose.yaml file to the root folder and add the following content

Step 8: Run the application

In your terminal, run the following command

docker-compose up -d

Once, the zookeeper and Kafka are up and running. Run the following in the terminal

./mvnw quarkus:dev

The application will start in dev mode. In another terminal, execute some HTTP request

curl -X POST \

  http://localhost:8080/employee \

  -H 'cache-control: no-cache' \

  -H 'content-type: application/json' \

  -H 'postman-token: 00fb55a5-7c90-af25-b1f4-9aa74a445405' \

  -d '{
	"firstName":"Vimal",

	"lastName":"Kumar",

	"empCode":"1823"
}'

Once the message is received you will be able to see the below message

Summary

You have seen how can we use reactive messaging with Quarkus and Kafka. You implemented a simple application that produces and consumes messages. The application code can be found here.