0 comments on “Interactive Queries in Apache Kafka”

Interactive Queries in Apache Kafka


    Apache Kafka v0.10 introduced a new feature Kafka Streams API - a client library which can be used for building applications and microservices, where the input and output data can be stored in Kafka clusters. Kafka Streams provides…

3 comments on “Structured Streaming: What is it?”

Structured Streaming: What is it?


With the advent of streaming frameworks like Spark Streaming, Flink, Storm etc. developers stopped worrying about issues related to a streaming application, like - Fault Tolerance, i.e., zero data loss, Real-time processing of data, etc. and started focussing only on solving business…

1 comment on “A Beginner’s Guide to Deploying a Lagom Service Without ConductR”

A Beginner’s Guide to Deploying a Lagom Service Without ConductR


How to deploy a Lagom Service without ConductR? This question has been asked and answered by many, on different forums. For example, take a look at this question on StackOverflow - Lagom without ConductR? Here the user is trying to know whether…

1 comment on “Kafka And Spark Streams: The happily ever after !!”

Kafka And Spark Streams: The happily ever after !!


Hi everyone, Today we are going to understand a bit about using the spark streaming to transform and transport data between Kafka topics. The demand for stream processing is increasing every day. The reason is that often, processing big volumes…

2 comments on “Error Registering Avro Schema | Multiple Schemas In One Topic”

Error Registering Avro Schema | Multiple Schemas In One Topic


org.apache.kafka.common.errors.SerializationException: Error registering Avro schema: {"type":"record","name":"schema1","namespace":"test","fields":[{"name":"Name","type":"string"},{"name":"Age","type":"int"},{"name":"Location","type":"string"}]} Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Schema being registered is incompatible with an earlier schema; error code: 409 at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:170) at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:188) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:245) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:237) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:232) at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:59) at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:91) at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:72) at io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:158) at…

2 comments on “Assimilation of Spark Streaming With Kafka”

Assimilation of Spark Streaming With Kafka


As we know Spark is used at a wide range of organizations to process large datasets. It seems like spark becoming main stream. In this blog we will talk about Integration of Kafka with Spark Streaming. So, lets get started. How Kafka…

1 comment on “KnolX: Learning Kafka Streams with Scala”

KnolX: Learning Kafka Streams with Scala


Hello everyone, Knoldus organized a session on 22nd September 2017. The topic was “Learning Kafka Streams with Scala”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video of the session. Slides:…

4 comments on “Case Study to understand Kafka Consumer and its offsets”

Case Study to understand Kafka Consumer and its offsets


In this blog post, we will discuss mainly Kafka Consumer and its Offsets. We will understand this using a case study implemented in Scala. This blog post assumes that you are aware of basic Kafka terminology. CASE STUDY: The Producer…

3 comments on “Join Semantics in Kafka Streams”

Join Semantics in Kafka Streams


Introduction to core concepts:   Apache Kafka is a distributed streaming platform which enables you to publish and subscribe to a stream of records also letting you process this stream of records as it occurs. Kafka Streams is a client…

1 comment on “A Java Lagom service which only consumes from Kafka topic (Subscriber only service)”

A Java Lagom service which only consumes from Kafka topic (Subscriber only service)


Subscriber only service means an application which only consumes, does not produce. We have generally seen the applications which both produces and consumes data from a Kafka topic but sometimes we need to write an application which only consumes data…