1 comment on “Structured Streaming: What is it?”

Structured Streaming: What is it?


With the advent of streaming frameworks like Spark Streaming, Flink, Storm etc. developers stopped worrying about issues related to a streaming application, like - Fault Tolerance, i.e., zero data loss, Real-time processing of data, etc. and started focussing only on solving business…

1 comment on “Kafka And Spark Streams: The happily ever after !!”

Kafka And Spark Streams: The happily ever after !!


Hi everyone, Today we are going to understand a bit about using the spark streaming to transform and transport data between Kafka topics. The demand for stream processing is increasing every day. The reason is that often, processing big volumes…

2 comments on “KnolX: Understanding Spark Structured Streaming”

KnolX: Understanding Spark Structured Streaming


Hello everyone, Knoldus organized a session on 05th January 2018. The topic was “Understanding Spark Structured Streaming”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video of the session. Slides: Video: If…

1 comment on “Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-II”

Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-II


Hello everyone, Knoldus organized a session on 25th November 2017. The topic was “Guaranteed No Stress Baby Steps Using Akka Streams Part-II”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video…

1 comment on “Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-I”

Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-I


Hello everyone, Knoldus organized a session on 28th October 2017. The topic was “Guaranteed No Stress Baby Steps Using Akka Streams Part-I”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video…

1 comment on “Rules while working with stream in Java 8”

Rules while working with stream in Java 8


First, let’s have a basic understanding of stream. Then we will have a look at the side effects that can occur while working with streams. Stream represents a sequence of objects from a source, which supports aggregate operations. One thing…

1 comment on “One-way & two-way streaming in a Lagom application”

One-way & two-way streaming in a Lagom application


Now a days streaming word is a buzz word and you should have heard many types of streaming till now i.e. kafka streaming, spark streaming etc etc. But in this blog we will see a new type of streaming i.e…

2 comments on “Apache Storm: Architecture”

Apache Storm: Architecture


Apache Storm is a distributed realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing the realtime computation. Storm is simple, can be used…

4 comments on “Case Study to understand Kafka Consumer and its offsets”

Case Study to understand Kafka Consumer and its offsets


In this blog post, we will discuss mainly Kafka Consumer and its Offsets. We will understand this using a case study implemented in Scala. This blog post assumes that you are aware of basic Kafka terminology. CASE STUDY: The Producer…

5 comments on “What’s new in Apache Spark 2.2”

What’s new in Apache Spark 2.2


Apache recently released a newer version of Spark i.e Apache Spark2.2. The new version comes with new improvements as well as the addition of new functionalities. The major addition to this release is Structured Streaming. It has been marked as production…