0 comments on “Stream a file to AWS S3 using Akka Streams (via Alpakka) in Play Framework”

Stream a file to AWS S3 using Akka Streams (via Alpakka) in Play Framework


In this blog post we'll see how a file can be streamed from a client (eg: browser) to Amazon S3 (AWS S3) using Alpakka's AWS S3 connector. Aplakka provides various Akka Stream connectors, integration patterns and data transformations for integration use…

1 comment on “Kafka And Spark Streams: The happily ever after !!”

Kafka And Spark Streams: The happily ever after !!


Hi everyone, Today we are going to understand a bit about using the spark streaming to transform and transport data between Kafka topics. The demand for stream processing is increasing every day. The reason is that often, processing big volumes…

2 comments on “Streaming data from PostgreSQL using Akka Streams and Slick in Play Framework”

Streaming data from PostgreSQL using Akka Streams and Slick in Play Framework


In this blog post I'll try to explain the process wherein you can stream data directly from PostgreSQL database using Scala Slick (which is Scala's database access/query library) and Akka Streams (which is an implementation of Reactive Streams specification on…

1 comment on “Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-II”

Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-II


Hello everyone, Knoldus organized a session on 25th November 2017. The topic was “Guaranteed No Stress Baby Steps Using Akka Streams Part-II”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video…

1 comment on “Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-I”

Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-I


Hello everyone, Knoldus organized a session on 28th October 2017. The topic was “Guaranteed No Stress Baby Steps Using Akka Streams Part-I”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video…

1 comment on “KnolX: Learning Kafka Streams with Scala”

KnolX: Learning Kafka Streams with Scala


Hello everyone, Knoldus organized a session on 22nd September 2017. The topic was “Learning Kafka Streams with Scala”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video of the session. Slides:…

2 comments on “Having Issue How To Order Streamed Dataframe ?”

Having Issue How To Order Streamed Dataframe ?


A few days ago, i have to perform aggregation on streaming dataframe. And the moment, i apply groupBy for aggregation, data gets shuffled. Now the situation arises how to maintain order? Yes, i can use orderBy with streaming dataframe using…

4 comments on “Unit Testing Of Kafka”

Unit Testing Of Kafka


Apache Kafka is a distributed publish-subscribe messaging system and a robust queue that can handle a high volume of data and enables you to pass messages from one end-point to another. Generally, data is published to topic via Producer API…

0 comments on “Meetup: Stream processing using Kafka”

Meetup: Stream processing using Kafka


Knoldus organized a Meetup on Friday, 7th April 2017 at 4:00 PM which was presented by Himani Arora and me(Prabhat Kashyap). Topics which were covered in this meetup: What is Stream processing Advantages of stream processing Type of stream processing What are KStreams Use cases…

4 comments on “Integrating Kafka With Spark Structure Streaming”

Integrating Kafka With Spark Structure Streaming


Kafka is a messaging broker system which facilitates the passing of messages between producer and consumer whereas Spark Structure streaming consumes static and streaming data from various sources like kafka, flume, twitter or any other socket which can be processed…