Comparison between different streaming engines

Reading Time: 5 minutes Distributed stream processing engines have been on the rise in the last few years, first Hadoop became popular as a batch processing engine, then focus shifted towards stream processing engines. Stream processing engines can make the job of processing data that comes in via a stream easier than ever before and by using clustering can enable processing data in larger sets in a timely manner. Continue Reading

Alpakka : Creating Data Pipelines

Reading Time: 4 minutes In my previous blogs I discussed about akka-streams , materialization and graphs in akka-stream . It was so easy to create sources, sinks and flows and connect them using methods and create runnable graphs. Now consider a situation where I’m getting data from Kafka and want to push it in Cassandra or Elasticsearch ? How to create such source or flow or sink that can Continue Reading

Akka-Streams: All About Graphs!

Reading Time: 4 minutes In my previous blogs, I discussed about the basics of akka-streams and materialization. Now let’s dig deeper into Graphs in Akka-Streams. Graphs Till now we know how to create a linear pipeline/linear graph. But in real life scenario we generally don’t have linear graphs to implement. The graphs can be complex. In Akka Streams computation graphs are written in a more graph-resembling DSL. It aims Continue Reading

Akka-Streams: All About Materialization!

Reading Time: 4 minutes In my previous blog, I discussed about the basics of akka-stream. Basically, it was an introduction to akka-stream to get you started. There I mentioned about materialization which is a really important concept in Akka-Streams. And in this blog I’m going to discuss about it and some related topics in detail. So, let’s begin ! Akka-Streams Basics Let’s revisit the basics first. So, the data Continue Reading

Getting Started with Akka-Streams

Reading Time: 4 minutes As the world is growing, so is the data. And analysis of this data has become important. But how will you do it? How will you work with the data whose size is unknown to you? A solution to this scenario is Akka-Streams and here I’m going to discuss it. In this blog you will get to know the basics of Akka-Streams just to get Continue Reading

Backpressure in Akka Streams

Reading Time: 4 minutes “Reactive Streams”, whenever we come across these words, there are two things that come to our mind. The first is asynchronous stream processing and the second is non-blocking backpressure. In this blog, we are going to learn about the latter part. Understanding back pressure According to the English dictionary, back pressure means “Resistance or force opposing the desired flow of fluid through pipes”. To define Continue Reading

Knolx: Alpakka-Connecting Kafka & ElasticSearch to Akka Streams

Reading Time: < 1 minute Hi all, Knoldus has organized a 30 min session on 1st  March 2019 at 3:30 PM. The topic was Alpakka – Connecting Kafka and ElasticSearch to Akka Streams.  Many people have joined and enjoyed the session. I am going to share the slides here. Please let me know if you have any question related to linked slides or video. The slides of the KnolX are here: And Continue Reading

Alpakka – Connecting Kafka and ElasticSearch to Akka streams

Reading Time: 4 minutes In our previous blog, we had a look at what Akka streams are and how they are different from the other streaming mechanisms we have. In this blog, we will be taking a little step forward into the world of Akka Streams. In order to work with Akka streams, we need a mechanism to connect Akka Streams to the existing system components. That is where Alpakka Continue Reading

Introduction to Akka Streams – Part 1

Reading Time: 4 minutes Akka Streams is a library that is used to process and transform a stream of data. In this blog, I’ll be discussing the components of Akka Streams. Akka Streams is an implementation of Reactive Streams and uses bounded buffer space, and this property is known as boundedness. To use Akka Streams, add the below dependency in your build.sbt:

Stream a file to AWS S3 using Akka Streams (via Alpakka) in Play Framework

Reading Time: 5 minutes In this blog post we’ll see how a file can be streamed from a client (eg: browser) to Amazon S3 (AWS S3) using Alpakka’s AWS S3 connector. Aplakka provides various Akka Stream connectors, integration patterns and data transformations for integration use cases. The example in this blog post uses Play Framework to provide a user interface to submit a file from a web page directly to Continue Reading

Streaming data from PostgreSQL using Akka Streams and Slick in Play Framework

Reading Time: 4 minutes In this blog post I’ll try to explain the process wherein you can stream data directly from PostgreSQL database using Scala Slick (which is Scala’s database access/query library) and Akka Streams (which is an implementation of Reactive Streams specification on top of Akka toolkit) in Play Framework. The process is going to be pretty straightforward in terms of implementation where data is read from one Continue Reading

Knolx: Guaranteed No Stress Baby Steps Using Akka Streams Part-II

Reading Time: < 1 minute Hello everyone, Knoldus organized a session on 25th November 2017. The topic was “Guaranteed No Stress Baby Steps Using Akka Streams Part-II”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video of the session. Slides: