Author: Mahesh Chand

Working with XML in Scala

Reading Time: 2 minutes In this blog, we will talk about how we can work with XML using Scala. Scala treats XML as the first-class citizen. So, instead of embedding XML documents into strings. , you can place them inline in your code like you place an int or double value. For example scala> val xml = Hello xml: scala.xml.Elem = Hello scala> xml.getClass res2: Class[_ <: scala.xml.Elem] = Continue Reading

How to throttle messages using Akka Fsm ?

Reading Time: 3 minutes A few days ago, I was having an issue that the rate of incoming message requests to the process was too high.  All requests were thus being processed in future. By default, every message request was getting processed in parallel. But this was too much parallelism. It was flooding the thread pool with a lot of simultaneous work. A better way to do it is Continue Reading

Exploring the Real Power of Functional Programming

Reading Time: 3 minutes We have been programming with object-oriented technology for quite a while. But now we are moving away from it. Functional Programming is getting lot of attention nowadays. Even every mainstream language is now supporting functional style i.e Java, C++, C# etc. Basically, it is one of the paradigm of writing code where we write less code and do more. Why do we really care about Continue Reading

Are you using Scala Collection efficiently?

Reading Time: 5 minutes In this blog, We will be going to explore how we can use scala collections efficiently . Though, we are taking care of immutability but still something more can be done to make your code more readable and faster. List vs Vector: Vector is a collection with good random access. List is indeed a linked list with very fast append-first operation (::), but the links Continue Reading

Getting Started With Mesos

Reading Time: 2 minutes In our earlier blog Introduction to Mesos , we have talked about basic idea of Mesos. And How it works and its basic terminology. Till now we have discussed the theoretical concepts to get ourselves familiar with Mesos and now we will be going to install the mesos and see how we can monitor mesos using UI, So let’s get onto it. Step1: Download Mesos Continue Reading

Introduction to Mesos

Reading Time: 4 minutes What is Mesos ? In layman’s term, Imagine a busy airport. Airplanes are constantly taking off and landing. There are multiple runways, and an airport dispatcher is assigning time-slots to airplanes to land or takeoff. So Mesos is the airport dispatcher, runways are compute nodes, airplanes are compute tasks, and frameworks like Hadoop, Spark and Google Kubernetes are airlines companies. In technical terms, Apache Mesos Continue Reading

Having Issue How To Order Streamed Dataframe ?

Reading Time: 3 minutes A few days ago, i have to perform aggregation on streaming dataframe. And the moment, i apply groupBy for aggregation, data gets shuffled. Now the situation arises how to maintain order? Yes, i can use orderBy with streaming dataframe using Spark Structured Streaming, but only in complete mode. There is no way of doing ordering of streaming data in append mode and update mode. I Continue Reading

Unit Testing Of Kafka

Reading Time: 2 minutes Apache Kafka is a distributed publish-subscribe messaging system and a robust queue that can handle a high volume of data and enables you to pass messages from one end-point to another. Generally, data is published to topic via Producer API and  Consumers API consume data from subscribed topics. In this blog, we will see how to do unit testing of kafka. Unit testing your Kafka Continue Reading

Exploring Spark Structured Streaming

Reading Time: 6 minutes Hello Spark Enthusiasts, Streaming apps are growing more complex. And it is getting difficult to do with current distributed streaming engines. Why streaming is hard ? Streaming computations don’t run in isolation. Data arriving out of time order is a problem for batch-processed processing. Writing stream processing operations from scratch is not easy. Problem with DStreams: Processing with event-time: dealing with late data. Interoperate streaming Continue Reading

Spark Streaming vs Kafka Stream

Reading Time: 4 minutes The demand for stream processing is increasing a lot these days. The reason is that often processing big volumes of data is not enough. Data has to be processed fast, so that a firm can react to changing business conditions in real time. Stream processing is the real-time processing of data continuously and concurrently. Streaming processing” is the ideal platform to process data streams or Continue Reading

Jump Higher With Trampoline

Reading Time: 6 minutes In our earlier blog The Tale of ‘Tail Recursion’, we have talked about tail recursion. And In this blog, we will be talking about limitation of tail recursion and its solution in scala. Unfortunately JVM doesn’t support tail call optimization, but scala does as we have seen. However, what if we are not doing simple tail recursion like we usually do. What if on the Continue Reading

The Tale of ‘Tail Recursion’

Reading Time: 3 minutes Recursion in computer science is a method where the solution to a problem depends on solutions to smaller instances of the same problem (as opposed to iteration). Recursions are really cool and they are highly expressive. For example consider factorial function: But if i increase the size of input, code will blow up. Recursions don’t scale very well for large input sizes. And in practical Continue Reading