Author: Anjali Sharma

Comparison between different streaming engines

akka-streams, Apache Flink, Apache Kafka, Apache Spark, Big Data and Fast Data, Scala, Spark, , , , , , ,

Reading Time: 5 minutes Distributed stream processing engines have been on the rise in the last few years, first Hadoop became popular as a batch processing engine, then focus shifted towards stream processing engines. Stream processing engines can make the job of processing data that comes in via a stream easier than ever before and by using clustering can enable processing data in larger sets in a timely manner. Continue Reading

kafka with spark

Dynamic Partition Pruning in Spark 3.0

Apache Spark, Scala, ,

Reading Time: 6 minutes Dynamic Partition Pruning in Spark 3.0 With the release of Spark 3.0, big improvements were implemented to enable Spark to execute faster and there came many new features along with it. Among them, dynamic partition pruning is one. Before diving into the features which are new in Dynamic Partition Pruning let us understand what is Partition Pruning. Partition Pruning in Spark In standard database pruning Continue Reading

Backpressure in Akka Streams

Scala, , , , , ,

Reading Time: 4 minutes “Reactive Streams”, whenever we come across these words, there are two things that come to our mind. The first is asynchronous stream processing and the second is non-blocking backpressure. In this blog, we are going to learn about the latter part. Understanding back pressure According to the English dictionary, back pressure means “Resistance or force opposing the desired flow of fluid through pipes”. To define Continue Reading

Beginners Guide for Mocking in Scala

Scala, Unit Testing, , , , ,

Reading Time: 3 minutes We all must know how the unit test cases are one of the most important part of an application. If not? Then I must tell you that unit testing is one of the earliest testings which is performed on the unit of code and the earlier the defects are detected, the easier it is to fix. It reduces the difficulties of discovering errors contained in Continue Reading

Akka Dispatcher- All that you need to know

Scala, , ,

Reading Time: 4 minutes Pre-requisite knowledge: Basics of Akka and Actor System. Fun fact: Did you know that ActorSystem has a heart? Well yes, the ActorSystem is considered to have a heart and it is the “Dispatchers”. We will see how, in this blog. In the real world, dispatchers are the communication coordinators responsible for receiving and passing messages. For example, in emergency services like 911, the dispatchers are Continue Reading

Knoldus logo

Knoldus is the world's largest pure-play Scala and Spark company. We modernize enterprise through cutting-edge digital engineering by leveraging Scala, Functional Java and Spark ecosystem. Our mission is to provide reactive and streaming fast data solutions that are message-driven, elastic, resilient, and responsive.

About Knoldus

Recent Posts

Subscribe to our newsletter

Contact us

+(1) 647-467-4396
hello@knoldus.com

Partners

  • Knoldus Lightbend
  • Knoldus Databricks
  • Knoldus IBM
  • Knoldus Datastax
  • Knoldus Confluent
Copyright 2017-20 © Knoldus