In Spark 2.3, it added support for stream-stream joins, i.e, we can join two streaming Datasets/DataFrames and in this blog we are going to see how beautifully spark now give support for joining the two streaming dataframes. I this example,…
Interactive Queries in Apache Kafka
Apache Kafka v0.10 introduced a new feature Kafka Streams API - a client library which can be used for building applications and microservices, where the input and output data can be stored in Kafka clusters. Kafka Streams provides…
SOLID Principles: Basic building block of the software system
"Good software system begins with clean code" and by "clean code", I mean it is code that is easy to understand and easy to change. This is the point where there is a need to know about SOLID Principles which…
Running Multinode Cassandra Cluster on a Single Machine
Hey folks, in this blog I am going to discuss how can you set up a Cassandra cluster on a single machine. This situation is not ideal because being a scalable distributed NoSQL database, Cassandra cluster runs on machines/nodes spanning across…
uPickle: a flexible Json Serializer
uPickle serializer is a lightweight Json library for scala. uPickle is built on top of uJson which are used for easy manipulation of json without the need of converting it to a scala case class. We can even use uJson as standalone too. In this…
uPickle : Comparison with other Json Serialiers
In my previous blog, I talked about how uPickle works. Now I will be comparing it will many other json serializers where I will be serializing and deserializing scala case class. Before that let me discuss all the json serializers that I have…
Scorex: A Modular Blockchain, Scala Framework
Scorex is an open-source project written in Scala with loosely coupled and pluggable components. Scorex provides many abstractions for which you have to provide concrete implementations to make the Blockchain as per your requirements on the top of Scorex. It is backed by…
Are you missing on the Digital wave?
It has been an interesting week for Knoldus. At this time, almost half of the organization is awake worldwide for us in Toronto, Singapore, Berlin, Chicago, Miami, Mumbai and Noida participating in our CodeCombat 2018 (24 hours long Hackathon). This…
Structured Streaming: What is it?
With the advent of streaming frameworks like Spark Streaming, Flink, Storm etc. developers stopped worrying about issues related to a streaming application, like - Fault Tolerance, i.e., zero data loss, Real-time processing of data, etc. and started focussing only on solving business…
Back2Basics: The Magic of Implicit – #2
In our previous blog, The Magic of Implicit, we have talked about how implicit values are being used. In this blog, we will explore Type Conversion with implicit and implicit classes. Let's start with a simple example, Here we were trying…
