Advertisements

Big Data and Fast Data

Kryo Serialization in Spark

Reading Time: 4 minutes Spark provides two types of serialization libraries: Java serialization (default) Kryo serialization For faster serialization and deserialization spark itself recommends to use Kryo serialization in any network-intensive application. Then why is it not set to default : Why Kryo is not set to default in Spark? The only reason Kryo is not set to default is because it requires custom registration. Although, Kryo is supported Continue Reading

Advertisements

KnolMoney: More than a Corporate Virtual Wallet

Reading Time: 3 minutes KnolMoney – An entry gate to the whole world of possibilities that you can do with a virtual wallet. In the age of the digital world, every transaction we do mostly is by digital wallets which can be used almost at every mall/shops. But in this wallet run, what we have forgotten is the Corporate Community. Here, comes Knoldus into a picture with a solution– KnolMoney, Continue Reading

Smart Office in coming 5 Years and why?

Reading Time: 3 minutes Smart office is defined as the integration of modern technology into a office space. All the tasks of an organization become simpler and flexible with the use of IOT (Internet of Things) devices. IoT is the network of devices such as electrical appliances, physical devices and other things embed with sensors and actuators. These sensors and actuators enable the objects to establish a connection with Continue Reading

real time analytics in banking

Banking on Real-Time Analytics for Better Customer Experience

Reading Time: 4 minutes The digital storm has now made its presence felt across various industries and the banking and financial sector is also not far behind. The way customers interact with banks is not the same anymore. Things have become “in-the-moment” and banks need to be responsive as things get up to speed. Customers have become so used to mobile and online banking, that it’s taken for granted Continue Reading

Custom Implementation of Blockchain In Rust(Part 1)

Reading Time: 3 minutes Hello folks, in this blog, we will be learning how to create our own Blockchain, which is a technology behind Bitcoin. Blockchain technology has been called the greatest innovation since the internet. A Blockchain is a chain of blocks which contain information. A block contains the following:-1) Transaction Data:- This contains information about the Sender, Receiver, number of bitcoins to be transferred. 2) PreviousHash:- Hash Continue Reading

Understanding Spark’s Logical and Physical Plan in layman’s term

Reading Time: 5 minutes This blog pertains to Apache SPARK 2.x, where we will find out how Spark SQL works internally in layman’s terms and try to understand what is Logical and Physical Plan. Also we will be looking into Catalyst Optimizer. So let’s get started. First let’s see what Apache Spark is. The official definition of Apache Spark says that “Apache Spark™ is a unified analytics engine for large-scale Continue Reading

Fast Data: The New Age Analytics For Enhanced Customer Experience

Reading Time: 6 minutes Data is evolving both in terms of quality and quantity in today’s enterprises and in the past few years, changes have occurred at a much faster pace. Not long ago, Big Data was considered the next big thing for digital transformation. Technologies like Hadoop and HBase made sense as batch processing of data was the norm. But things are not the same now.  By the Continue Reading

Apache Spark

Deep Dive into Apache Spark Transformations and Action

Reading Time: 4 minutes In our previous blog of Apache Spark, we discussed a little about what Transformations & Actions are? Now we will get deeper into the topic and will understand what actually they are & how they play a vital role to work with Apache Spark? What is Spark RDD? Spark introduces the concept of an RDD (Resilient Distributed Dataset), an immutable fault-tolerant, distributed collection of objects Continue Reading

Performance Benchmarking Akka Actors vs Java Threads

Reading Time: 3 minutes Think of a scenario where you are standing in front of a long queue of your cafeteria to order your favorite food. Some people might get so frustrated that they leave the queue without even ordering. Thinking of these types of situations cafeteria management decided to introduce a token system. You can simply sit and chit-chat with your friends while waiting for your token number. Continue Reading

Diving deeper into Delta Lake

Reading Time: 6 minutes Delta Lake is an open-source storage layer that brings reliability to data lakes. It has numerous reliability features including ACID transactions, scalable metadata handling, and unified streaming and batch data processing.

Delta Lake To the Rescue

Reading Time: 4 minutes Welcome Back. In our previous blogs, we tried to get some insights about Spark RDDs and also tried to explore some new things in Spark 2.4. You can go through those blogs here: RDDs – The backbone of Apache Spark Spark 2.4: Adding a little more Spark to your code In this blog, we will be discussing something called a Delta Lake. But first, let’s Continue Reading

Application Logs: Pitfalls and Insights

Reading Time: 3 minutes Introduction In this blog article,we aim to give reader a sense of what is the need of monitoring application logs and a methodology of how we can monitor them using an example. Lets say you have a use case where you would like to monitor important aspects of your application for e.g. temperature” as a measure” or “facet” for multiple devices emitting data for complete Continue Reading

Knoldus Pune Careers - Hiring Freshers

Get a head start on your career at Knoldus. Join us!