Serialization

All you need to know about Avro schema

Reading Time: 4 minutes In this post, we are going to dive into the basics of the Avro Schema. We will create a sample avro schema and serialize it to a sample output file and also read the file as an example according to the avro schema. Intro to Avro Apache Avro is a data serialization system developed by Doug Cutting, the father of Hadoop that helps with data Continue Reading

Serialization in Lagom

Reading Time: 2 minutes In my previous blogs on Lagom, we discussed Persistent Entity and Read-side and Write side in Lagom. In this blog, I’m going to discuss how serialization is handled in Lagom. But before that let’s understand what serialization is ? Serialization is a process of converting an object into the stream of bytes so that you can store or transmit it in memory, database etc. Its Continue Reading

Kryo Serialization in Spark

Reading Time: 4 minutes Spark provides two types of serialization libraries: Java serialization and (default) Kryo serialization. For faster serialization and deserialization spark itself recommends to use Kryo serialization in any network-intensive application. Then why is it not set to default : Why Kryo is not set to default in Spark? The only reason Kryo is not set to default is because it requires custom registration. Although, Kryo is Continue Reading

Using Protocol Buffers in Scala

Reading Time: 2 minutes What is Protocol Buffers? Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. Continue Reading

Protobuf Serialization in Akka

Reading Time: 4 minutes Before Protobuf, lets have a look at what role does serialization play in Akka. The messages that Akka actors send to each other are JVM objects (e.g. instances of Scala case classes). Message passing between actors that live on the same JVM is straightforward. It is simply done via reference passing. However, messages that have to escape the JVM to reach an actor running on Continue Reading