Knoldus Blogs

Marshmallow: Easy Serialization in Python

November 20, 2020November 20, 2020pythonpython, Serialization

Reading Time: 2 minutes Introduction Marshmallow, stylized as “marshmallow”, is an object-relational mapping library which is used to convert objects to and from Python data types. It is often used alongside SQLAlchemy, an ORM that maps database schemas to Python objects. Marshmallow is often used to deserialize Python dicts to SQLAlchemy models and vice versa. Let’s focus on how to use Marshmallow. Creating Schemas First, we need to create Continue Reading

All you need to know about Avro schema

August 11, 2020August 11, 2020Studio-Scalaavro, scala, schema, Serialization

Reading Time: 4 minutes In this post, we are going to dive into the basics of the Avro Schema. We will create a sample avro schema and serialize it to a sample output file and also read the file as an example according to the avro schema. Intro to Avro Apache Avro is a data serialization system developed by Doug Cutting, the father of Hadoop that helps with data Continue Reading

Serialization in Lagom

May 6, 2020May 6, 2020Lagom, Microservices, Play, Play Framework, Studio-Scalalagom, Serialization, Serialization using play

Reading Time: 2 minutes In my previous blogs on Lagom, we discussed Persistent Entity and Read-side and Write side in Lagom. In this blog, I’m going to discuss how serialization is handled in Lagom. But before that let’s understand what serialization is ? Serialization is a process of converting an object into the stream of bytes so that you can store or transmit it in memory, database etc. Its Continue Reading

Kryo Serialization in Spark

December 12, 2019December 17, 2019Apache Spark, Studio-ScalaApache Spark, scala, Serialization

Reading Time: 4 minutes Spark provides two types of serialization libraries: Java serialization and (default) Kryo serialization. For faster serialization and deserialization spark itself recommends to use Kryo serialization in any network-intensive application. Then why is it not set to default : Why Kryo is not set to default in Spark? The only reason Kryo is not set to default is because it requires custom registration. Although, Kryo is Continue Reading

Using Protocol Buffers in Scala

December 13, 2018Studio-Scalabuffer, google, protocol, scala, Serialization

Reading Time: 2 minutes What is Protocol Buffers? Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. Continue Reading

Protobuf Serialization in Akka

November 11, 2017August 16, 2018Akka, Studio-ScalaActor, protobuf, scalaPB, Serialization

Reading Time: 4 minutes Before Protobuf, lets have a look at what role does serialization play in Akka. The messages that Akka actors send to each other are JVM objects (e.g. instances of Scala case classes). Message passing between actors that live on the same JVM is straightforward. It is simply done via reference passing. However, messages that have to escape the JVM to reach an actor running on Continue Reading