State Of the World(SOW) in AMPS

Reading Time: 4 minutes

Hi readers, in this blog I am going to explain SOW in amps. So in order to understand this, you need to go through my previous blog on amps so as to get basic knowledge about it.

AMPS’s SOW

State of the World(SOW) in amps is one of the main stands out feature which provides the ability to persist the most recent update for each distinct message published to a given topic.

The State of the World (SOW) can be thought of as a database where messages published to AMPS are filtered into topics, and where the topics store the latest update to each distinct message. It provides the subscriber to resolve any conflicts regarding the data it has and the data that is updated in the SOW topic by querying the current state of the topic.

It can also be used for caching data which is stored as a snapshot in key/value stores. SOW is best suited for storing offsets for the data that is already read by the system. So it will work as a checkpoint for the data that we have read so far.

AMPS also provides the ability to keep historical snapshots of the contents of the State of the World, which allows subscribers to query the contents of the SOW at a particular point in time and replay changes from that point in time.

AMPS can maintain the SOW for a topic in a persistent file, which will be available across restarts of the AMPS server. The SOW can also be transient, in which case the state of the SOW does not persist across server restarts.

How SOW works?

AMPS identifies a message by using a unique key for the message. It maintains a key know as SOW Key for each unique data it receives. The first time a message is received with a particular SOW key, AMPS adds the message to the SOW. Subsequent messages with the same SOW key, the value gets updated.

There are many ways to create a SOW key:

1) SOW key based on the content of the message

This is the recommended approach where we can specify the fields to use for the key in a configuration file. AMPS assigns a SOW key based on the content of the message. AMPS finds the specified fields in the message and computes a SOW key based on the name of the topic and the values in these fields.

Eg.

<key>/orderId</key>

Here orderId will be used to generate the SOW Key.

2) Explicitly providing the SOW Key

A topic can also be configured to require that a publisher provide a SOW key for each message when publishing the message to AMPS.

3) Custom SOW key

AMPS also supports the ability for custom SOW key generation logic to be defined in an AMPS module, which will be invoked to generate the SOW key for each message.

Now let’s understand how it actually works.

I am going to take the same example specified in the amps document.

Screenshot from 2019-02-12 13-37-54

Let’s say we have configured amps to take orderId to create the sow key then each message comes in, AMPS uses the contents of the /orderId field to generate a SOW key for the message. The SOW key is used to identify unique records in the SOW, so AMPS will store a distinct record for each distinct /orderId value published to this topic. The calculated SOW key will be returned in the SowKey header of messages received from the topic in the SOW.

Screenshot from 2019-02-12 13-41-23

Two messages are published where neither of the messages has matching keys existing in the ORDERS topic, the messages are both inserted as new messages. Sometime after these messages are processed, an update comes in for the order with an orderId of 2. This message changes the price from 120 to 95. Since the incoming message has an orderId of 2, this matches an existing record and overwrites the existing message for the same SOW key.

Now let’s do a small demo. I will be going with the recommended approach i.e the unique identifier will be set in the amps configuration file and the AMPS will create a SOW key based on that. Now our default configuration file will also contain:

where we specify the topic name and the key which is used to create sow keys. We can also specify the message type which is used ie. json, fix, nvfix messages.

To publish a message:

hAClient.publish(SOW_TOPIC_NAME, SOW_MESSAGE)

 

For subscribing to a SOW topic we can use sowAndSubscribe() to subscribe to the SOW topic.

hAClient.sowAndSubscribe(SOW_TOPIC_NAME)

You can find the complete code on my GitHub.

Reference:

Amps Documentation


Written by 

I am a Software Consultant at Knoldus Inc. I am a Scala Enthusiast. I am familiar with Object Oriented Programming Paradigms, and has also worked upon .NET based technologies. Aside from being a programmer, I am familiar with NoSQL database technologies such like Cassandra. I also worked on Lagom microservice architecture.

Knoldus Pune Careers - Hiring Freshers

Get a head start on your career at Knoldus. Join us!