#kinesis

Handling failed SQS Events using AWS Dead Letter Queue(DLQ)

Reading Time: 3 minutes Amazon SQS is an amazing simple queuing service which offers us a secure, durable hosted queue which lets us integrate and decouple distributed software component. One of the exciting features that SQS provides us with is the support of dead letter queue. Whenever we are using SQS for queuing messages for our services there may be times that our message gets corrupted or for some Continue Reading

Stream your data using Amazon Kinesis Firehose

Reading Time: 4 minutes We live in a world driven by data and every second we are processing a large amount of data, using it, analyzing it, and transforming it. Data is very essential for businesses these days. Therefore need for handling the Dynamically generating data is important. As the number, variety, and velocity of data sources grow, new architectures and technologies are needed. Technologies like Amazon Kinesis are Continue Reading

Using Apache Flink for Kinesis to Kafka Connect

Reading Time: 3 minutes In this blog, we are going to use kinesis as a source and kafka as a consumer. Let’s get started. Step 1: Apache Flink provides the kinesis and kafka connector dependencies. Let’s add them in our build.sbt: Step 2: The next step is to create a pointer to the environment on which this program runs. Step 3: Setting parallelism of x here will cause all Continue Reading