Knoldus Blogs

Handling failed SQS Events using AWS Dead Letter Queue(DLQ)

March 26, 2021March 26, 2021Java, Scala#kinesis, AWS, Java8, Streaming

Reading Time: 3 minutes Amazon SQS is an amazing simple queuing service which offers us a secure, durable hosted queue which lets us integrate and decouple distributed software component. One of the exciting features that SQS provides us with is the support of dead letter queue. Whenever we are using SQS for queuing messages for our services there may be times that our message gets corrupted or for some Continue Reading

Stream your data using Amazon Kinesis Firehose

September 30, 2020October 14, 2020AWS, AWS Services, Scala#kinesis, firehose

Reading Time: 4 minutes We live in a world driven by data and every second we are processing a large amount of data, using it, analyzing it, and transforming it. Data is very essential for businesses these days. Therefore need for handling the Dynamically generating data is important. As the number, variety, and velocity of data sources grow, new architectures and technologies are needed. Technologies like Amazon Kinesis are Continue Reading

Using Apache Flink for Kinesis to Kafka Connect

June 24, 2020June 24, 2020Apache Flink, Flink, Scala#apache-flink, #kinesis, apache, Flink Streaming, kafka, scala

Reading Time: 3 minutes In this blog, we are going to use kinesis as a source and kafka as a consumer. Let’s get started. Step 1: Apache Flink provides the kinesis and kafka connector dependencies. Let’s add them in our build.sbt: Step 2: The next step is to create a pointer to the environment on which this program runs. Step 3: Setting parallelism of x here will cause all Continue Reading