Big Data and Fast Data

Protein Structure determination aided by Stochastic Search (Replica Exchange Monte-Carlo Method)

Introduction Proteins are large molecules, which occur in abundance in every single living organism. They carryout vital functions such as transporting oxygen, converting the food you eat into energy your body can use, and many more. Proteins are long chains of linked units called amino acids. There are 20 types of amino acids. Proteins fold into different shapes depending upon their sequence of amino acids. Continue Reading

Monitoring Kafka with Prometheus and Grafana

Kafka monitoring is an operation which is used for the optimization of the Kafka deployment. This process is easy and efficient, by applying one of the existing monitoring solutions instead of building your own. Let’s say, we use Apache Kafka for message transfer and processing and we want to monitor it.But, before learning the steps for monitoring, let’s first understand the prerequisites. Kafka It is Continue Reading

Getting Started with Akka Remoting

When we start with Akka we generally start with one actor system on our local machine but when we talk about the business application we can have multiple parts of an application and those part can run on different machine or node. Akka Remoting is a communication module to connect the actor system in peer to peer fashion. It also serves as the foundation for Continue Reading

Flinkathon: Guide to setting up a Local Flink Custer

In our previous blog post, Flinkathon: First Step towards Flink’s DataStream API, we created our first streaming application using Apache Flink. It was easy, clean, and concise. However, the real power of Apache Flink is seen on a cluster, where data is processed in a distributed manner, with the advantage of multi-core/multi-memory systems. So, in this blog post, we will see how to set up Continue Reading

Determine Kafka broker health using Kafka stream application’s JMX metrics and setup Grafana alert

As we all know, Kafka exposes the JMX metrics whether it is Kafka broker, connectors or Kafka applications. A few days ago, I got the scenario where I needed to determine Kafka broker health with the help of Kafka stream application’s JMX metrics. It looks bit awkward, right? I should use the broker’s JMX metrics to do this, why am I looking to application JMX Continue Reading

HAWK-Rust Series: Automate Infrastructure using Terraform

HAWK is a Rust based Image Recognition project, which implements a two-factor authentication by using the RFID card for user identification and Image for user validation. In this project, we have used AWS services and the whole AWS infrastructure required by this project is automated using Terraform (a tool for building, changing, and versioning infrastructure safely and efficiently).

Need the Grafana slack monitoring alert? Let’s create it now.

In my previous blog, we created the first Grafana dashboard. Now in this blog, we will create our first monitoring Grafana alert. Before creating monitoring alerts, we need to add the notification channel through which we will be notified. Here, we will be adding slack as the notification channel. Slack notification channel: First, open the URL http://localhost:3000 in the browser and log in to the Continue Reading

Let’s create your first Grafana dashboard

In my previous blog, we discussed the setup of Grafana-Graphite for JMX monitoring.  Now we will create a first Grafana dashboard where we will create Grafana queries to visualize JMX metrics stored in Graphite. As we know, Grafana UI runs on http://localhost:3000/ by default so let’s open the URL in the browser with the default username and password which is admin: admin After login either Continue Reading

Knolx: Alpakka – Connecting Kafka and ElasticSearch to Akka Streams

Hi all, Knoldus has organized a 30 min session on 1st  March 2019 at 3:30 PM. The topic was Alpakka – Connecting Kafka and ElasticSearch to Akka Streams.  Many people have joined and enjoyed the session. I am going to share the slides here. Please let me know if you have any question related to linked slides or video. The slides of the KnolX are here: And Continue Reading

Integrating Lagom Service Discovery with Kubernetes

Consider a situation where a microservice is deployed on multiple pods and on condition one pod got restarted with any of the failure reason makes unreachable and at the same time interdependent services are registers its IP for communication. Now, since other pods of the service are alive but not able to communicate makes the communication failure as well as making the failure of every Continue Reading

Flinkathon: First Step towards Flink’s DataStream API

In our previous blog posts: Flinkathon: Why Flink is better for Stateful Streaming applications? Flinkathon: What makes Flink better than Kafka Streams? We saw why Apache Flink is a better choice for streaming applications. In this blog post, we will explore how easy it is to express a streaming application using Apache Flink’s DataStream API. DataStream API DataStream API is used to develop regular programs Continue Reading

Knoldus Pune Careers - Hiring Freshers

Get a head start on your career at Knoldus. Join us!