Monitoring Kafka with Prometheus and Grafana

Kafka monitoring is an operation which is used for the optimization of the Kafka deployment. This process is easy and efficient, by applying one of the existing monitoring solutions instead of building your own. Let’s say, we use Apache Kafka for message transfer and processing and we want to monitor it.But, before learning the steps for monitoring, let’s first understand the prerequisites. Kafka It is Continue Reading

Tired off creating Grafana dashboards manually? Let’s automate it.

We have already discussed the creation of Grafana dashboards and alerts in my previous blogs. We were doing that manually. But think of, if we need to do that in more than 10 environments then we need to repeat that manual process again and again and sometimes we get frustrated by doing all these repetitive stuff. We should have some automated process for doing this. So Continue Reading

Determine Kafka broker health using Kafka stream application’s JMX metrics and setup Grafana alert

As we all know, Kafka exposes the JMX metrics whether it is Kafka broker, connectors or Kafka applications. A few days ago, I got the scenario where I needed to determine Kafka broker health with the help of Kafka stream application’s JMX metrics. It looks bit awkward, right? I should use the broker’s JMX metrics to do this, why am I looking to application JMX Continue Reading

Need the Grafana slack monitoring alert? Let’s create it now.

In my previous blog, we created the first Grafana dashboard. Now in this blog, we will create our first monitoring Grafana alert. Before creating monitoring alerts, we need to add the notification channel through which we will be notified. Here, we will be adding slack as the notification channel. Slack notification channel: First, open the URL http://localhost:3000 in the browser and log in to the Continue Reading

Let’s create your first Grafana dashboard

In my previous blog, we discussed the setup of Grafana-Graphite for JMX monitoring.  Now we will create a first Grafana dashboard where we will create Grafana queries to visualize JMX metrics stored in Graphite. As we know, Grafana UI runs on http://localhost:3000/ by default so let’s open the URL in the browser with the default username and password which is admin: admin After login either Continue Reading

Running jmx2graphite as a java agent to push the JMX metrics into Graphite

In my previous blog, we discussed how to monitor a Kafka stream application using Grafana and Graphite. In this solution, we used jmx2graphite as a metrics exporter which takes the metrics from the Jolokia URL where Jolokia exposes the JMX metrics and pushes those metrics to Graphite. But, there is a problem with this solution that we need to deploy one jmx2graphite per service. So Continue Reading

Transport Cinnamon Matrices From Lagom To Prometheus

Monitoring is a pain when it comes to distributed applications, and even more when you have shared or non-shared variables to monitor in your application. Here in this blog, I’ll explain two tools which can ease the monitoring efforts, one for generating metrics called Cinnamon and other to visualize them, called Prometheus. Let’s have a quick brief intro about these two – Prometheus – An open-source monitoring system with Continue Reading

Knoldus Pune Careers - Hiring Freshers

Get a head start on your career at Knoldus. Join us!