Monitoring

Linking Logs to Grafana Panels using Loki

Reading Time: 4 minutes Hi folks, in this blog, we will be discussing what challenges we face when we want to view logs with respect to metrics and how we can achieve this using Grafana, Loki, and Promtail. Problem Statement When using monitoring solutions like Grafana, Prometheus, Elatic Stack etc for our infrastructure, we have metrics and logs decoupled to an extent that it is very difficult to look Continue Reading

Monitoring application using Prometheus and grafana

Reading Time: 5 minutes Hello readers, In this blog I will discuss about application and system monitoring using Prometheus and Grafana. we need something to monitor! Part of that is going to be our Ubuntu host and the application will be a simple web page host program called sample-App that uses the Express web framework to do most of the hard work for us. we are going to download Continue Reading

Data Lake – Build it in Phases

Reading Time: 3 minutes Data Lake – How to build a data lake and what are the phases involved in the same.

Application Logs: Pitfalls and Insights

Reading Time: 3 minutes Introduction In this blog article,we aim to give reader a sense of what is the need of monitoring application logs and a methodology of how we can monitor them using an example. Lets say you have a use case where you would like to monitor important aspects of your application for e.g. temperature” as a measure” or “facet” for multiple devices emitting data for complete Continue Reading

Monitoring with DataDog

Reading Time: 3 minutes Hello people! Monitoring is one of the major phases in DevOps lifecycle, It makes sure that application running in your production environments runs continuously without any failure and if any failure comes into the picture then it can be tackled as soon as possible. A simple is to integrate a solution with environments that intelligently monitor, analyze and manage cloud, on-premise, and hybrid applications and Continue Reading

Monitoring Kafka with Prometheus and Grafana

Reading Time: 3 minutes Kafka monitoring is an operation which is used for the optimization of the Kafka deployment. This process is easy and efficient, by applying one of the existing monitoring solutions instead of building your own. Let’s say, we use Apache Kafka for message transfer and processing and we want to monitor it.But, before learning the steps for monitoring, let’s first understand the prerequisites. Kafka It is Continue Reading

Tired off creating Grafana dashboards manually? Let’s automate it.

Reading Time: 3 minutes We have already discussed the creation of Grafana dashboards and alerts in my previous blogs. We were doing that manually. But think of, if we need to do that in more than 10 environments then we need to repeat that manual process again and again and sometimes we get frustrated by doing all these repetitive stuff. We should have some automated process for doing this. So Continue Reading

Determine Kafka broker health using Kafka stream application’s JMX metrics and setup Grafana alert

Reading Time: 3 minutes As we all know, Kafka exposes the JMX metrics whether it is Kafka broker, connectors or Kafka applications. A few days ago, I got the scenario where I needed to determine Kafka broker health with the help of Kafka stream application’s JMX metrics. It looks bit awkward, right? I should use the broker’s JMX metrics to do this, why am I looking to application JMX Continue Reading

Need the Grafana slack monitoring alert? Let’s create it now.

Reading Time: 3 minutes In my previous blog, we created the first Grafana dashboard. Now in this blog, we will create our first monitoring Grafana alert. Before creating monitoring alerts, we need to add the notification channel through which we will be notified. Here, we will be adding slack as the notification channel. Slack notification channel: First, open the URL http://localhost:3000 in the browser and log in to the Continue Reading

Let’s create your first Grafana dashboard

Reading Time: 4 minutes In my previous blog, we discussed the setup of Grafana-Graphite for JMX monitoring.  Now we will create a first Grafana dashboard where we will create Grafana queries to visualize JMX metrics stored in Graphite. As we know, Grafana UI runs on http://localhost:3000/ by default so let’s open the URL in the browser with the default username and password which is admin: admin After login either Continue Reading

Running jmx2graphite as a java agent to push the JMX metrics into Graphite

Reading Time: 2 minutes In my previous blog, we discussed how to monitor a Kafka stream application using Grafana and Graphite. In this solution, we used jmx2graphite as a metrics exporter which takes the metrics from the Jolokia URL where Jolokia exposes the JMX metrics and pushes those metrics to Graphite. But, there is a problem with this solution that we need to deploy one jmx2graphite per service. So Continue Reading

Transport Cinnamon Matrices From Lagom To Prometheus

Reading Time: 3 minutes Monitoring is a pain when it comes to distributed applications, and even more when you have shared or non-shared variables to monitor in your application. Here in this blog, I’ll explain two tools which can ease the monitoring efforts, one for generating metrics called Cinnamon and other to visualize them, called Prometheus. Let’s have a quick brief intro about these two – Prometheus – An open-source monitoring system with Continue Reading