In my previous blog, we discussed how to monitor a Kafka stream application using Grafana and Graphite. In this solution, we used jmx2graphite as a metrics exporter which takes the metrics from the Jolokia URL where Jolokia exposes the JMX metrics and pushes those metrics to Graphite.
But, there is a problem with this solution that we need to deploy one jmx2graphite per service. So for example, consider the following situations:
- Having 3 instances of the same service
- or a single instance of 3 difference services
In both the above scenarios, we need to deploy 3 jmx2graphite instances. So if we will be having 10 services, each having 2 instances then we would need to deploy a total of 20 jmx2graphite. This will create an overhead to the deployment and the maintenance will be very complex.
So, we looked for some other ways to find out the optimal approach as my use case was to monitor multiple applications and I was not ready to do so much amount of deployment.
Earlier, we were running jmx2graohite as a docker instance but now we have got a new way of running jmx2graphite which is as a java agent in a new release. In this approach, the deployment overhead will be removed as it will be run as a java agent inside the service container only.
Run jmx2graphite as java agent:
- First of all, need to download the jmx2graphte java agent jar.
- Then, need to add the following arguments to the java command: -javaagent:/jmx2graphite-1.2.5-javaagent.jar
- Need to provide some environment variables which are as follows:
- SERVICE_NAME: The name of the service.
- GRAPHITE_HOST: The hostname/IP of graphite
- GRAPHITE_PORT: Protocol port of graphite. Defaults to 2004.
- SERVICE_HOST: The host on which service is running
The full command:
exec java -cp “$APP_LIB_DIR/*” -javaagent:/jmx2graphite-jvm-1.2.5-javaagent.jar=GRAPHITE_HOSTNAME=$GRAPHITE_HOST,GRAPHITE_PORT=$GRAPHITE_PORT,SERVICE_HOST=$SERVICE_HOST,SERVICE_NAME=$SERVICE_NAME com.knoldus.monitoring.StreamsExecutor
docker run -d –name monitoring_app –network=host -e GRAPHITE_HOST=127.0.0.1 -e GRAPHITE_PORT=2004 -e SERVICE_HOST=local -e SERVICE_NAME=monitoring-app stream-app:latest
I would like to give a high-level diagram of 2 different scenarios here:
Multiple instances of one service:
Multiple instances of different services:
Hope, it will help you to use jmx2graphite to monitor your application.
Happy reading !!!
References:
https://github.com/logzio/jmx2graphite
