How to deploy Rabbit MQ on Kubernetes

Table of contents

Reading Time: 3 minutes

RabbitMQ is message broker which follows AMQP protocol.One of the most important component of Rabbit MQ is broker. The brokers has the responsibility of storing the messages which the producers publish to the message queue. The Consumers consume the message from the broker and send an acknowledgement. Once the broker receives the acknowledgement, the messages are deleted. So, the main focus is to deploy a highly available rabbitmq cluster. The cluster should have more than one nodes. The messages should be in sync. This increases the complexity of deploying and managing rabbitmq on Kubernetes. To simplify rabbitmq on kubernetes, rabbitmq operator was released. The operator induces CRDs of type RabbitmqCluster . We can then use this resource to define a cluster without worrying about replication and other problems.

Prerequisites

Helm 3
kubectl client
Kubernetes Cluster

Deploy Rabbit MQ Cluster

First, we will need to deploy the rabbitmq operator, which will create the rabbitmq CRDs. To deploy the operator execute the command:

helm repo add bitnami https://charts.bitnami.com/bitnami

helm install rabbit-operator bitnami/rabbitmq-cluster-operator --namespace rabbitmq --create-namespace --atomic

To verify execute the command:

kubectl get crds

We will see a CRD RabbitmqCluster present. We will use this CRD to create a rabbitmq cluster. Create a file cluster.yaml with the following content:

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmq-cluster
spec:
  replicas: 3
  override:
    statefulSet:
      spec:
        podManagementPolicy: OrderedReady
  service:
    type: ClusterIP
  persistence:
    storageClassName: standard
    storage: 20Gi
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
            - key: app.kubernetes.io/name
              operator: In
              values:
              - rabbitmq-cluster
        topologyKey: kubernetes.io/hostname
  rabbitmq:
    additionalPlugins:
    - rabbitmq_federation
    additionalConfig: |
      disk_free_limit.absolute = 500MB
      vm_memory_high_watermark.relative = 0.6

Apply the above yaml using the command:

kubectl apply -f cluster.yaml -n rabbitmq

# To verify the pods are created
kubectl get po -n rabbitmq

Exposing the Rabbit MQ dashboard

We will create an ingress to expose the dashboard. Execute the following command:

kubectl create ing rabbit-ing-1 --class <ingress-class> --rule="<hostname>/*=rabbitmq-cluster:15672" -n rabbitmq

Replace the placeholder <ingress-class> and <hostname> in the above command and execute it. Add the specific entry for the hostname to the IP in DNS. When we open the link, we can see the RabbitMQ Dashboard.

We can also expose the dashboard using port-forwarding. To expose using port-forwarding:

kubectl port-forward svc/rabbitmq-cluster -n rabbitmq 8091:15672

We can then expose the dashboard on port localhost:8091. The dashboard looks like:

Getting the Rabbitmq username and password

Execute the following command to get the username and password for rabbitmq:

# Username
kubectl get secret rabbitmq-cluster-default-user -o jsonpath="{.data.username}" -n rabbitmq | base64 -d

# Password
kubectl get secret rabbitmq-cluster-default-user -n rabbitmq -o jsonpath="{.data.password}" | base64 -d

Use the above credentials to login to the rabbitmq. On the homepage we can see the node status, their memory usage and free space.

Testing the deployment

To test if the rabbitmq is working, create a publisher.yaml with following content:

apiVersion: v1
kind: Secret
metadata:
  name: rabbitmq-publisher
type: Opaque
stringData:
  RABBIT_USERNAME: <username>
  RABBIT_PASSWORD: <password>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rabbitmq-publisher
  labels:
    app: rabbitmq-publisher
spec:
  selector:
    matchLabels:
      app: rabbitmq-publisher
  replicas: 1
  template:
    metadata:
      labels:
        app: rabbitmq-publisher
    spec:
      containers:
      - name: rabbitmq-publisher
        image: aimvector/rabbitmq-publisher:v1.0.0
        imagePullPolicy: Always
        ports:
        - containerPort: 80
        env:
        - name: RABBIT_HOST 
          value: "rabbitmq-cluster.rabbitmq.svc.cluster.local"
        - name: RABBIT_PORT
          value: "5672"
        - name: RABBIT_USERNAME
          valueFrom:
            secretKeyRef:
              name: rabbitmq-publisher
              key: RABBIT_USERNAME
        - name: RABBIT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: rabbitmq-publisher
              key: RABBIT_PASSWORD
---
apiVersion: v1
kind: Service
metadata:
  name: rabbitmq-publisher
  labels:
    app: rabbitmq-publisher
spec:
  type: LoadBalancer
  selector:
    app: rabbitmq-publisher
  ports:
    - protocol: TCP
      name: http
      port: 80
      targetPort: 80

Replace the placeholders <username> and <password> with the values and apply the yaml file. Now execute the following command and replace the placeholder <load-balancer-ip>.

curl -X POST http://<load-balancer-ip>/publish/hello

If we check the logs of the publisher pod, we will see the message in the log. And if we go to the rabbit mq dashboard, in the queue we will find the message published.

Conclusion

In this blog, we have seen how to deploy a highly available Rabbit MQ cluster on kubernetes. To setup mirroring, we will need to use plugins, which will be covered in a different blog as it is a different topic on its own.