Deploying Elasticsearch on Kubernetes

Hello guys, in this article, we will be looking how to deploy Elasticsearch cluster on Kubernetes on a cluster setup on Google Kubernetes Engine.

We will be following these steps given below to successfully deploy Elasticsearch on Kubernetes:
1. Setup Persistant Volume via Storage Classes
2. Enable node discovery for Elasticsearch through Headless Service
3. Deploy Elasticsearch cluster using StatefulSet

Prerequisites

1. Cluster setup on Google Kubernetes Engine with altleast 1.5 CPU and 3 Gb memory.
2. Kubectl synced to nodes setup on Google Kubernetes Engine and manage our cluster.

Setting our Persistent Volume

As a first step, we will have to create a Storage Class on our cluster. Let us start by creating a manifest file for the same and name it storage.yaml:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: ssd
provisioner: kubernetes.io/gce-pd
parameters:
  type: pd-ssd

Observe that the Storage Class provisioner we have used is of Google Cloud Engine with the type parameter as pd-ssd. Also, we can also use pd-standard to enable standard disk as persistent disk.

Let us go on and enable this Storage Class we have made:

kubectl apply -f storage.yaml

We can verify our that our Storage Class was created by using the below command:

kubectl get sc

If we want to deploy the Elasticsearch Cluster on other than GKE, we will have to update the provisioner. The available provisioners and parameters can be checked here.

Enabling Node Discovery

The next step is for us to enable node discovery for Elasticsearch using Headless Service. So, let us create a file named service.yaml for the same:

apiVersion: v1
kind: Service
metadata:
 name: es
 labels:
  service: elasticsearch
spec:
 clusterIP: None
 ports:
 - port: 9200
   name: serving
 - port: 9300
   name: node-to-node
 selector:
  service: elasticsearch

To enable this Headless Service run the following command:

kubectl apply -f service.yaml

Now every pod that will have label service:elasticsearch would be accessible via <PODNAME>.es.default.cluster.local inside a k8s cluster.

To lookup our service, we can use the following command:

kubectl get svc es

We can also use the below command for more detailed description:

kubectl describe svc es

Setting Up Elasticsearch Cluster

The final stage is for us to deploy the Elasticsearch cluster. Let us create one more file with name elasticsearch.yaml for our StatefulSet:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: elasticsearch
  labels:
    service: elasticsearch
spec:
  serviceName: es
  replicas: 3
  selector:
    matchLabels:
      service: elasticsearch
  template:
    metadata:
      labels:
        service: elasticsearch
    spec:
      terminationGracePeriodSeconds: 300
      initContainers:
      - name: fix-the-volume-permission
        image: busybox
        command:
        - sh
        - -c
        - chown -R 1000:1000 /usr/share/elasticsearch/data
        securityContext:
          privileged: true
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
      - name: increase-the-vm-max-map-count
        image: busybox
        command:
        - sysctl
        - -w
        - vm.max_map_count=262144
        securityContext:
          privileged: true
      - name: increase-the-ulimit
        image: busybox
        command:
        - sh
        - -c
        - ulimit -n 65536
        securityContext:
          privileged: true
      containers:
      - name: elasticsearch
        image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.2.4
        ports:
        - containerPort: 9200
          name: http
        - containerPort: 9300
          name: tcp
        resources:
          requests:
            memory: 4Gi
          limits:
            memory: 6Gi
        env:
          - name: cluster.name
            value: elasticsearch-cluster
          - name: node.name
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: discovery.zen.ping.unicast.hosts
            value: "elasticsearch-0.es.default.svc.cluster.local,elasticsearch-1.es.default.svc.cluster.local,elasticsearch-2.es.default.svc.cluster.local"
          - name: ES_JAVA_OPTS
            value: -Xms4g -Xmx4g
        volumeMounts:
        - name: data
          mountPath: /usr/share/elasticsearch/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
        - ReadWriteOnce
      storageClassName: ssd
      resources:
        requests:
          storage: 10Gi

We have used multiple initContainers to setup our node before running the Elasticsearch image. Let us look into them one-by-one:

1. fix-the-volume-permission: This is to fix the volume permission because by default Elasticsearch container is not run as non root user.
2. increase-the-vm-max-map-count: This is used to increace the default vm.max_map_count to 262144
3. increase-the-ulimit: Used for increasing the ulimit

We can also request and limit storage as per our cluster configurations.

We will use the following command to deploy this StatefulSet:

kubectl apply -f elasticsearch.yaml

Conclusion

When our cluster is ready, we can check if cluster is created or not by accessing one of the elasticsearch node via port-forward:

kubectl port-forward elasticsearch-0 9200:9200

This will forward all request to localhost:9200 to the elasticsearch-0 node.

Then we can use curl to look at our Elasticsearch cluster:

curl http://localhost:9200/_cluster/state?pretty

It will show us a cluster of size 3 is formed.

This is how we can deploy Elasticsearch cluster on Kubernetes with node discovery enabled.

References

https://www.elastic.co/products/elastic-cloud-kubernetes
https://dzone.com/articles/running-elasticsearch-on-kubernetes