Karpenter : A Kubernetes Cluster AutoScaler

Table of contents

Reading Time: 5 minutes

abhishek

Hello Readers, Today we are going to talk about Karpenter: a kubernetes cluster autoscaler but before understanding karpenter it’s good to discuss what is an autoscaler and its importance in kubernetes cluster.

What is an AutoScaler?

Today many organisations use kubernetes for micro services orchestration as kubernetes comes with many important features like Service Discovery, Load balancing, Self-Healing

which makes it popular tools for Container management and orchestration. But One of the great promises of using Kubernetes is that it has the ability to scale infrastructure dynamically based on user demand. It provides multiple layers of auto scaling functionality: Pod-based scaling with the Horizontal Pod AutoScaler and the Vertical Pod AutoScaler, as well as node-based with the Cluster AutoScaler.

Kubernetes cluster is a group of machines which are composed of nodes in which pods run and in pod containers that request resources such as CPU, Memory, and GPU. The Cluster Autoscaler adds or removes Nodes in a Cluster based on resource requests from Pods. The cluster autoscaler increases the size of the cluster when there are pods that are not able to be scheduled due to resource shortages. It can be configured to not scale up or down past a certain number of machines. Today mostly cloud providers have their way of AutoScaling of Clusters.

Karpenter : A Kubernetes Cluster AutoScaler

But cluster autoscaler also have some limitations like :

It doesn’t take into account actual CPU/GPU/Memory usage, just resource requests and limits
Scaling up isn’t immediate which causes services to experience downtime or latency, etc.
scaling down not guaranteed.

To overcome this AWS Labs developed a node lifecycle management solution tool called Karpenter which is an open-source solution and vendor neutral cluster autoscaling tool.

What is Karpenter?

Karpenter is an open-source, flexible, high-performance Kubernetes cluster autoscaler. It helps improve application availability and cluster efficiency by rapidly launching right-sized compute resources in response to changing application load. Karpenter also provides just-in-time compute resources to meet application’s needs and will soon automatically optimize a cluster’s compute resource footprint to reduce costs and improve performance.

Before Karpenter, Kubernetes users needed to dynamically adjust the compute capacity of their clusters to support applications using Amazon EC2 Auto Scaling groups and the Kubernetes Cluster Autoscaler. Nearly half of Kubernetes customers on AWS report that configuring cluster auto scaling using the Kubernetes Cluster Autoscaler is challenging and restrictive.

How does Karpenter work?

Watching: Observes the pod resource requests of unscheduled pods.
Evaluating: Direct provision of Just-in-time capacity of the node. (Groupless Node Autoscaling).
Provisioning: Nodes that meet the requirements of the pods.
Scheduling: The pods to run on the new nodes.
Removal: Unused node removal.

What makes a Karpenter efficient?

Karpenter has two control loops that maximize the availability and efficiency of the cluster.

Allocator — Allocator ensure fast scheduling of pending pods on nodes. It acts as fast-acting controller.
Reallocator — When excess node capacity reallocated as pods are evicted then reallocator comes in Picture. The Reallocator is a slow-acting cost-sensitive controller that

Difference between AutoScaler(AS) & Karpenter ?

In the case of Cluster Autoscaler , users don’t have direct control over instances, we control them through Auto Scaling Group. It just asked ASG to increase or decrease the count of nodes. But here karpenter differs Cluster AutoScaler, Karpenter manages the node directly which enables it to retry in milliseconds instead of minutes when capacity is unavailable.It also allows Karpenter to leverage diverse instance types, availability zones, and purchase options without the creation of hundreds of node groups.

Hands-on

Install Karpenter

Prerequisite

AWS CLI
kubectl – the Kubernetes CLI
eksctl – the CLI for AWS EKS
EKS Cluster – Setup eks cluster
helm – the package manager for Kubernetes

After setting up the tools, set the following environment variables to store commonly used values.

export CLUSTER_NAME=<cluster-name>
export AWS_DEFAULT_REGION=<region-code>
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

2. Now, connect to created EKS cluster through following commands

Connect to aws account

    aws configure

Create cluster with ekstcl

cat <<EOF > demo-cluster.yaml
---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: ${CLUSTER_NAME}
  region: ${AWS_DEFAULT_REGION}
  version: "1.21"
  tags:
    karpenter.sh/discovery: ${CLUSTER_NAME}
managedNodeGroups:
  - instanceType: t3.micro
    amiFamily: AmazonLinux2
    name: ${CLUSTER_NAME}
    desiredCapacity: 1
    minSize: 1
    maxSize: 3
iam:
  withOIDC: true
EOF

eksctl create cluster -f demo-cluster.yaml

3. Creation of IAM Role for Karpenter and Karpenter Controller.

* IAM Resource creation

TEMPOUT=$(mktemp)

curl -fsSL https://karpenter.sh/v0.6.1/getting-started/cloudformation.yaml  > $TEMPOUT \
&& aws cloudformation deploy \
  --stack-name Karpenter-${CLUSTER_NAME} \
  --template-file ${TEMPOUT} \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides ClusterName=${CLUSTER_NAME}

* Grant access to instances using the profile to connect to the cluster.

eksctl create iamidentitymapping \
  --username system:node:{{EC2PrivateDNSName}} \
  --cluster  ${CLUSTER_NAME} \
  --arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/KarpenterNodeRole-${CLUSTER_NAME} \
  --group system:bootstrappers \
  --group system:nodes

Now, Karpenter is good to launch new EC2 instances and instances will connected to eks cluster.

* KarpenterController IAM Role

eksctl create iamserviceaccount \
  --cluster $CLUSTER_NAME --name karpenter --namespace karpenter \
  --attach-policy-arn arn:aws:iam::$AWS_ACCOUNT_ID:policy/KarpenterControllerPolicy-$CLUSTER_NAME \
  --approve

4. Install Karpenter Helm Chart

helm repo add karpenter https://charts.karpenter.sh
helm repo update
helm upgrade --install karpenter karpenter/karpenter --namespace karpenter \
  --create-namespace --set serviceAccount.create=false --version v0.6.1 \
  --set controller.clusterName=${CLUSTER_NAME} \
  --set controller.clusterEndpoint=$(aws eks describe-cluster --name ${CLUSTER_NAME} --query "cluster.endpoint" --output json) \
  --set aws.defaultInstanceProfile=KarpenterNodeInstanceProfile-${CLUSTER_NAME}

Deploy Provisioner

Scheduling and provisioning decisions based on pod attributes such as labels and affinity by karpenter are handled by single karpenter provisioner. Provisioner use security gropu selector and subnet selector for resource discovery for launching nodes. If you remember we have used a tag (tags:
karpenter.sh/discovery: ${CLUSTER_NAME}) in eksctl command earlier

cat <<EOF | kubectl apply -f -
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot"]
  limits:
    resources:
      cpu: 1000
  provider:
    subnetSelector:
      karpenter.sh/discovery: ${CLUSTER_NAME}
    securityGroupSelector:
      karpenter.sh/discovery: ${CLUSTER_NAME}
  ttlSecondsAfterEmpty: 30
EOF

Now karpenter is ready for provisioning nodes, now we will deploy some pods and see how karpenter works.

* Automatic Node Provisioning

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: inflate
spec:
  replicas: 0
  selector:
    matchLabels:
      app: inflate
  template:
    metadata:
      labels:
        app: inflate
    spec:
      terminationGracePeriodSeconds: 0
      containers:
        - name: inflate
          image: busybox
          resources:
            requests:
              cpu: 1
EOF

kubectl scale deployment inflate --replicas 5
kubectl logs -f -n karpenter $(kubectl get pods -n karpenter -l karpenter=controller -o name)

Now, delete the deployment. After 30 seconds (ttlSecondsAfterEmpty), Karpenter should terminate the now empty nodes.

kubectl delete deployment inflate
kubectl logs -f -n karpenter $(kubectl get pods -n karpenter -l karpenter=controller -o name)

Conclusion

So in this blog we have seen importance of kubernetes cluster autoscaler and use and how Karpenter acts as primary tool for kubernetes cluster autoscaler. Through this we can achieve high availability of kubernetes cluster or application deployed in cluster. That’s pretty much it for this article. If you have any feedback or queries, please do let me know in the comments. Also, if you liked the article, please give me a thumbs up and I will keep writing blogs like this for you in the future as well. Keep reading and Keep coding.