Kubecost Metrics Exporter on Kubernetes

Reading Time: 3 minutes

Kubecost helps you & your FinOps team to monitor and manage the cost and also capacity in Kubernetes environments. It helps you integrate various tools with your infrastructure to help your team track, manage, and reduce the overall cost.

You can monitor & have insights on your Kubernetes spends at a very granular level additionally you can have insights on other cloud resources out of the cluster.

Kubecost Prometheus Metrics exporter exports your Kubernetes cost metrics to the Prometheus server. Then the exported metrics can be visualized in a Grafana dashboard.

This exporter is used in cost monitoring & insights & promotes FinOps practices.

Prerequisites

  • Kubernetes Cluster
  • Helm v3
  • Prometheus Operator

Grafana Dashboard for the Exporter

Setup Prometheus

1 Create custom-values.yaml

This file contains configuration to deploy a Grafana instance & add Prometheus datasource which has metrics from kubecost exporter.

grafana:
  enabled: true
  additionalDataSources: 
  - name: kubecost
    access: proxy
    basicAuth: false
    basicAuthPassword: pass
    basicAuthUser: daco
    editable: false
    jsonData:
        tlsSkipVerify: true
    orgId: 1
    type: prometheus
    url: "http://prometheus-operated.cost-model.svc.cluster.local:9090"
    version: 1

prometheus:
  enabled: false
  

2 Deploy prometheus operator

Run these commands to deploy a Prometheus operator in your Kubernetes cluster. It uses helm to deploy the operator.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm helm upgrade --install prometheus-operator prometheus-community/kube-prometheus-stack -f custom-values.yaml --namespace prometheus-operator --create-namespace

Deploy Kubecost exporter

1 Create rb-service-account-prometheus.yaml

Create file names rb-service-account-prometheus.yaml and add the below content in it. This will create a service account for the Prometheus pod & binds a role with it in a namespace.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: prometheus
rules:
- apiGroups: [""]
  resources:
  - nodes
  - nodes/metrics
  - services
  - endpoints
  - pods
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources:
  - configmaps
  verbs: ["get"]
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: prometheus
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: prometheus
subjects:
- kind: ServiceAccount
  name: prometheus

2 Create exporter.yaml file

Place the below content in exporter.yaml file & apply the file to create resources.

---

apiVersion: v1
kind: Namespace
metadata:
    name: cost-model

---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cost-model

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: cost-model 
rules:
  - apiGroups:
      - ''
    resources:
      - configmaps
      - deployments
      - nodes
      - pods
      - services
      - resourcequotas
      - replicationcontrollers
      - limitranges
      - persistentvolumeclaims
      - persistentvolumes
      - namespaces
      - endpoints
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - daemonsets
      - deployments
      - replicasets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - apps
    resources:
      - statefulsets
      - deployments
      - daemonsets
      - replicasets
    verbs:
      - list
      - watch
  - apiGroups:
      - batch
    resources:
      - cronjobs
      - jobs
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - autoscaling
    resources:
      - horizontalpodautoscalers
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - policy
    resources:
      - poddisruptionbudgets
    verbs:
      - get
      - list
      - watch
  - apiGroups: 
      - storage.k8s.io
    resources: 
      - storageclasses
    verbs:
      - get
      - list
      - watch

---

# Bind the role to the service account
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cost-model
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cost-model
subjects:
  - kind: ServiceAccount
    name: cost-model
    namespace: cost-model

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cost-model
  labels:
    app: cost-model
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cost-model
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: cost-model
    spec:
      restartPolicy: Always
      serviceAccountName: cost-model
      containers:
        - image: quay.io/kubecost1/kubecost-cost-model:latest
          name: cost-model
          resources:
            requests:
              cpu: "10m"
              memory: "55M"
          env:
            - name: PROMETHEUS_SERVER_ENDPOINT
              value: "http://prometheus-operated.cost-model" 
            - name: CLOUD_PROVIDER_API_KEY
              value: "AIzaSyD29bGxmHAVEOBYtgd8sYM2gM2ekfxQX4U" # The GCP Pricing API requires a key. This is supplied just for evaluation.
            - name: CLUSTER_ID
              value: "cluster-one" # Default cluster ID to use if cluster_id is not set in Prometheus metrics.
          imagePullPolicy: Always

---

kind: Service
apiVersion: v1
metadata:
  name: cost-model
  labels:
    app: kubecost
    prometheus: cost-model
spec:
  selector:
    app: cost-model
  type: ClusterIP
  ports:
    - name: cost-model
      port: 9003
      targetPort: 9003

---

3 Create service-monitor-cost-model.yaml

Create this file and place the below content in the file. Then just apply the file to create a service monitor resource.

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    prometheus: cost-model
  name: cost-model
spec:
  endpoints:
  - path: /metrics
    port: cost-model
  selector:
    matchLabels:
      app: kubecost
      prometheus: cost-model

4 Create prometheus-custom-resource.yaml

Now, create prometheus resource file & apply it. It deploys a statefulset & headless service for prometheus.

apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  labels:
    prometheus: cost-model
    app: kubecost
  name: kubecost

spec:
  replicas: 1
  scrapeInterval: 5s
  securityContext:
    fsGroup: 2000
    runAsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: prometheus
  serviceMonitorNamespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: cost-model
  serviceMonitorSelector:
    matchLabels:
      prometheus: cost-model

Exporter Deployed

Now we’re done here, exporter is now deployed on Kubernetes with Prometheus instance & Grafana. Now we can visualize & monitor the Kubernetes cluster cost metrics & optimize the cost.

Written by 

Rahul Soni is a Software Consultant at Knoldus Software. He is always charged up for new things & learnings. He is dedicated to his work and believes in quality output. He loves to take deep dives into cloud technologies & different tools.