Kubecost helps you & your FinOps team to monitor and manage the cost and also capacity in Kubernetes environments. It helps you integrate various tools with your infrastructure to help your team track, manage, and reduce the overall cost.
You can monitor & have insights on your Kubernetes spends at a very granular level additionally you can have insights on other cloud resources out of the cluster.
Kubecost Prometheus Metrics exporter exports your Kubernetes cost metrics to the Prometheus server. Then the exported metrics can be visualized in a Grafana dashboard.
This exporter is used in cost monitoring & insights & promotes FinOps practices.
Prerequisites
- Kubernetes Cluster
- Helm v3
- Prometheus Operator
Grafana Dashboard for the Exporter
Setup Prometheus
1 Create custom-values.yaml
This file contains configuration to deploy a Grafana instance & add Prometheus datasource which has metrics from kubecost exporter.
grafana:
enabled: true
additionalDataSources:
- name: kubecost
access: proxy
basicAuth: false
basicAuthPassword: pass
basicAuthUser: daco
editable: false
jsonData:
tlsSkipVerify: true
orgId: 1
type: prometheus
url: "http://prometheus-operated.cost-model.svc.cluster.local:9090"
version: 1
prometheus:
enabled: false
2 Deploy prometheus operator
Run these commands to deploy a Prometheus operator in your Kubernetes cluster. It uses helm to deploy the operator.
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm helm upgrade --install prometheus-operator prometheus-community/kube-prometheus-stack -f custom-values.yaml --namespace prometheus-operator --create-namespace
Deploy Kubecost exporter
1 Create rb-service-account-prometheus.yaml
Create file names rb-service-account-prometheus.yaml and add the below content in it. This will create a service account for the Prometheus pod & binds a role with it in a namespace.
apiVersion: v1
kind: ServiceAccount
metadata:
name: prometheus
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: prometheus
rules:
- apiGroups: [""]
resources:
- nodes
- nodes/metrics
- services
- endpoints
- pods
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources:
- configmaps
verbs: ["get"]
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: prometheus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: prometheus
subjects:
- kind: ServiceAccount
name: prometheus2 Create exporter.yaml file
Place the below content in exporter.yaml file & apply the file to create resources.
---
apiVersion: v1
kind: Namespace
metadata:
name: cost-model
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cost-model
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: cost-model
rules:
- apiGroups:
- ''
resources:
- configmaps
- deployments
- nodes
- pods
- services
- resourcequotas
- replicationcontrollers
- limitranges
- persistentvolumeclaims
- persistentvolumes
- namespaces
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- daemonsets
- deployments
- replicasets
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- statefulsets
- deployments
- daemonsets
- replicasets
verbs:
- list
- watch
- apiGroups:
- batch
resources:
- cronjobs
- jobs
verbs:
- get
- list
- watch
- apiGroups:
- autoscaling
resources:
- horizontalpodautoscalers
verbs:
- get
- list
- watch
- apiGroups:
- policy
resources:
- poddisruptionbudgets
verbs:
- get
- list
- watch
- apiGroups:
- storage.k8s.io
resources:
- storageclasses
verbs:
- get
- list
- watch
---
# Bind the role to the service account
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: cost-model
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cost-model
subjects:
- kind: ServiceAccount
name: cost-model
namespace: cost-model
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: cost-model
labels:
app: cost-model
spec:
replicas: 1
selector:
matchLabels:
app: cost-model
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: cost-model
spec:
restartPolicy: Always
serviceAccountName: cost-model
containers:
- image: quay.io/kubecost1/kubecost-cost-model:latest
name: cost-model
resources:
requests:
cpu: "10m"
memory: "55M"
env:
- name: PROMETHEUS_SERVER_ENDPOINT
value: "http://prometheus-operated.cost-model"
- name: CLOUD_PROVIDER_API_KEY
value: "AIzaSyD29bGxmHAVEOBYtgd8sYM2gM2ekfxQX4U" # The GCP Pricing API requires a key. This is supplied just for evaluation.
- name: CLUSTER_ID
value: "cluster-one" # Default cluster ID to use if cluster_id is not set in Prometheus metrics.
imagePullPolicy: Always
---
kind: Service
apiVersion: v1
metadata:
name: cost-model
labels:
app: kubecost
prometheus: cost-model
spec:
selector:
app: cost-model
type: ClusterIP
ports:
- name: cost-model
port: 9003
targetPort: 9003
---3 Create service-monitor-cost-model.yaml
Create this file and place the below content in the file. Then just apply the file to create a service monitor resource.
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
labels:
prometheus: cost-model
name: cost-model
spec:
endpoints:
- path: /metrics
port: cost-model
selector:
matchLabels:
app: kubecost
prometheus: cost-model4 Create prometheus-custom-resource.yaml
Now, create prometheus resource file & apply it. It deploys a statefulset & headless service for prometheus.
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
labels:
prometheus: cost-model
app: kubecost
name: kubecost
spec:
replicas: 1
scrapeInterval: 5s
securityContext:
fsGroup: 2000
runAsGroup: 2000
runAsNonRoot: true
runAsUser: 1000
serviceAccountName: prometheus
serviceMonitorNamespaceSelector:
matchLabels:
kubernetes.io/metadata.name: cost-model
serviceMonitorSelector:
matchLabels:
prometheus: cost-model
Exporter Deployed
Now we’re done here, exporter is now deployed on Kubernetes with Prometheus instance & Grafana. Now we can visualize & monitor the Kubernetes cluster cost metrics & optimize the cost.
