Use Infracost CI Template for Gitlab To forecast Cost

Infracost is an open-source tool used to forecast & estimates your cloud cost on every pull request on terraform.

Multiple scenarios & scripts can be created to forecast cloud costs. It supports AWS, Azure, GCP cloud platforms & over 230 Terraform resources.

It also works with Terraform Cloud & Terragrunt. Infracost can use hosted Cloud Pricing API or self-host.

Infracost can be integrated with any CICD tool which will break down the cost of new terraform resources every time a Pull request or Merge request is created. In this blog, we will see how we can use Gitlab CI templates for the merge request pipeline to estimate & forecast cloud costs.

Prerequisites

• Gitlab knowledge
• Gitlab Repo
• CICD variables
  • INFRACOST_API_KEY: Contains API Key from https://dashboard.infracost.io/ organization
  • GITLAB_TOKEN: Contains Gitlab personal token for commenting on MRs.

Directory Structure

The directory structure for your application or pipeline repo should look like this. Command to check directory structure is tree -I .git -a

tree -I .git -a

The output will look like this:

.
├── .gitlab
│   └── plan-json.yml
├── .gitlab-ci.yml
├── README.md
└── terraform
    ├── .infracost
    │   └── terraform_modules
    ├── main.tf
    └── README.md

Steps to create job template

1. Create .gitlab-ci.yml file in the main directory. The main Gitlab pipeline is defined in .gitlab-ci.yml file.

This acts as parent job which triggers a downstream pipeline that is called the child pipeline.

stages:
- all_stage

mr-gitlab-terraform:
  stage: all_stage
  rules:
  - if: "$CI_MERGE_REQUEST_IID"
  - if: "$CI_COMMIT_TAG"
  - if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
  trigger:
    include: ".gitlab/plan-json.yml"
    strategy: depend

2. Create GitLab merge request job template.

Create a file plan-json.yml & copy the below content in it. This will act as downstream pipeline triggered by the parent pipeline.

workflow:
  rules:
  - if: "$CI_MERGE_REQUEST_IID"
  - if: "$CI_COMMIT_TAG"
  - if: "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"
  

variables:
  # If your terraform files are in a subdirectory, set TF_ROOT accordingly
  TF_ROOT: terraform

stages:
  - plan
  - infracost

cache:
  key: "${TF_ROOT}"
  paths:
    - ${TF_ROOT}/.terraform

plan:
  stage: plan
  image:
    name: hashicorp/terraform:latest
    entrypoint: [""]
  before_script:
    - cd ${TF_ROOT}
    - terraform init
  script: 
    - terraform plan -out tfplan.binary
    - terraform show -json tfplan.binary > tplan.json
  artifacts:
    paths:
      - ${TF_ROOT}/tplan.json

infracost:
  stage: infracost
  image:
    name: infracost/infracost:ci-0.10
    entrypoint: [""] 
  dependencies:
    - plan
  script:
    - git clone $CI_REPOSITORY_URL --branch=$CI_MERGE_REQUEST_TARGET_BRANCH_NAME --single-branch /tmp/base
    - ls /tmp/base
    - infracost configure set api_key $INFRACOST_API_KEY
    - infracost breakdown --path=/tmp/base/${TF_ROOT} --format=json --out-file=infracost-base.json
    - INFRACOST_ENABLE_CLOUD=true infracost diff --path=${TF_ROOT} --compare-to=infracost-base.json --format=json --out-file=infracost.json
    - infracost comment gitlab --path=infracost.json --repo=$CI_PROJECT_PATH --merge-request=$CI_MERGE_REQUEST_IID --gitlab-server-url=$CI_SERVER_URL --gitlab-token=$GITLAB_TOKEN --behavior=update

  variables:
    INFRACOST_API_KEY: $INFRACOST_API_KEY
    GITLAB_TOKEN: $GITLAB_TOKEN 

Explanation: First this downstream pipeline creates terraform plan in JSON from the current branch. Then it plans out in JSON but from the target branch that is main in this case. Then it compares both terraform plans & breakdowns the cost difference and finally comments on the merge request.

3. Now whenever you create a MR in Gitlab it will forecast terraform infrastructure cost for you

1. Checkout a branch from Master
2. Make some changes in terraform file
3. Push the newly created branch
4. Create an MR on the base main branch
5. The pipeline will automatically trigger as soon as MR is created.

When pipeline succeeds It will comment down on your MR with cost estimation & breakdowns

Usage

• Use extends: keyword to extend this job template.
• (Optionally) You can keep this stage as it is and apply rules to control the behavior of the job.
• Put your terraform code into terraform/ dir.
• Create a MR on your GitLab repo.

Conclusion

Infracost is used to forecast cloud infra costs before it creates any resources. You can integrate this tool with any CICD to forecast cost whenever a pull or merge request is created.