Managing Terraform State

Table of contents

Reading Time: 4 minutes

In this blog, We are going to learn how Terraform keeps tracks the state of your infrastructure and configuration. With the help of an example, we will learn how can we store state file to a remote location.

We can create infrastructure on a cloud in various ways using CLI, directly using UI or any automation tool like terraform etc. Then how terraform would know about what infrastructure has to create, update or destroy?

Before moving out tracking the state of infrastructure. We learn why there is need of it?
In a large project, we have a different team who work on the same project from different locations. Let suppose one of the team members has created the infrastructure. What if requirements changes and other team member want to modify it or created resources are no longer required at all. How would others run the same script with some changes?

In that case, we need some file or any document that maintains all this information. So that Cloud would know about these are resources created by Terraform and we need to modify only those resources.

Another concern, Okay we can create one file for your tracking. But what if a system crashes. The file would be lost. In that case, how do we modify the resources?
A solution is to store this file to a remote location which would available all the time.

We come up with two benefits storing and using the state file

1. Anyone from a team who had the right access can use it and modify the infrastructure as per demand.
2. And without worry about having issues with the user machine.

Remote State

If we are executing terraform file first time, then a file is created at local on root directory with a name called terraform.tfstate. It is in JSON format. Now whenever there is a change in the script and perform any successive operation to terraform script like

terraform init
terraform plan
terraform apply
terraform destroy

Terraform does refresh the state file and update the information with the change value of real infrastructure (resources).

Locking
If locking is supported by a backend. Then terraform can use remote locking in order to avoid two or more different users running Terraform script at the same time. Terraform will lock your state for all operations that could write state. This prevents others from acquiring the lock and potentially corrupting your state file.

Isolation
This terraform file will be executed in isolation. It means there should be different state file for a different environment (dev, test, stage or production). So that whenever any change in infra. The infra would be affected by a particular environment only.

Using the terraform remote config command, you can configure Terraform to fetch and store state file from a remote location during terraform runs.
There is a different store that can be used to store the state file. Some of them are
Amazon S3, Azure Storage, HashiCorp Consul, and HashiCorp Atlas.

Use Case:

Case 1: When no remote state present :

Two teams from a different location working on the same project. A team member from location A spins up 1 ec2-instance. But now requirement changes and need 1 more instance. In that case, if a member from location B run same terraform script by changing the count to 2. Because team member from location B doesn’t know about there is already one instance present. Since there is no referencing document for the script.
In that case, there will be 3 instances but the requirement is to create 1 more instance.

Now, see another case how remote state file solves this problem

Case 2: When a remote state present :

Here we are first creating 1 ec2-instance of t2.micro type. Then we re-run the infra by increasing count to 2. And see how Terraform manages resource using state file. In this example, we are pushing and pulling the state file from Amazon S3 and assuming bucket is already present on s3.

Instance number: 1

This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters

Show hidden characters

	provider "aws" {
	access_key = "${var.access_key}"
	secret_key = "${var.secret_key}"
	region = "us-west-2"
	}

	terraform {
	backend "s3" {
	bucket = "terraform-remote-state"
	key = "terraform/dev/terraform.tfstate"
	region = "us-east-1"
	}
	}

	resource "aws_instance" "aws-instance" {
	count = 1
	ami = "ami-a0cfeed8"
	instance_type ="t2.micro"
	key_name = "${var.key_name}"

	connection {
	user = "ec2-user"
	private_key = "${file(var.private_key_path)}"
	}

	}

view raw

manage_terraform_state_instance_count_1.tf

hosted with ❤ by GitHub

This image shows what resources going to be created on AWS using terraform plan

This image shows what resources created on AWS using terraform apply

This image is a snippet of remote state file which is in JSON format containing ids of resources created on AWS.

Instance count: 2

Show hidden characters

	provider "aws" {
	access_key = "${var.access_key}"
	secret_key = "${var.secret_key}"
	region = "us-west-2"
	}

	terraform {
	backend "s3" {
	bucket = "terraform-remote-state"
	key = "terraform/dev/terraform.tfstate"
	region = "us-east-1"
	}
	}

	resource "aws_instance" "aws-instance" {
	count = 2
	ami = "ami-a0cfeed8"
	instance_type ="t2.micro"
	key_name = "${var.key_name}"

	connection {
	user = "ec2-user"
	private_key = "${file(var.private_key_path)}"
	}

	}

view raw

manage_terraform_state_instance_count_2.tf

hosted with ❤ by GitHub

This image shows when we run terraform apply again. It refers to the state file and creates 1 more instance rather than creating 2 more instances.