Terraform: Loops with Count and Problems

Table of contents

Reading Time: 6 minutes

In this blog, we are going to expand our Terraform toolbox with some more advanced tips & tricks, such as how to use loops with the count. We’ll also discuss some of Terraform’s weaknesses so we can avoid the most common problems.

Introduction

As we know, Terraform is a declarative language.
Infrastructure-as-code in a declarative language tends to provide a more accurate about deployed items.
It is easier to reason about and makes it easier to keep the code base small.
However, without access to a full programming language, certain types of tasks become more difficult in a declarative language.
For example, since declarative languages typically don’t have for-loops, how do we repeat a piece of logic — such as creating multiple similar EC2 Instances — without copy and paste?
And if the declarative language doesn’t support if-statements, how can we conditionally configure resources, such as creating public IP addresses for frontend services, but not for backend services?
Fortunately, Terraform provides a few primitives—namely, the count meta-parameter, for_each and for expressions.
A lifecycle block called create_before_destroy, a ternary operator, plus a large number of functions—that allow we to do certain types of loops, if-statements, and other logic.
Here are the topics we’ll go over:
- Loops
- Conditionals
- Problems

Loops

In addition it, Terraform offers several different looping constructs:
- count parameter: loop over resources.
- for_each expressions: loop over resources and inline blocks within a resource.
- for expressions: loop over lists and mapsLet’s go through these one at a time.

Count Parameter in terraform

First of all, let us assume, we are creating an IAM user by clicking around the AWS console.

Secondly, we can create and manage all future IAM users with Terraform.

Moreover, consider the following Terraform code:

provider "aws" {
  region = "us-east-2"
}
resource "aws_iam_user" "example" {
  name = "neo"
}

This code uses the aws_iam_user resource to create a single new IAM user.
What if we wanted to create three IAM users?
In a general-purpose programming language, we would probably use a for-loop:

# This is just pseudo code. It won't actually work in Terraform.
for (i = 0; i < 3; i++) {
  resource "aws_iam_user" "example" {
    name = "neo"
  }
}

Terraform does not have for-loops or other traditional procedural logic
As a result, the above syntax will not work.
However, every Terraform resource has a meta-parameter we can use called count.
This is Terraform’s oldest, simplest, and most limited iteration construct: all it does is define how many copies of the resource to create.
Therefore, we can create three IAM users as follows:

resource "aws_iam_user" "example" {
  count = 3
  name  = "neo"
}

One problem with this code is that all three IAM users would have the same name
As a result, this would cause an error, since usernames must be unique.
If we had access to a standard for-loop, we might use the index in the for loop, i, to give each user a unique name:

# This is just pseudo code. It won't actually work in Terraform.
for (i = 0; i < 3; i++) {
  resource "aws_iam_user" "example" {
    name = "neo.${i}"
  }
}

To accomplish the same thing in Terraform, we can use count.index to get the index of each “iteration” in the “loop”:

resource "aws_iam_user" "example" {
  count = 3
  name  = "neo.${count.index}"
}

If we run the plan command on the preceding code, we will see that Terraform wants to create three IAM users, each with a slightly different name (“neo.0”, “neo.1”, “neo.2”):

Terraform will perform the following actions:
  # aws_iam_user.example[0] will be created
  + resource "aws_iam_user" "example" {
      + arn           = (known after apply)
      + force_destroy = false
      + id            = (known after apply)
      + name          = "neo.0"
      + path          = "/"
      + unique_id     = (known after apply)
    }
  # aws_iam_user.example[1] will be created
  + resource "aws_iam_user" "example" {
      + arn           = (known after apply)
      + force_destroy = false
      + id            = (known after apply)
      + name          = "neo.1"
      + path          = "/"
      + unique_id     = (known after apply)
    }
  # aws_iam_user.example[2] will be created
  + resource "aws_iam_user" "example" {
      + arn           = (known after apply)
      + force_destroy = false
      + id            = (known after apply)
      + name          = "neo.2"
      + path          = "/"
      + unique_id     = (known after apply)
    }
Plan: 3 to add, 0 to change, 0 to destroy.

Of course, a username like “neo.0” isn’t particularly readable.
If we combine count.index with some built-in functions from Terraform, we can customize each “iteration” of the “loop” even more.
For example, we could define all of the IAM usernames we want in an input variable called user_names:

variable "user_names" {
  description = "Create IAM users with these names"
  type        = list(string)
  default     = ["neo", "trinity", "morpheus"]
}

If we were using a general-purpose programming language with loops and arrays, we would configure each IAM user to use a different name by looking up index i in the array var.user_names:

# This is just pseudo code. It won't actually work in Terraform.
for (i = 0; i < 3; i++) {
  resource "aws_iam_user" "example" {
    name = vars.user_names[i]
  }
}

Lookup In Array in terraform

In Terraform, we can accomplish the same thing by using count plus two new tricks.
The first trick is to use array lookup syntax to look up elements in an array at a given index. This syntax is identical to what we see in most other programming languages:

LIST[<INDEX>]

For example, here’s how we would look up the element at index 1 of var.user_names:

var.user_names[1]

The second trick is to use Terraform’s built-in function length, which has the following syntax:

length(<LIST>)

Putting all these together, we get:

resource "aws_iam_user" "example" {
  count = length(var.user_names)
  name  = var.user_names[count.index]
}

Now when we run the plan command, we will see that Terraform wants to create three IAM users, each with a completely different name (“neo,” “trinity,” “morpheus”)
Note that once we have used count on a resource, it becomes a list of resources, rather than just one resource.
Since aws_iam_user.example is now a list of IAM users, instead of using the standard syntax to read an attribute from that resource (<PROVIDER>_<TYPE>.<NAME>.<ATTRIBUTE>), we have to specify which IAM user we are interested in by specifying its index in the list using the same array lookup syntax:

<PROVIDER>_<TYPE>.<NAME>[INDEX].ATTRIBUTE

For example, if we wanted to provide the Amazon Resource Name (ARN) of one of the IAM users as an output variable, we would need to do the following:

output "neo_arn" {
  value       = aws_iam_user.example[0].arn
  description = "The ARN for user Neo"
}

If we want the ARNs of all the IAM users, we need to use a splat expression, "*", instead of the index:

output "all_arns" {
  value       = aws_iam_user.example[*].arn
  description = "The ARNs for all users"
}

When we run the apply command, the neo_arn output will contain just the ARN for Neo while the all_arns output will contain the list of all ARNs:

$ terraform apply
(...)
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
all_arns = [
  "arn:aws:iam::123456789012:user/neo",
  "arn:aws:iam::123456789012:user/trinity",
  "arn:aws:iam::123456789012:user/morpheus",
]
neo_arn = arn:aws:iam::123456789012:user/neo

Problems With Count

Unfortunately, count has two limitations that significantly reduce its usefulness.
First, while we can use count to loop over an entire resource, we can’t use count within a resource to loop over inline blocks.
An inline block is an argument we set within a resource of the format:

resource "xxx" "yyy" {
  <NAME> {
    [CONFIG...]
  }
}

where NAME is the name of the inline block (e.g., tag) and CONFIG consists of one or more arguments that are specific to that inline block (e.g., key and value).
For example, consider how are tags are set in the aws_autoscaling_group resource:

resource "aws_autoscaling_group" "example" {
  launch_configuration = aws_launch_configuration.example.name
  vpc_zone_identifier  = data.aws_subnet_ids.default.ids
  target_group_arns    = [aws_lb_target_group.asg.arn]
  health_check_type    = "ELB"
  min_size = var.min_size
  max_size = var.max_size
  tag {
    key                 = "Name"
    value               = var.cluster_name
    propagate_at_launch = true
  }
}

Each tag requires us to create a new inline block with values for key, value, and propagate_at_launch.
The preceding code hardcodes a single tag, but we may want to allow users to pass in custom tags.
We might be tempted to try to use the count parameter to loop over these tags and generate dynamic inline tag blocks.
But unfortunately, using count inside of an inline block is not supported.

The second limitation with count is what happens when we try to change it.
Consider the list of IAM users we created earlier:

variable "user_names" {
  description = "Create IAM users with these names"
  type        = list(string)
  default     = ["neo", "trinity", "morpheus"]
}

Imagine we removed "trinity" from this list. What happens when we run terraform plan?

terraform plan
(...)
Terraform will perform the following actions:
  # aws_iam_user.example[1] will be updated in-place
  ~ resource "aws_iam_user" "example" {
        id            = "trinity"
      ~ name          = "trinity" -> "morpheus"
    }
  # aws_iam_user.example[2] will be destroyed
  - resource "aws_iam_user" "example" {
      - id            = "morpheus" -> null
      - name          = "morpheus" -> null
    }
Plan: 0 to add, 1 to change, 1 to destroy.

Instead of just deleting the "trinity" IAM user, the plan output is indicating that Terraform wants to rename the "trinity" IAM user to "morpheus" and delete the original "morpheus" user.

When we use the count parameter on a resource, that resource becomes a list or array of resources.
Unfortunately, the way Terraform identifies each resource within the array is by its position (index) in that array.
That is, after running apply the first time with three user names, Terraform’s internal representation of these IAM users looks something like this:

aws_iam_user.example[0]: neo
aws_iam_user.example[1]: trinity
aws_iam_user.example[2]: morpheus

When we remove an item from the middle of an array, all the items after it shift back by one.
So after running plan with just two bucket names, Terraform’s internal representation will look something like this:

aws_iam_user.example[0]: neo
aws_iam_user.example[1]: morpheus

Conclusion

Notice how morpheus has moved from index 2 to index 1.
Since Terraform sees the index as a resource’s identity.
To Terraform, this change roughly translates to “rename the bucket at index 1 to morpheus and delete the bucket at index 2.”
In other words, every time we use count to create a list of resources, if we remove an item from the middle of the list, Terraform will delete every resource after that item and then recreate those resources again from scratch.
The end result, of course, is exactly what we requested (i.e., two IAM users named morpheus and neo), but deleting and modifying resources is probably not how we want to get there.
To solve these two limitations, Terraform 0.12 introduced for_each expressions.
We would study for_each expressions in next blog