How to run packer and terraform with a shell script.

We all are tired of repeating the same steps for creating the image and instance in AWS. But no longer! After reading this blog you will make an image using Packer and an instance through Terraform with help a shell script. You should have prior knowledge of Packer and Terraform. Below, I have briefed them for your reference.

Packer is an open source tool for creating identical machine images for multiple platforms from a single source configuration. You can find more about packer at the official website – Packer doc

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions. A good primer on Terraform can be found here – Terraform Doc

Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon’s proven computing environment.

Before we start make sure that your system ssh public key should be added in the AWS account. To find your system SSH key.

For Ubuntu system:

1. Go to your home directory.
2. Press ctrl+H to see the hidden file.
3. There would be a folder named .ssh.
4. open the file id_rsa.pub.
5. This file contents the public SSH Key.

If that folder does not exist then you can run the following command. This will generate the ssh key.

Command: ssh-keygen

To generate ssh-key for ubuntu system you can refer the following link – DigitalOcean

To generate ssh-key for windows system you can refer the following link – Joyent

To Add SSH key in AWS account

1. Click the username dropDown menu.
2. Select the My Security Credentials. A new window will be open.
3. Select the User item from the menubar on the left side.
4. Select the Security Credentials tag.
5. If score down the page you will find the upload SSH public key button click on the button.
6. Paste your SSH key there.

Project Directory Structure

Let’s get started by creating our project directory structure. In the main project folder, create two folders – packer and terraform. Both folders will have the respective configuration file. The main project folder will three shell scripts which we will create.

-project_folder
     --packer
         ---packer.json
         ---docker_download1.sh
     --terraform
         ---demo_terraform.tf
         ---vars.tf
         ---provider.tf
         ---terraform.tfvar
     --build.sh
     --packer.sh
     --terraform.sh

In the packer folder, we will create a configuration file in JSON format which will be used to define how and what image we want to build. In packer terminology, it is called a template.

Below packer template will create an ec2 image with Docker installed in it.

Explanation

The above template will do the following task.

• create an image in AWS in us-east-1 region with the image name as defined by the environment variable.
• Builder will build the ubuntu machine with some external memory.
• Through provisioner key in JSON, we will install docker in AWS image.
• docker_download1.sh script is the script through which docker is installed in the image.

Terraform

Now its time to create the instance of the image through terraform. The set of files used to describe infrastructure in terraform is simply known as a terraform configuration.

Explanation

The above configuration will do the following task.

• Data key of the terraform script will help us to find the ami_id of the image. In data key, there is a filter attribute that will filter the ami on the bases of the ami_name.
• Resource “aws_instance bootstrap” will create the instance of the image that is filtered by the data key. Ami attribute value is given by the data key.
• The connection attribute will make a connection with the new instance that is created through ssh key in it we mention the private key file path and the username of the remote system.
• Resource “aws_key_pair mykey” mention the key pair name and public key file path which is used to make a secure connection.

To make it work we have to do one little work. As we have made the AWS instance through packer it acts as a normal machine. So if we want to install the docker in the AWS instance writing in the script is not enough. We have to make a connection to the instance then only we can install docker on the machine. To make the connection we have to add the security group in AWS.

To Add Security Group

1. Login to your AWS account.
2. Go to Services drop down to select the EC2.
3. Left menuBar Under the heading NETWORK & SECURITY selects the Security Group. 
4. Go to Inbound tag click the edit button a pop box will open.
5. Select Add Rules.
6. In Type tag, select the All TCP  item.
7. In Source tag, selects My IP.
8. Click the save button.

In vars.tf file we mention the variable’s value for terraform configuration file.

Now is the time for the build script.

We will make one script called build.sh that will call two other scripts that will run the packer and terraform command. This script will run through two command line arguments – access key and secret key.

In line number 18 we are replacing default value of image name that is given vars.tf file. This is the name of which we have made the AWS image.
In line number 24 we again replacing the value of image_name with ami_image_name so that line number 18 can run again.
In line number 20 packer script is called with two command line parameter that is received by the build script.

packer script

The above script will run successfully if two command line arguments are given, otherwise, it will get terminated with a message “environment variable AWS_PROFILE or access keys (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) to be set“. From line number 20 to 23 packer build command is written that will call our above packer template.

Finally, the terraform script.

In line number 2, the command is for ssh key generation. This will create the ssh public key and private key to make the remote connection. The path for these keys has to be mentioned in vars.tf file.

Our work is done! We made the image and its instance using packer and terraform.

This is how we dynamically create our image and instance using the script. To run the build script again we have to change the image_name in the build.sh file as AWS won’t create an image with the same name.

This method has an advantage over creating an image using amazon interface as it requires repetitive manual tasks. However, with help of these scripts, we can create multiple instances easily.

To see the full code you can refer to the git link – Packer-terraform-project

For this blog, I referred to the following sites.

• Terraform documentation
• Packer documentation
• Amazon web service
• DigitalOcean
• Joyent