Learning Terraform and AWS: Configuring Routing & Gateways

The 2nd part in my Virtual Private Cloud with Terraform series of posts.

Hopefully you've read part 1 of this series. At the end of that post we had created a VPC and defined two sets of sub-nets to implement our application on.

This entry will create the routing tables, Internet gateways and security groups needed to access our VPC from the outside world and how the internal sub-nets can communicate with each other.

After initially creating the VPC any instance within the VPC can connect to any other instance but none of the instances have the ability to connect to the outside world. During creation of the VPC a default route table was generated by AWS but it only defines routes within the VPC itself.

Defining host reach-ability goals:
In the VPC diagram below we want to handle normal operations:

  • Everyone on the internet to be able to connect to our application, via our load balancers.
  • To restrict connections to our application servers so that they will only accept traffic from our load balancers.
  • To restrict connections to our database servers so that they will only accept connections from our application servers.
  • Allow our application to send responses to any Internet address. That means that we need to add an Internet Gateway to the VPC.

We also have to consider any additional access that may be required during startup/shutdown of any of the servers. I.E. How do we go from a base linux AMI image and install a webserver or database on it?

First, we'll create an Internet Gateway and attach it to our VPC. We don't want to add a default route to the routing table for the VPC, instead we will add route tables for the sub-nets later.
Doing this with AWS CLI would look like:
#!/bin/bash
# Get the VPC we're working with:
AWS_ENVIRONMENT="Production"
read OBJECTTYPE CIDR DHCPOPTS TENANCY DEFAULT STATE VPC_ID < <(aws ec2 describe-vpcs --filters Name=tag:Name,Values=HoN_CLI_VPC_Demo | head -1)
 
# Create a new Internet Gateway
read OBJECTTYPE IG_ID < <(aws ec2 create-internet-gateway)
aws ec2 create-tags --resources "${IG_ID}" --tags Key=Name,Value=HoN_CLI_Demo_InternetGateway Environment=${AWS_ENVIRONMENT}
 
# Attach the Internet Gateway to the VPC
aws ec2 attach-internet-gateway --vpc-id "${VPC_ID}" --internet-gateway-id "${IG_ID}"

Now lets configure the Route Tables for the publicly accessible sub-nets within the VPC. To do this we need to create 2 new Route Tables (LoadBalancer, WebServer), add a default route to the Internet and associate the sub-nets from Availability Zone A & B to them. The Database sub-nets will need access to the internet for maintenance but not operations. To isolate the Database sub-nets we'll use NAT Gateways, placed into the corresponding WebServer sub-nets. NAT Gateways need Elastic IPs to function so we'll create those too. Finally we'll create a Route Table for each of the Database sub-nets and create default routes pointing to the corresponding NAT Gateways and then associate the Route Tables with the Database sub-nets.
#
# Create Route Tables and Routes for Public subnets (LoadBalancer & WebServer). Then associate these public subnets to the Route Tables.
# *** Private subnets (Database) should be private - inaccessible from Internet ***
#
# Create a new route table for the LoadBalancerPools (A&B).
read OBJECTTYPE LB_RT_ID VPC_ID < <(aws ec2 create-route-table --vpc-id "${VPC_ID}")
aws ec2 create-tags --resources "${LB_RT_ID}" --tags Key=Name,Value=HoN_CLI_Demo_LoadBalancerRT
# Create a new route table for the WebServerPools (A&B).
read OBJECTTYPE WS_RT_ID VPC_ID < <(aws ec2 create-route-table --vpc-id "${VPC_ID}")
aws ec2 create-tags --resources "${WS_RT_ID}" --tags Key=Name,Value=HoN_CLI_Demo_WebServerRT
 
# Create a new default route for the LoadBalancerPools (A&B)
aws ec2 create-route --route-table-id "${LB_RT_ID}" --destination-cidr-block "0.0.0.0/0" --gateway-id "${IG_ID}"
# Create a new default route for the WebServerPools (A&B)
aws ec2 create-route --route-table-id "${WS_RT_ID}" --destination-cidr-block "0.0.0.0/0" --gateway-id "${IG_ID}"
 
# Associate the LoadBalancer Route Table with the LoadBalancer subnets
read OBJECTTYPE JUNK AZ_NAME AVAIL_IPS CIDR DEFAULTFORAZ MAPPUBLIC STATE LBA_SNID VPC_ID < <(aws ec2 describe-subnets --filters Name=vpc-id,Values=${VPC_ID} Name=tag:Name,Values=LoadBalancerPoolA)
read OBJECTTYPE JUNK AZ_NAME AVAIL_IPS CIDR DEFAULTFORAZ MAPPUBLIC STATE LBB_SNID VPC_ID < <(aws ec2 describe-subnets --filters Name=vpc-id,Values=${VPC_ID} Name=tag:Name,Values=LoadBalancerPoolB)
aws ec2 associate-route-table --route-table-id "${LB_RT_ID}" --subnet-id "${LBA_SNID}"
aws ec2 associate-route-table --route-table-id "${LB_RT_ID}" --subnet-id "${LBB_SNID}"
 
# Associate the WebServer Route Table with the WebServer subnets
read OBJECTTYPE JUNK AZ_NAME AVAIL_IPS CIDR DEFAULTFORAZ MAPPUBLIC STATE WSA_SNID VPC_ID < <(aws ec2 describe-subnets --filters Name=vpc-id,Values=${VPC_ID} Name=tag:Name,Values=WebServerPoolA)
read OBJECTTYPE JUNK AZ_NAME AVAIL_IPS CIDR DEFAULTFORAZ MAPPUBLIC STATE WSB_SNID VPC_ID < <(aws ec2 describe-subnets --filters Name=vpc-id,Values=${VPC_ID} Name=tag:Name,Values=WebServerPoolB)
aws ec2 associate-route-table --route-table-id "${WS_RT_ID}" --subnet-id "${WSA_SNID}"
aws ec2 associate-route-table --route-table-id "${WS_RT_ID}" --subnet-id "${WSB_SNID}"
 
#
# Create NAT Internet Gateways for each Availability Zone.
# Create Route Tables and Routes for Private subnets. Then associate private subnets to the Route Tables.
# *** Here we create NAT Gateways on a public subnet in each Availability Zone ***
#
# Create 2 new Elastic IPs (one for each NAT Gateway)
read ELASTIC_IP_ID_A DOMAIN PUBLIC_IP_A < <(aws ec2 allocate-address --domain "vpc" --region ca-central-1)
aws ec2 create-tags --resources "${ELASTIC_IP_ID_A}" --tags Key=Name,Value=HoN_CLI_Demo_ElasticIPAvailabilityZoneA
read ELASTIC_IP_ID_B DOMAIN PUBLIC_IP_B < <(aws ec2 allocate-address --domain "vpc" --region ca-central-1)
aws ec2 create-tags --resources "${ELASTIC_IP_ID_B}" --tags Key=Name,Value=HoN_CLI_Demo_ElasticIPAvailabilityZoneB
# Create 2 NAT Gateways - one in each Availability Zone (inferred by EC2 via the subnet location)
read OBJECTTYPE DATETIME NATGW_ID_A STATE SUBNETID VPCID < <(aws ec2 create-nat-gateway --subnet-id "${WSA_SNID}" --allocation-id "${ELASTIC_IP_ID_A}")
aws ec2 create-tags --resources "${NATGW_ID_A}" --tags Key=Name,Value=HoN_CLI_Demo_NAT_GW_AVAIL_ZONE_A
read OBJECTTYPE DATETIME NATGW_ID_B STATE SUBNETID VPCID < <(aws ec2 create-nat-gateway --subnet-id "${WSB_SNID}" --allocation-id "${ELASTIC_IP_ID_B}")
aws ec2 create-tags --resources "${NATGW_ID_B}" --tags Key=Name,Value=HoN_CLI_Demo_NAT_GW_AVAIL_ZONE_B
# create-nat-gateway is asynchronous. Provisioning NAT Gateways takes a while. This will wait until they are available.
aws ec2 wait nat-gateway-available --nat-gateway-ids "${NATGW_ID_A}" "${NATGW_ID_B}"
 
# Create a new route table for the DatabasePools (A) subnet.
read OBJECTTYPE DBA_RT_ID VPC_ID < <(aws ec2 create-route-table --vpc-id "${VPC_ID}")
aws ec2 create-tags --resources "${DBA_RT_ID}" --tags Key=Name,Value=HoN_CLI_Demo_DatabaseRT_A
# Create a new route table for the DatabasePools (B) subnet.
read OBJECTTYPE DBB_RT_ID VPC_ID < <(aws ec2 create-route-table --vpc-id "${VPC_ID}")
aws ec2 create-tags --resources "${DBB_RT_ID}" --tags Key=Name,Value=HoN_CLI_Demo_DatabaseRT_B
# Create a new default route for the DatabasePools (A) subnet
aws ec2 create-route --route-table-id "${DBA_RT_ID}" --destination-cidr-block "0.0.0.0/0" --gateway-id "${NATGW_ID_A}"
# Create a new default route for the DatabasePools (B) subnet
aws ec2 create-route --route-table-id "${DBB_RT_ID}" --destination-cidr-block "0.0.0.0/0" --gateway-id "${NATGW_ID_B}"
# Associate the Database Route Table with the Database subnets
read OBJECTTYPE JUNK AZ_NAME AVAIL_IPS CIDR DEFAULTFORAZ MAPPUBLIC STATE DBA_SNID VPC_ID < <(aws ec2 describe-subnets --filters Name=vpc-id,Values=${VPC_ID} Name=tag:Name,Values=DatabasePoolA)
read OBJECTTYPE JUNK AZ_NAME AVAIL_IPS CIDR DEFAULTFORAZ MAPPUBLIC STATE DBB_SNID VPC_ID < <(aws ec2 describe-subnets --filters Name=vpc-id,Values=${VPC_ID} Name=tag:Name,Values=DatabasePoolB)
aws ec2 associate-route-table --route-table-id "${DBA_RT_ID}" --subnet-id "${DBA_SNID}"
aws ec2 associate-route-table --route-table-id "${DBB_RT_ID}" --subnet-id "${DBB_SNID}"

Doing this with Terraform would look like:
I've broken the Terraform configuration into files based on the underlying AWS Objects.
provider.tf
provider "aws" {
region = "ca-central-1"
shared_credentials_file = "/home/jardine/Documents/Terraform/permissions"
}

variables.tf
variable "Environment" {
default = "Production"
}
 
data "aws_vpc" "HoN_TF_VPC_Demo" {
filter {
name = "tag:Name"
values = ["HoN_TF_VPC_Demo"]
}
}
 
data "aws_subnet" "LoadBalancerPoolA" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
filter {
name = "tag:Name"
values = ["LoadBalancerPoolA"]
}
}
 
data "aws_subnet" "LoadBalancerPoolB" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
filter {
name = "tag:Name"
values = ["LoadBalancerPoolB"]
}
}
 
data "aws_subnet" "WebServerPoolA" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
filter {
name = "tag:Name"
values = ["WebServerPoolA"]
}
}
 
data "aws_subnet" "WebServerPoolB" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
filter {
name = "tag:Name"
values = ["WebServerPoolB"]
}
}
 
data "aws_subnet" "DatabasePoolA" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
filter {
name = "tag:Name"
values = ["DatabasePoolA"]
}
}
 
data "aws_subnet" "DatabasePoolB" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
filter {
name = "tag:Name"
values = ["DatabasePoolB"]
}
}

internetGateway.tf
resource "aws_internet_gateway" "gw" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
 
tags {
Name = "HoN_TF_Demo_InternetGateway"
Environment = "${var.Environment}"
}
}

RouteTables.tf
resource "aws_route_table" "LoadBalancerRT" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
 
route {
cidr_block = "0.0.0.0/0"
gateway_id = "${aws_internet_gateway.gw.id}"
}
 
tags {
Name = "HoN_TF_Demo_LoadBalancerRT"
}
}
 
resource "aws_route_table" "WebServerRT" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
 
route {
cidr_block = "0.0.0.0/0"
gateway_id = "${aws_internet_gateway.gw.id}"
}
 
tags {
Name = "HoN_TF_Demo_WebServerRT"
}
}
 
resource "aws_route_table" "DatabaseRT_A" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
 
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = "${aws_nat_gateway.DatabaseNAT_GW_A.id}"
}
 
tags {
Name = "HoN_TF_Demo_DatabaseRT_A"
}
}
 
resource "aws_route_table" "DatabaseRT_B" {
vpc_id = "${data.aws_vpc.HoN_TF_VPC_Demo.id}"
 
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = "${aws_nat_gateway.DatabaseNAT_GW_B.id}"
}
 
tags {
Name = "HoN_TF_Demo_DatabaseRT_B"
}
}

RouteTableAssociation.tf
resource "aws_route_table_association" "RouteTableAssociation_LB_A" {
subnet_id = "${data.aws_subnet.LoadBalancerPoolA.id}"
route_table_id = "${aws_route_table.LoadBalancerRT.id}"
}
 
resource "aws_route_table_association" "RouteTableAssociation_LB_B" {
subnet_id = "${data.aws_subnet.LoadBalancerPoolB.id}"
route_table_id = "${aws_route_table.LoadBalancerRT.id}"
}
 
resource "aws_route_table_association" "RouteTableAssociation_WS_A" {
subnet_id = "${data.aws_subnet.WebServerPoolA.id}"
route_table_id = "${aws_route_table.WebServerRT.id}"
}
 
resource "aws_route_table_association" "RouteTableAssociation_WS_B" {
subnet_id = "${data.aws_subnet.WebServerPoolB.id}"
route_table_id = "${aws_route_table.WebServerRT.id}"
}
 
resource "aws_route_table_association" "RouteTableAssociation_DS_A" {
subnet_id = "${data.aws_subnet.DatabasePoolA.id}"
route_table_id = "${aws_route_table.DatabaseRT_A.id}"
}
 
resource "aws_route_table_association" "RouteTableAssociation_DS_B" {
subnet_id = "${data.aws_subnet.DatabasePoolB.id}"
route_table_id = "${aws_route_table.DatabaseRT_B.id}"
}

ElasticIP.tf
resource "aws_eip" "ElasticIP_AZ_A" {
vpc = true
tags {
Name = "HoN_TF_Demo_ElasticIPAvailabilityZoneA"
}
}
 
resource "aws_eip" "ElasticIP_AZ_B" {
vpc = true
tags {
Name = "HoN_TF_Demo_ElasticIPAvailabilityZoneB"
}
}

NAT_Gateway.tf
resource "aws_nat_gateway" "DatabaseNAT_GW_A" {
allocation_id = "${aws_eip.ElasticIP_AZ_A.id}"
subnet_id = "${data.aws_subnet.WebServerPoolA.id}"
 
tags {
Name = "HoN_TF_Demo_NAT_GW_AVAIL_ZONE_A"
}
}
 
resource "aws_nat_gateway" "DatabaseNAT_GW_B" {
allocation_id = "${aws_eip.ElasticIP_AZ_B.id}"
subnet_id = "${data.aws_subnet.WebServerPoolB.id}"
 
tags {
Name = "HoN_TF_Demo_NAT_GW_AVAIL_ZONE_B"
}
}

Stay Tuned for the next section which defines the Load Balancers, Target Groups, Security Groups and Launch Configurations