awscli

Using Ansible Roles to Create a Scenario 2 VPC in AWS

In my last post, I talked about a set of CloudFormation templates I created to quickly and flexibly create/teardown a securely configured Scenario 2 VPC. As an experiment, I decided to see if I could create an Ansible role to do the same thing. The experiment was mostly successful but I ran into some complications.

Ansible’s cloud module support for AWS is pretty comprehensive for most use cases as of this writing (I am using version 2.2.1 that has been recently updated from the v2.2 origin). Still, I discovered a couple of gaps compared to my CloudFormation solution.

First of all, unlike CloudFormation, Ansible doesn’t allow for complete automated deprovisioning of VPC components. When you use Ansible to create AWS resources like a VPC, things are relatively straightforward. However, removal of those resources has to be orchestrated (unlike CloudFormation which handles the stack teardown in a completely automated fashion) just like the creation phase requires. Order of resource deprovisioning matters and you will go through some trial and error to figure out what works for your configuration.

One major issue I ran into with my VPC deployment is that the Ansible module for managing route tables, ec2_vpc_route_table, does not seem to remove route table objects after they’ve been created in a Scenario 2 VPC configuration. Typically, when you specify the attribute “state: absent” in an action, a module will decommission the resource. In this case, I found that I had to fall back to using the shell module to run awscli commands to delete the route tables. A quick perusal of the Ansible GitHub issue queue for extras modules didn’t suggest a workaround nor were there any pull requests related to the problem (note to self: file a bug report). Here’s the relevant section of my delete.yml playbook for VPC removal:

- name: Delete private subnet in AZ2
  ec2_vpc_subnet:
    state: absent
    vpc_id: "{{ vpc_id }}"
    cidr: "{{ private_subnet_az2_cidr }}"

# For some reason, ec2_vpc_route_table won't delete these
# but we'll keep them in here commented out for posterity.
#
#- name: Delete public route table
#  ec2_vpc_route_table:
#    state: absent
#    vpc_id: "{{ vpc_id }}"
#    route_table_id: "{{ public_rt_id }}"
#  ignore_errors: yes
#
#- name: Delete AZ1 private route table
#  ec2_vpc_route_table:
#    state: absent
#    vpc_id: "{{ vpc_id }}"
#    route_table_id: "{{ private_rt_az1_id }}"
#  ignore_errors: yes
#
#- name: Delete AZ2 private route table
#  ec2_vpc_route_table:
#    state: absent
#    vpc_id: "{{ vpc_id }}"
#    route_table_id: "{{ private_rt_az2_id }}"
#  ignore_errors: yes
#
# Instead, we will decomm route tables using awscli

- name: Delete public route table via awscli
  shell: aws ec2 delete-route-table --route-table-id "{{ public_rt_id }}"

- name: Delete AZ1 private route table via awscli
  shell: aws ec2 delete-route-table --route-table-id "{{ private_rt_az1_id }}"

- name: Delete AZ2 private route table via awscli
  shell: aws ec2 delete-route-table --route-table-id "{{ private_rt_az2_id }}"

- name: Delete Internet Gateway
  ec2_vpc_igw:
    vpc_id: "{{ vpc_id }}"
    state: absent

ec2_vpc_route_table is a relatively new “extras” module, appearing in the v2.0.0 release so it is likely to be fixed down the road, but be aware of this issue, especially in the context of a Scenario 2 VPC deployment (I have a hunch the NAT Gateway may be related to the problem…).

Another issue I encountered, albeit a minor one, is lack of support for VPC Endpoints, which my Scenario 2 CloudFormation templates support. There is an open PR for this functionality, so it may be available soon.

Aside from these problems, I found using Ansible plays for generating a Scenario 2 VPC to be relatively simple (like anything you do with Ansible) and useful. You can download the role I created via Galaxy:

$ ansible-galaxy install rcrelia.aws-vpc-scenario2

or clone the GitHub repository I created for the Galaxy integration:

$ git clone git@github.com:rcrelia/aws-vpc-scenario2.git

Until next time, have fun getting your Ansible on!

 

 

Confusing syntax error with AWS CLI validation of CF templates

When using awscli to create CloudFormation stacks, there’s a pre-create step of template validation that checks for template syntax issues:

aws cloudformation validate-template --template-body file:///path/to/template.json

The URI prefix file:// indicates that we are using local templates while templates at web-accessible locations like your S3 bucket are accessed using the –template-url option. For more information see the awscli docs or CLI help:

aws cloudformation validate-template help

For local templates, make sure you don’t forget the file:// URI and try to refer to the template via normal filesystem paths, otherwise you’ll get a confusing syntax error.

Without file:// URI:

aws cloudformation validate-template --template-body /Users/foo/git-repos/aws-mojo/cloudformation/aws-deploy-codecommit-repo.yml

An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: unsupported structure.

That’s a rather unhelpful error message that makes me think I’ve got some sort of template content error.

When we run the same command with file:// URI, we get the expected output:

aws cloudformation validate-template --template-body file:///Users/foo/git-repos/aws-mojo/cloudformation/aws-deploy-codecommit-repo.yml
{
   "Description": "CloudFormation template for creating a CodeCommit repository along with a SNS topic for repo activity trigger notifications",
   "Parameters": [
 {
   "NoEcho": false,
   "Description": "Email address for SNS notifications on repo events",
   "ParameterKey": "Email"
 },
 {
   "NoEcho": false,
   "Description": "A unique name for the CodeCommit repository",
   "ParameterKey": "RepoName"
 },
 {
   "DefaultValue": "Dev",
   "NoEcho": false,
   "Description": "Environment type (can be used for tagging)",
   "ParameterKey": "Environment"
 },
 {
   "NoEcho": false,
   "Description": "A description of the CodeCommit repository",
   "ParameterKey": "RepoDescription"
 }
 ]
}