May 11 2015

Using Ansible to create AWS instances

by Maciej Jaworski

Ansible
Ansible is a great tool for enhancing productivity. With a vast array of modules to choose from, it can save you a lot of time by automating away common tasks. At Tivix we use it for single-command deployment, with the most common destination being Amazon EC2 instances created beforehand. Since Ansible is capable of managing EC2 resources, we can improve this setup by making a playbook to create an instance for us.

EC2 modules for Ansible require the boto python library to work

The only other prerequisite is setting up 2 environment variables:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

Our tasks will use a couple of variables:

env: "staging" # env name, this will be used as an inventory file name
app_code_user: "ubuntu" # remote user
aws_region: eu-central-1 # AWS region, where instance will be created
instance_type: t2.micro  # AWS instance type
ami: ami-b83c0aa5  # AMI id, example uses Ubuntu 14.04.1 LTS

With that completed we can write the first task, which will create the security group for our instance:

-   name: Create security group
    ec2_group:
        name: "{{ project_name }}_security_group"
        description: "{{ project_name }} security group"
        region: "{{ aws_region }}"
        rules:
            - proto: tcp
              type: ssh
              from_port: 22
              to_port: 22
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              type: http
              from_port: 80
              to_port: 80
              cidr_ip: 0.0.0.0/0
            - proto: tcp
              type: https
              from_port: 443
              to_port: 443
              cidr_ip: 0.0.0.0/0
        rules_egress:
            - proto: all
              type: all
              cidr_ip: 0.0.0.0/0
    register: basic_firewall

We allow ssh, http, and https traffic, then register the output of that command for later use. After that we need to create a new ssh key pair to use for logging in.

-   name: Create an EC2 key
    ec2_key:
        name: "{{ project_name }}-{{ env }}-key"
        region: "{{ aws_region }}"
    register: ec2_key

After we save the private key contents to a file (this should never be committed into VCS, I'd recommend updating the ignore list to include *.pem files) to use later when we log in to add public developer keys to the instance.

-   name: save private key
    copy: content="{{ ec2_key.private_key }}" dest="./aws-{{ env }}-private.pem" mode=0600
    when: ec2_key.changed

Now we can create an EC2 instance.

-   name: Create an EC2 instance
    ec2:
        key_name: "{{ project_name }}-{{ env }}-key"
        region: "{{ aws_region }}"
        group_id: "{{ basic_firewall.group_id }}"
        instance_type: "{{ instance_type }}"
        image: "{{ ami }}"
        wait: yes
        instance_tags:
            env: "{{ env }}"
        count_tag: env
        exact_count: 1
    register: ec2

After that we save the instance IP as a new inventory file. Ansible Dynamic Inventory could be used to access IPs dynamically, but would require setting up  AWS_ACCESS_KEY_ID and  AWS_SECRET_ACCESS_KEY every time the deployment playbook is run, which is inconvenient. We also create a new group, which we will be using in the task to copy developers' public keys.

-   name: save IP to inventory file
    copy: content="[webservers]{{'\n'}}{{ item.public_ip }}" dest=./{{ env }}
    with_items: ec2.tagged_instances

-   name: Add IP to ec2_hosts group
    add_host: hostname={{ item.public_ip }} groups=ec2_hosts
    with_items: ec2.tagged_instances

This is the end of the create task. We'll need another one to add developer keys to the instance.

-   name: Make sure user is on server and generate ssh key for it
    user: name={{ app_code_user }}
        generate_ssh_key=yes

-   name: Add public keys for developers
    authorized_key: user={{ app_code_user }}
                    key="{{ lookup('file', item) }}"
    with_fileglob:
        - ../public_keys/*.pub

First we make sure the user is on the server, then we update  authorized_keys for the user with all the developer keys found in the  public_keys directory. This is an example directory structure of the  users role:

├── users
│   ├── public_keys
│   │   └── bruce-wayne.pub
│   │   └── tony-stark.pub
│   │   └── clark-kent.pub
│   └── tasks
│       └── main.yml

Finally we create the playbook, where we run previously-created tasks.

-   name: Create AWS instance
    hosts: 127.0.0.1
    connection: local
    gather_facts: False
    remote_user: ubuntu
    roles:
        - create

-   name: Add user keys
    hosts: ec2_hosts
    sudo: yes
    sudo_user: root
    remote_user: ubuntu
    roles:
        - users

After you verify everything is working properly (you can ssh into the instance, etc.), the private key can be deleted and the inventory file created by the role can be committed to VCS so everyone working on the project can deploy to new instance.

This is a simple configuration that creates a single instance. EC2 modules for Ansible have many more possible uses, like batch creation of multiple instances, allocating RDS resources, etc. I highly recommend taking the time to learn and set up Ansible in your projects, as the initial time investment ends up paying for itself fast.

Want more? Head back to the Tivix blog