Menu
Article devops

Testing AWS Launch Scripts

Christian Kemper

Christian Kemper

• 3 min read

Just the other day I had to launch AWS instances and bootstrap them automatically without using tools like knife and chef-server. I still wanted to use chef-solo on the server itself to continue the bootstrapping process, there is only so much you would want to do using a shell script…

So how do I get a server up and running quickly and get it to install software and apply configuration itself without me having to log in, install chef-solo, download the cookbooks and run it manually?

Hash bang is your friend.

When you launch an AWS instance you can provide a shell script that runs as root during the bootstrapping process. This script will only run once, ever, when AWS provisions the server. So writing a shell script sounds fairly easy and it isn’t rocket science. I am not necessarily the patient type and I like to get feedback something went wrong sooner rather than later, so launching an instance on AWS only to find out the bootstrap script failed isn’t going to cut the mustard. By the time I run that script I need to know it will just work. So how do you test a shell script during a vanilla server initialisation process?

Let’s virtualise this locally: Vagrant

Vagrant allows you to choose different mechanisms of bootstrapping and just so happens that it supports shell scripts too. Those scripts are also run as root. So, what I need the shell script to do is the following:

  • Install required publicly available packages
  • Install chef-solo
  • Check out private git repo containing cookbooks (I have public/private key access setup for the repo)
  • Configure chef-solo to be able to use encrypted data bags
  • Run chef-solo to finish setting up the server

This is roughly the layout of my git repo with all my chef and vagrant related files.

chef
├── Vagrantfile**
├── aws
│   ├── cloudformation
│   ├── jenkins
│   └── scripts
│       └── basebuild.sh**
├── cookbooks
├── data_bags
├── environments
├── nodes
│   └── basebuild.json**
└── roles

** the files we are interested in right now and where they live…

Assuming that all the other folders contain all the right content we should be good to go.

The contents of basebuild.sh:

#!/bin/sh

cd /root

apt-get update 

apt-get upgrade -y

apt-get install -y ruby1.9.1 ruby1.9.1-dev make git vim tree

#make sure that we use the right versions
update-alternatives --set ruby /usr/bin/ruby1.9.1
update-alternatives --set gem /usr/bin/gem1.9.1

## install chef and ignore the ruby documentation, take way to long and we don't need it here...
gem install chef --version 11.8.2 --no-rdoc --no-ri --conservative

## prepare git deploy key access
mkdir /root/.ssh
echo "L[
....
]=" | base64 -d > /root/.ssh/basebuild
chmod 600 /root/.ssh/basebuild

## create ssh config to use the right key for the right host
echo "Host          bitbucket.org
HostName        bitbucket.org
IdentityFile    ~/.ssh/basebuild" >> /root/.ssh/config

## stop the prompt from asking to accept the host fingerprint...
ssh -o StrictHostKeyChecking=no git@bitbucket.org

## clone the devops repo
git clone git@bitbucket.org:yourorganisation/devops.git

cd /root/devops

## switch to the develop branch, will be changed to master when released
git checkout develop

## add the not so secret key
echo "a...........K" | base64 -d > .chef/the_key

## build chef-solo config for root user
path_prefix=/root/devops
echo "file_cache_path \"/tmp/chef\"
cookbook_path \"$path_prefix/cookbooks\"
role_path \"$path_prefix/roles\"
environment_path \"$path_prefix/environments\"
data_bag_path \"$path_prefix/data_bags\"
encrypted_data_bag_secret \"$path_prefix/.chef/the_key\"" > basebuild.solo.rb

## run the basebuild chef-solo config
chef-solo -c basebuild.solo.rb -j nodes/basebuild.json

#reboot if necessary...
reboot

Now I am no shell expert but this one worked fine for me…

The Vagrantfile should need only one setting added: config.vm.provision :shell, :path => "aws/scripts/basebuild.sh" This tells Vagrant to provision the server using the provided shell script. I am using Ubuntu 12.04 LTS here in case you missed it… so you need to configure Vagrant to download the right box for you. VirtualBox and Parallel Desktop provide plugins and boxes to download. The VMWare plugin will cost you a bit… Both VirtualBox and Parallel Desktop happily bootstrap the instance using this shell script.

To run this setup you need to cd into the directory containing the Vagrantfile and run vagrant up. You’ll see via the log to stdout what is going right or wrong and you can kill the process, destroy the instance, edit your shell and start again until the bootstrapping is successful and your server is what you need it to be.

Once this has completed successfully you are good to go and start up the real thing on AWS, using exactly the same shell script.

Your milage may vary disclaimer bla bla bla…

Feel free to comment and suggest improvements!

Share