Developing Ansible Playbooks for Arch Linux with Vagrant

I’m a big fan of automated configuration management software, and an even bigger fan of utilizing Vagrant for developing configuration modules/cookbooks/states/playbooks/whatever in a fast and easily reproducible environment. I previously created Puppet Sandbox for just this purpose, but have more recently taken an interest in using Ansible for configuration and orchestration.

I also have a long history of working with Arch Linux, and wanted to develop Ansible playbooks specifically for managing Arch machines. Vagrant supports automatically provisioning machines via Ansible out of the box, but there were still a couple of hurdles to get over:

  1. Up-to-date Vagrant base boxes for Arch are hard to find.
  2. Arch Linux doesn’t have Python 2 installed by default, which is a dependency for Ansible.

Packer Arch

To solve the first problem, I decided to create a generic Arch Linux base box myself. In the not too distant past, the way to do that in a repeatable fashion was Veewee, but the project has gotten progressively more complicated to set up and use. Lucky for me, there’s a new kid on the block for creating machine images named Packer, built and maintained by the author of Vagrant, Mitchell Hashimoto.

To make a long story short, I wrote Packer Arch, which is a bare bones Packer template and installation script that can be used to quickly generate Vagrant base boxes for Arch Linux. My goal with the box was to be as minimal as possible, and to roughly duplicate what you’d get when purchasing an Arch Linux VPS from a provider like DigitalOcean. Starting from that point, I wanted to configure everything else via Ansible.

Bootstrapping the Virtual Machine

Solving the Python 2 problem was a little trickier. Ansible itself provided a possible solution with their raw module, but Vagrant’s provisioning integration with Ansible requires Python 2 to be on the base box before you can run any playbooks. It’s the classic “chicken or the egg” problem.

Since utilizing Ansible for configuration as well as orchestration tasks was desirable, but would require having a proper setup outside of Vagrant anyway, I just decided to ignore Vagrant’s provisioner altogether. Instead, I wrote a short script to handle the one-time tasks so I could interact with the VM using Ansible in the exact same fashion as I would any other server.

Preparation

The prerequisites for running the bootstrap script include assigning a known IP address to the machine via the Vagrantfile:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "arch"
  config.vm.network :private_network, ip: "192.168.111.222"
end

…recording that same IP in an inventory file named hosts for Ansible to reference:

[vagrant]
192.168.111.222

…and finally pointing Ansible to the correct Python binary by creating a group_vars/all file containing:

---
# Variables listed here are applicable to all host groups.

ansible_python_interpreter: /usr/bin/python2

The Script

On top of installing Python 2, I have the bootstrap script handle a few other items for convenience:

  1. Create my user account and grant it full sudo privileges.
  2. Add my SSH public keys to the newly created account.
  3. Download a current package mirrorlist based on my geography.

The user management steps are handled by running the tasks tagged “bootstrap” from my regular master playbook, and the mirrorlist is downloaded and then transferred to the machine via the copy module.

Without further ado, here’s the bootstrap script:

#!/usr/bin/env bash

export ANSIBLE_HOSTS="${PWD}/hosts"
export ANSIBLE_HOST_KEY_CHECKING='False'

vagrant up
ansible vagrant -m raw -a 'pacman -Sy --noconfirm python2' --user=vagrant --private-key="${HOME}/.vagrant.d/insecure_private_key" --sudo
ansible-playbook site.yml --tags=bootstrap --user=vagrant --private-key="${HOME}/.vagrant.d/insecure_private_key" --sudo

COUNTRY='US'
URL="https://www.archlinux.org/mirrorlist/?country=${COUNTRY}&protocol=http&ip_version=4&use_mirror_status=on"

if /usr/bin/curl --silent --fail --output mirrorlist "${URL}"; then
    case $OSTYPE in
        darwin*)
            /usr/bin/sed -i '' 's/#Server/Server/g' mirrorlist
            ;;
        linux*)
            /usr/bin/sed -i 's/#Server/Server/g' mirrorlist
            ;;
    esac
    ansible vagrant -m copy -a 'src=mirrorlist dest=/etc/pacman.d/mirrorlist owner=root group=root mode=0644 backup=yes' --user=vagrant --private-key="${HOME}/.vagrant.d/insecure_private_key" --sudo
    rm mirrorlist
fi

echo "export ANSIBLE_HOSTS=${PWD}/hosts"
echo 'export ANSIBLE_HOST_KEY_CHECKING=False'
echo

Once the script runs, I paste the environment variable export lines that it echos into my shell. This makes Ansible purposefully ignore SSH host key checking; since the VM is transient, we don’t need to permanently store its key.

Achievement Unlocked!

From there on out, all of the Ansible modules work as expected and no longer require connecting as the vagrant user:

$ ansible vagrant -m ping
192.168.111.222 | success >> {
    "changed": false,
    "ping": "pong"
}

To give you an idea of the actual playbooks I’m using with this setup, take a look at my Monarch project, and in particular, the users.yml file under the common role. As always, let me know if you need any help putting all of the pieces together.

Use sshuttle to Keep Safe on Insecure Wi-Fi

If you’ve ever used an insecure Wi-Fi connection in a public place (think coffee shop, airport, etc.), your personal information could be at risk. Anyone on the local network could potentially be monitoring your traffic and they could even hijack your sessions to control accounts that you’ve logged into.

Every week I meet up with some friends to work on personal projects, and the venue only offers an open wireless connection with no encryption. To mitigate the risk of leaking personal information, I’ve tried a handful of strategies and eventually settled on using sshuttle.

What is sshuttle?

The sshuttle utility is part transparent proxy, part Virtual Private Network (VPN), and part SSH. It solves a lot of common problems with encrypting your traffic, and it does so in a very efficient way.

There’s no need for a complicated pre-existing infrastructure. All you need is Python 2.x, and a remote machine you can SSH to that also has Python installed. You don’t even have to be an administrator on the remote machine. The project’s README has a lot more information on its theory, design, and alternate uses.

Previous Method

As an aside, I had previously relied on creating a SOCKS proxy with SSH, and then tunneling my web traffic through it…

$ ssh -fqND 1080 example.com

But, you have to adjust your machine’s proxy configuration settings each time you set up the tunnel, and then tear it all down when you’re done. This method will not force absolutely all traffic over the tunnel, as things like DNS lookups and a lot of command-line tools will just ignore the proxy. You’re also encapsulating TCP-over-TCP, which can cause performance problems.

I briefly messed with the Sidestep project to automate these setup/teardown steps on OS X, but ran across repeated authentication issues and wasn’t overly impressed with its developer. Overall, I needed a more elegant solution.

Installation

I wanted to use sshuttle on OS X Lion, and even though it’s primarily a command-line utility, it also comes with a graphical user interface wrapper that will sit in the menu bar. If you just want the command-line script and already use Homebrew, brew install sshuttle is the way to go. If you don’t use Homebrew and/or want the GUI, I’ve got your back…

Building sshuttle from Source

Unfortunately, pre-compiled binaries are not made available for every release, and documentation for building sshuttle is non-existent. The project uses a build system called ‘redo’ instead of the more-customary ‘make’, so the process is a bit different than what most people are used to.

Luckily, the sshuttle repository includes a shell script named ‘do’, which is a simplified version of redo, so you won’t need to install any additional build software. The only real dependencies are Python and two Python modules to generate the man page (feel free to use pip install instead of easy_install, if you prefer):

$ easy_install markdown BeautifulSoup

Once you’ve installed those two modules, you’re all set to build…

NOTE: You may have to adjust paths or permissions slightly depending on your machine’s setup.

$ mkdir -p /usr/local/libexec
$ git clone git://github.com/apenwarr/sshuttle.git /usr/local/libexec/sshuttle
$ cd /usr/local/libexec/sshuttle/
$ ./do all

Once sshuttle is built, we just make symlinks so the binary and man page are in the proper places:

$ ln -s /usr/local/libexec/sshuttle/sshuttle /usr/local/bin
$ mkdir -p /usr/local/share/man/man8
$ ln -s /usr/local/libexec/sshuttle/Documentation/sshuttle.8 /usr/local/share/man/man8

For the GUI app, we need to create an alias instead a symlink, because Spotlight will not index symlinks. Yes, you could just copy Sshuttle VPN.app directly into /Applications, but then you’d have to remember to copy it every time you update/rebuild sshuttle.

Without getting too technical, aliases are a lot like symlinks but they use extra metadata called “resource forks” to follow a file (even when it’s moved) rather then blindly pointing to a path. More importantly, Spotlight will index them. There’s no way to create an alias from the command line, so you have to use Finder:

  1. Open Finder and press Command-Shift-G to open the “Go to the folder:” dialog box.
  2. Type /usr/local/libexec/sshuttle/ui-macos in the box and click the Go button.
    Go to the folder: Dialog Box
  3. Click and hold Sshuttle VPN.app, then while holding down both the Command and Option keys, drag the app over to Applications on the left and release the mouse button.
    Create the sshuttle Alias

Configuration

Due to some sysctl changes in OS X Lion, you will have to reboot your machine after you run sshuttle for the first time. An adjustment to the net.inet.ip.scopedroute kernel flag needs to be made, and it can no longer be done during runtime. This is required so forwarding rules will be honored by the firewall.

Everything should be handled automatically by sshuttle, but if your curious to see the change, you can display the boot kernel flags and current settings with the following commands:

$ defaults read /Library/Preferences/SystemConfiguration/com.apple.Boot
{
    "Kernel Flags" = "";
}
$ sysctl -a | grep scopedroute
net.inet.ip.scopedroute: 1
net.inet6.ip6.scopedroute: 1

After you run sshuttle for the first time and then reboot the machine, you can confirm the updated status:

$ defaults read /Library/Preferences/SystemConfiguration/com.apple.Boot
{
    "Kernel Flags" = "net.inet.ip.scopedroute=0";
}
$ sysctl -a | grep scopedroute
kern.bootargs: net.inet.ip.scopedroute=0
net.inet.ip.scopedroute: 0
net.inet6.ip6.scopedroute: 1

Usage

Finally we can get down to actually using sshuttle! It’s flexible enough to do fancier things, but for our particular use case, we just need to forward all traffic over the tunnel.

The basic command to achieve our goal looks like this:

$ sshuttle --dns -r example.com 0/0

NOTE: Sshuttle will honor your ~/.ssh/config connection settings, but you can also manually specify a different username/port, if necessary:

$ sshuttle --dns -r username@example.com:2222 0/0

Run that, and all of your traffic (including DNS requests) will be transparently proxied through an SSH connection to example.com. You can verify this by browsing to https://ifconfig.me.

To stop forwarding traffic, just press Ctrl-c back in the terminal. We can do a bit better though by forking the process into the background so we don’t tie up our terminal session. These are the aliases I use to make setting up and tearing down the tunnel easier:

alias tunnel='sshuttle --dns --daemon --pidfile=/tmp/sshuttle.pid --remote=example.com 0/0'
alias tunnelx='[[ -f /tmp/sshuttle.pid ]] && kill $(cat /tmp/sshuttle.pid) && echo "Disconnected."'

The Sshuttle VPN.app GUI is pretty self-explanatory, just make sure to enable the Send DNS requests through this server checkbox, and select Send all traffic through this server for the “Network routes:” option.

Known Bugs

You may see a bunch of “warning: closed channel …” messages when running sshuttle (either on STDOUT or in your system.log), but these warnings are safe to ignore. The developer knows about the issue and is thinking of the best way to suppress/eliminate the condition.

Also, prior to version 0.60 of sshuttle, there was a bug in the GUI application when disconnecting tunnels, where the controlling terminal would disappear before firewall rules could cleaned up. This would leave the network in an odd state and all DNS lookups would fail. It should be fixed now, but if you experience networking trouble after disconnecting a sshuttle tunnel, you can see the current firewall rules using:

$ sudo ipfw list

…and you can manually flush out those rules to fix connectivity by running:

$ sudo ipfw -f flush

Be aware though, this will flush out all rules, not just the rules set by sshuttle, so it may have unintended consequences. If you’re unsure, you can always reboot the machine to get connectivity back to normal.

Update sshuttle

Since our installation of sshuttle is managed via git, updating to the latest version is pretty straightforward:

$ cd /usr/local/libexec/sshuttle
$ git pull
$ ./do all

Conclusion

Admittedly, sshuttle takes a bit more work than other solutions to get up and running, but the security it provides gives me peace of mind when forced to use insecure Wi-Fi networks.

If you have any issues, there is an active mailing list for the project, or you can always send me a note and I’ll see what I can do to help.

Shelf-Made Standing Desk

Since I started working from home, I’ve found that I spend even more time sitting at my desk than I did when I worked at the office. I’ve never had great posture and started noticing that my back and shoulders were pretty achy by the end of the day. Once I got to that point, it was harder to stay focused on work and I was becoming noticeably fatigued.

I started researching alternative workspaces, and found ideas ranging from sitting on an exercise ball to walking on a treadmill desk. Having sat on exercise balls before, I knew that I would quickly regress back to slouching; and the idea of working on a treadmill seemed too impractical. I needed something in between…

A Standing Desk

There seems to be a surge of people converting to standing desks recently, and there’s no shortage of personal stories and information on why they can be beneficial. Some very nice commercially-made standing desks are available, but not knowing if I’d actually like using one, I didn’t want to make a big up-front investment.

My Old Desk

For reference, my old desk was simply a cubicle desk surface that had been reclaimed from a dumpster. I always liked the amount of horizontal space it provided, and it was extremely solid, but not completely ideal.

My Old Sitting Desk

My New Desk

I decided to convert a cheap shelving unit into a standing desk as a way to test the waters. At 48"W x 24"D x 72"H, this $80 rack from Lowes was pretty ideal, and I figured that after a month if I didn’t like it, the shelf could always be used in my garage.

Muscle Rack

I tried to follow OSHA guidelines as closely as possible when setting up the height of each shelf. The bottom shelf is about 7" above the ground to give my feet ample space below the rack; the upper shelf was placed so the top of the displays are right about eye-level; and the middle shelf was placed just below my elbow height for the keyboard and mouse. I used the extra shelf (which would have been on the very top) as a keyboard “tray” by cutting some notches in the corners to create a ledge that sticks out about six inches in the front.

Keyboard Tray Wrapped with Contact Paper

Some friends pointed out that MDF could potentially contain Formaldehyde in the resin used to bind it together, and that prolonged contact might cause skin irritation. To prevent this, I wrapped the front part of the keyboard tray in faux leather contact paper. It looks nice, and prevents my wrists from getting itchy.

Here’s how everything came together…

Complete Standing Desk

So my hands always stay at the proper height, I use Synergy to control both my Linux desktop and my MacBook Pro laptop with the same keyboard and mouse.

Complete Standing Desk with Me
Complete Standing Desk using Footrest

The medical community points out that staying in one position for too long (whether that’s sitting or standing) can be bad for you, so there’s also a futon in my office where I can switch to using my laptop with a Logitech Comfort Lapdesk. Eventually, I may order a tall stool, so the option to sit at the desk will also be available; although, I do plan on standing for the majority of the time.

Accessories

To make the standing desk more ergonomic, I purchased a few accessories:

Here’s an idea of roughly how the keyboard tray looks without the LED strip light installed; and if you mouse over the image, you’ll see what it looks like with the LED lights turned on…

Shelf Lighting

Shelf LED Lights

Conclusions

My first full day of standing was January 19th, 2011, and so far, I really love the desk. The first week was definitely a little rough, with my feet and back becoming sore by the end of the day, but it’s getting better. I’ve read that it takes a couple of weeks to fully adjust, so after I’ve given the desk a bit more time, I’ll post another update. At this point, I’m optimistic that it will be a permanent change to my home office setup.

Update

It has been almost 8 months since I started using the standing desk, and I don’t think I could go back. My posture has noticeably improved, and I quickly got over the initial soreness. I stand up about 85–90% of the time, and sit down with my lapboard for the rest (mostly later in the day). I’ve only made a few changes to my setup so far:

  1. I switched to an Kenesis Freestyle ergonomic keyboard to help improve wrist positioning and limit overreaching when using the mouse
  2. I added an Imprint anti-fatigue floor mat to make standing more comfortable (even though my office is carpeted)
  3. I added an Apple Magic Trackpad near my mouse pad so I wouldn’t need to reach to the upper level of the desk to perform multi-touch gestures

It’s safe to say that I enjoy utilizing a standing desk, and even though my shelving unit was originally purchased as an inexpensive way to get my feet wet, it has worked out better than expected. For now, it gets the job done and I don’t see the need to purchase a more expensive standing desk.

The only other planned improvements I have would be, to finally cut off the extra portion of each of the metal support rails that stick out from the top, and to build small side shelves that could support a pair of KRK Rokit 5 studio monitors. Once I’ve done those things, I’ll post an updated photo of how it looks.

Since posting this original article, I’ve inspired a handful of friends to try out standing desks, and all of them have stuck with it. If you’ve been on the fence about trying a standing desk, I can easily say that it’s worth the effort to give it a shot. I’d love to hear from you if you end up taking the plunge!

Update2

Well, I finished all of my planned improvements and here are the updated pics…ignore the dust and messy cabling :-)

Updated Desk

User Input

I found the shelves as a kit from Knape & Vogt for ~$20 each, but had to purchase some additional hardware (bolts, washers, and locking nuts) since they were originally designed for mounting with wood screws. The shelves can supposedly support up to 50 pounds each; they are stable, but I wouldn’t be comfortable adding much more than the current 15 pounds on them.

Glass Shelf

Shelf Supports

Cutting the metal support rails and trimming the shelf mounts to fit took a fair bit of time, a steady hand, and a Dremel. If you try it, the one thing I’ll say is to definitely wear safety glasses!

Anyway, I think I’m finally done tweaking things for a bit, but am incredibly happy with the results. Go forth and stand!