At Mind Candy, to support our games, we we have increasingly been looking toward AWS in order to take advantage of features such as autoscaling and RDS. With an existing estate spread across three datacentres, introducing instances in EC2 brought about a number of challenges.
Every time an EC2 instance is spun up, it’s given a random dynamic IP and hostname. It’s not possible to know in advance what this will be. While this might sound completely crazy if you’re used to working only with physical hardware, it’s actually been a great catalyst in the move towards ‘disposable’ infrastructure, where your server nodes can build and destroy themselves when required, with no sysadmin intervention. We’ll be talking more about how we do this another time, but today I wanted to talk about security.
As with any organisation, our physical locations have firewalls with very restictive policies. All of our servers have static addressing, and we know exactly where to find them. Creating firewall rules is predictable and no problem. Unfortunately, introducing EC2 nodes into the mix has caused us a bit of bother – how do we allow these dynamic nodes to talk back to our datacentres?
Amazon publish their list of IP ranges every now and then, via their support forum. Predictably, it’s huge. A number of companies I know have gone down the route of allowing the entire AWS address space to access their private networks. For me, this is simply asking for trouble.
We realised that we would need a more robust and targeted way of setting up access rules to only the hosts we were using, and ensuring that we didn’t persist rules for addresses which were no longer under our control.
This tool is written in Ruby, using the AWS SDK. Its job is to read a list of AWS account details, build a list of all active hosts and their IP addresses in every account and region, and then ensure that each IP is present in an object-group on a number of Cisco firewalls. Knowing that a certain object-group will always contain our entire AWS estate, we can happily create firewall rules without worrying about opening our network up to half the Internet.
Take a look at our github project page: