fog

Compute

version
Gem Version
install
gem install fog
source
fog/fog

Compute is the lifeblood of the cloud, but with great power comes great complication. Compute opens up huge swaths of potential, but it varies greatly in both capabilities and usage from provider to provider. Thankfully fog helps to abstract these idiosyncrasies to provide a more seamless experience.

Installing fog

fog is distributed as a RubyGem:

1
gem install fog

Or for bundler users, you can add it in your Gemfile:

1
gem "fog"

Using Amazon EC2 and fog

Sign up for an account here and copy down your secret access key and access key id from here. We are about to get into the code samples, so be sure to fill in anything in ALL_CAPS with your own values!

First, create a connection with your new account:

1
2
3
4
5
6
7
8
9
require 'rubygems'
require 'fog'

# create a connection
connection = Fog::Compute.new({
  :provider                 => 'AWS',
  :aws_access_key_id        => YOUR_AWS_ACCESS_KEY_ID,
  :aws_secret_access_key    => YOUR_AWS_SECRET_ACCESS_KEY
})

With that in hand we are ready to start making EC2 calls!

Servers the EC2 way

Creating a server on EC2 is very easy if you are willing to accept the defaults (the smallest server size, using Ubuntu 10.04 LTS). NOTE: the default EC2 image uses the ‘ubuntu’ username, rather than ‘root’ like other services.

1
server = connection.servers.create

You can then list your servers to see that it now appears:

1
connection.servers

Rather than worrying about the whole list, we can also just get the latest data for just our server:

1
server.reload

That can get tedious quickly however, especially when servers can take several minutes to boot. Fog has

1
wait_for
for cases like this and
1
ready?
for checking to see when a server has completed its start up.

1
server.wait_for { ready? }

Once we are done with that we can shut it down.

1
server.destroy

Bootstrap: Servers the fog Way

Cycling servers is great, but in order to actually ssh in we need to setup ssh keys and open ports. But rather than worrying about the nitty gritty, we will utilize

1
bootstrap
. NOTE: normally we could leave out username and use the default (root), but the default Ubuntu from Canonical uses the ubuntu username instead.

1
server = connection.servers.bootstrap(:private_key_path => '~/.ssh/id_rsa', :public_key_path => '~/.ssh/id_rsa.pub', :username => 'ubuntu')

Bootstrap will create the server, but it will also make sure that port 22 is open for traffic and has ssh keys setup. The ssh key pair you specified will be registered under the name “fog_default” unless you’ve set

1
Fog.credential
to a custom string value. In order to hook everything up
1
bootstrap
will need the server to be running, so by the time it finishes it will be ready. You can then make commands to it directly:

1
2
server.ssh('pwd')
server.ssh(['pwd', 'whoami'])

These return an array of results, where each has stdout, stderr and status values so you can check out what your commands accomplished. Now just shut it down to make sure you don’t continue getting charged.

1
server.destroy

Managing multiple ssh key pairs on EC2

The key pair you’ve specified, will be registered as “fog_default” after running

1
bootstrap
for the first time. If you want to use multiple key pairs with the same AWS credentials, you need to set
1
Fog.credential
to register your other key pairs under different names. Your additional key pair will then be registered as “fog_#{Fog.credential}”:

1
2
Fog.credential = 'my_custom_key'
connection.servers.bootstrap(:private_key_path => '~/.ssh/my_custom_key', :public_key_path => '~/.ssh/my_custom_key.pub')

If you’ve already registered a custom key pair e.g. using

1
connection.create_key_pair
or
1
connection.import_key_pair
, you can set your key paths using
1
Fog.credentials
and pass in the name of this key so
1
bootstrap
will use it instead of “fog_default”:

1
2
3
Fog.credentials = Fog.credentials.merge({ :private_key_path => "~/.ssh/my_custom_key", :public_key_path => "~/.ssh/my_custom_key.pub" })
connection.import_key_pair('my_custom_key', IO.read('~/.ssh/my_custom_key.pub')) if connection.key_pairs.get('my_custom_key').nil?
server = connection.servers.bootstrap(:key_name => 'my_custom_key')

Rackspace Cloud Servers

Rackspace has Cloud Servers and you can sign up here and get your credentials here.

1
2
3
4
5
6
# create a connection
connection = Fog::Compute.new({
  :provider           => 'Rackspace',
  :rackspace_username => RACKSPACE_USERNAME,
  :rackspace_api_key  => RACKSPACE_API_KEY
})

If you work with the European cloud from Rackspace you have to add the following:

1
:rackspace_auth_url => "lon.auth.api.rackspacecloud.com"

We will skip over learning how to do this the ‘Rackspace Way’ and instead jump right to using bootstrap to get their smallest Ubuntu 10.04 LTS server.

1
server = connection.servers.bootstrap

You can run all the same ssh commands and do what you need to, then once again shutdown to ensure you are not charged once you are done.

1
server.destroy

Google Compute Engine

Google has Compute Engine. To get your authorization key, visit the Google Developers Console. Once there, select your project, go to “APIs & auth - Credentials”. Click “Create another client ID…” and select “service account”. Next, download either the JSON key (preferred) or the P12 key.

If you have generated a P12 key then you will need to specify the location of your P12 key file:

1
2
3
4
5
6
7
# create a connection
connection = Fog::Compute.new({
  :provider => 'google',
  :google_project => GOOGLE_PROJECT_ID,
  :google_client_email => GOOGLE_SERVICE_EMAIL,
  :google_key_location => GOOGLE_P12_KEY_LOCATION,
})

If you have generated a JSON key then then you will need to specify the location of your JSON key file:

1
2
3
4
5
6
7
# create a connection
connection = Fog::Compute.new({
  :provider => 'google',
  :google_project => GOOGLE_PROJECT_ID,
  :google_client_email => GOOGLE_SERVICE_EMAIL,
  :google_json_key_location => GOOGLE_JSON_KEY_LOCATION,
})

The easiest way to launch a server is to:

1
server = connection.servers.bootstrap

Which is equivalent to running:

1
2
3
4
5
6
7
8
server = connection.servers.create({
  :name => "fog-#{Time.now.to_i}",       # Instance Name
  :image_name => "gcel-12-04-v20130225", # An ubuntu 12.04 based image
  :machine_type => "n1-standard-1",      # A single core VM
  :zone_name => "us-central1-a",         # Central United States
})

server.wait_for { ready? }

You can run all the same ssh commands and do what you need to, then once again shutdown to ensure you are not charged once you are done.

1
server.destroy

Mocking out Compute

You can also start any of these scripts with

1
Fog.mock!
or start the fog interactive tool from the command line with
1
FOG_MOCK=true fog
to run in mock mode. In this mode commands are run as local simulation, so no cloud resources are ever consumed and things operate much faster.

Cleaning up

To cover your tracks its a good idea to check for running servers and shut them down, here is one way you might do that.

1
connection.servers.select {|server| server.ready? && server.destroy}

Summary

Compute can be tricky, but the abstractions in fog make it much easier to get started. With your servers up and running you can then focus on the task at hand and get some work done. Congratulations on adding a new tool to your arsenal and let us know what we can do better.

About