RabbitMQ - Amazon EC2 Images

RabbitMQ Amazon EC2 Images

Using RabbitMQ Amazon EC2 Images, RabbitMQ can be easily deployed on the Amazon cloud. Images containing Ubuntu 9.10 (Karmic Koala) with pre-installed RabbitMQ can be found below. Both 32 and 64 bit architectures are available.

32-bit RabbitMQ Images: Can be run on the smallest and cheapest m1.small cloud computer instances. Fine for development purposes, but due to small 32-bit virtual memory space not recommended for production.
64-bit RabbitMQ Images: Run on more demanding 64-bit instances. Recommended for production usage.

Amazon EC2 Published Images

Following Ubuntu images with preinstalled RabbitMQ 1.7.0 have been published:

Availability zone	Arch	Ami	Ec2 command
eu-west-1	x86_64	ami-c9002bbd	`ec2-run-instances ami-c9002bbd --key ${EC2_KEYPAIR} --instance-type m1.large --region eu-west-1`
eu-west-1	i386	ami-b5002bc1	`ec2-run-instances ami-b5002bc1 --key ${EC2_KEYPAIR} --instance-type m1.small --region eu-west-1`
us-east-1	x86_64	ami-36cc2e5f	`ec2-run-instances ami-36cc2e5f --key ${EC2_KEYPAIR} --instance-type m1.large`
us-east-1	i386	ami-3ecc2e57	`ec2-run-instances ami-3ecc2e57 --key ${EC2_KEYPAIR} --instance-type m1.small`

To start RabbitMQ instance just run the command from the fourth column. The string ${EC2_KEYPAIR} should be replaced with the public ssh key name that is going to be used to log in to the instance. You can find more information about ssh keys in Getting Started Guide. To run the instance, you should run a command like one of the above:

localhost:~$ ec2-run-instances ami-XXX --key XXXX --instance-type m1.large --region eu-west-1

Integration with Amazon EBS

RabbitMQ stores some data on disk. It may be worth storing this persistent data on an Amazon EBS (Elastic Block Store) Volume. Decoupling data from the instance using EBS brings some benefits:

Data is less likely to be lost
Migration data to different instance is straightforward
EBS has a proper backup strategy

RabbitMQ images provide features to simplify integration with Amazon EBS block storage.

Using EBS volumes with RabbitMQ instances is straightforward. All you need to do is:

Create an EBS volume based on our snapshot
Attach it under /dev/sdh
Finally, to apply the changes, reboot the instance

Setting up an EBS volume

Public snapshots of pre-configured RabbitMQ EBS volumes are listed below.

Availability zone	Public snapshot id	Ec2 command
eu-west-1	snap-321df35b	`ec2-create-volume --snapshot snap-321df35b --size 8 \ --region eu-west-1 --availability-zone eu-west-1b`
us-east-1	snap-3640df5f	`ec2-create-volume --snapshot snap-3640df5f --size 8 \ --region us-east-1 --availability-zone us-east-1b`

Create an EBS volume based on our pre-configured shared EBS snapshot. Our snapshot was created from 8GB EBS, so your volume must have at least that size.

To create an EBS volume run the command from third column. Please tweak the --availability-zone parameter to match the availability zone of your instance. You can also specify size to be more than 8GB. The command might look like:

localhost:~$ ec2-create-volume --snapshot snap-XXX --size 8 \
                 --region us-east-1 --availability-zone us-east-XX

Attach a volume to an instance

To attach a volume to the instance, use the ec2-attach-volume command. Please remember to attach the volume under /dev/sdh.

localhost:~$ ec2-attach-volume --region eu-west-1 vol-xxxxxxxx -i i-xxxxxxxx -d /dev/sdh

Attaching volumes is sticky, so EBS will remain attached until you kill the instance or unattach it manually. Volumes attached under /dev/sdh are automatically mounted as /vol.

Configure the RabbitMQ instance to use EBS

First, log into the instance as root:

localhost:~$ ssh -i KEYPAIR.pem root@ec2-XXXX.compute.amazonaws.com

To persuade RabbitMQ to use disk space from /vol actual paths must be specified in its configuration. A script called rabbit-copy-config should be run on start-up. It sets RabbitMQ's configuration to be read from an EBS volume. It links the configuration options prepared for RabbitMQ under /vol/etc to the standard /etc location. Now the default RabbitMQ installation will be affected. It will store data in the correct path automatically after a reboot.

rabbitmq:~$ reboot

If, however, you want to avoid reboots, you can run the scripts manually:

rabbitmq:~$ mount /vol
rabbitmq:~$ /etc/init.d/rabbit-copy-config
rabbitmq:~$ /etc/init.d/rabbitmq-server restart

From now on, RabbitMQ will store persistent data on EBS in /vol.

Resizing EBS

If your volume is bigger than 8GB, you need to expand the size the filesystem to use all the available space.

rabbitmq:~$ xfs_growfs /vol

EBS migration strategy

Moving RabbitMQ storage from one RabbitMQ instance to another is trivial.

Detach EBS volume from old RabbitMQ EC2 instance
Attach it to the new RabbitMQ instance
Reboot the instance (or follow alternative steps)

EBS backup strategy

Amazon tools can create a snapshot of EBS volume as a file on S3. This approach is often used to backup EBS data.

localhost:~$ ec2-create-snapshot --region eu-west-1 vol-xxxxxxxx -d "Backup"

Running custom scripts on start-up

By using the ec2-run-instances --user-data-file option, you can specify a script to be executed during virtual machine start-up. This script will be run before RabbitMQ starts.

localhost:~$ ec2-run-instances <usual_parameters> --user-data-file init-script.sh

For example:

#!/bin/sh
# This scrit will be executed before RabbitMQ has started
export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get -y upgrade
apt-get -y install man screen links htop

RabbitMQ status plugin

RabbitMQ EC2 Images have the status plugin preinstalled by default. To use it you must be able to access port 55672 of the instance. You can achieve that by opening that port on Amazon firewall. Be aware that the status plugin is secured only using RabbitMQ AMQP credentials. Unless you change default guest password, the status page will be open to everyone that can connect to the port. To change guest password run rabbitmqctl change_password guest <new_pass> inside the instance.

Alternatively, you can forward a TCP port on your local machine.

localhost:~$ ssh -L 55672:localhost:55672 root@ec2-xx-xx-xx-xx.eu-west-1.compute.amazonaws.com

You should be able to see RabbitMQ status page under the forwarded port on your local machine, for example: http://guest:guest@localhost:55672/.

Testing RabbitMQ instance

Once the RabbitMQ instance has been started and configured RabbitMQ instance, check if it actually works. You can do this by installing the RabbitMQ java client, and running the test scripts that come with it. Use the following instructions to install the required packages on the instance:

apt-get update
apt-get install sun-java6-jre wget
wget http://www.rabbitmq.com/releases/rabbitmq-java-client/v1.7.0/rabbitmq-java-client-bin-1.7.0.tar.gz
tar xzf rabbitmq-java-client-bin-1.7.0.tar.gz
cd rabbitmq-java-client-bin-1.7.0

When the dependencies are available, you can run the test:

rabbitmq:~/rabbitmq-java-client-1.7.0$ sh ./runjava.sh com.rabbitmq.examples.MulticastMain

On the screen you should see output similar to this:

starting consumer #0
starting producer #0
recving rate: 2920 msg/s, min/avg/max latency: 13431/137515/349586 microseconds
sending rate: 7182 msg/s
recving rate: 4513 msg/s, min/avg/max latency: 352244/535973/715864 microseconds
sending rate: 4897 msg/s
recving rate: 4119 msg/s, min/avg/max latency: 654393/829167/1021911 microseconds
sending rate: 4989 msg/s

If you followed the steps above, about the status plugin, you should be able to see the load on the status page: http://guest:guest@localhost:55672/.

Please remember, this is all quite new and its status is experimental. If you run into any difficulties please let us know.

Table of Contents