Menu

Java Tools

This page documents some Java-based utility programs (PerfTest, Tracer).

PerfTest

RabbitMQ has a basic throughput testing tool, PerfTest (docs, source code and releases), that is based on the Java client and can be configured to simulate basic workloads. PerfTest has extra tools that produce HTML graphs of the output. A RabbitMQ cluster can be limited by a number of factors, from infrastructure-level constraints (e.g. network bandwidth) to RabbitMQ configuration and topology to applications that publish and consume. PerfTest can demonstrate baseline performance of a node or a cluster of nodes.

Installation

PerfTest is distributed as a binary build archive as well as Maven (the latter allows it to be also used as a library).

The distribution contains a script (bin/runjava or bin/runjava.bat) to run Java with the class path correctly configured, e.g. bin/runjava com.rabbitmq.perf.PerfTest runs the PerfTest Java class.

To verify a PerfTest installation, use

bin/runjava com.rabbitmq.perf.PerfTest --help

Using PerfTest

The most basic way of running PerfTest only specifies a URI to connect to, a number of publishers to use (say, 1) and a number of consumers to use (say, 2). Note that RabbitMQ Java client can achieve high rates for publishing (up to 80 to 90K messages/second), given enough bandwidth and some safety measures (publisher confirms) enabled, so overprovisioning publishers is rarely necessary for a basic test. The following command runs PerfTest with a single publisher without publisher confirms, two consumers (each receiving a copy of every message) that use automatic acknowledgement mode and a single queue named “throughput-test-x1-y2”. Publishers will publish as quickly as possible, without any rate limiting. Results will be prefixed with “test 1” for easier identification and comparison:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-1" -a --id “test 1”

This modification will use 2 publishers and 4 consumers, typically yielding higher throughput given enough CPU cores on the machine and RabbitMQ nodes:

bin/runjava com.rabbitmq.perf.PerfTest -x 2 -y 4 -u "throughput-test-2" -a --id “test 2”

This modification switches consumers to manual acknowledgements:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-3" --id “test 3”

This modification changes message size from default (1 kB) to 4 kB:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-4" --id “test 4” -s 4000

PerfTest can use durable queues and persistent messages:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-5" --id "test-5" -f persistent

When PerfTest is running, it is important to monitor various publisher and consumer metrics provided by the management UI. For example, it is possible to see how much network bandwidth a publisher has been using recently on the connection page.

Queue page demonstrates message rates, consumer count, acknowledgement mode used by the consumers, consumer utilisation and message location break down (disk, RAM, paged out transient messages, etc). When durable queues and persistent messages are used, node I/O and message store/queue index operation metrics become particularly important to monitor.

Consumers can ack multiple messages at once, for example, 100 in this configuration:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-6" --id "test-6" -f persistent --multiAckEvery 100

Consumer prefetch (QoS) can be configured as well (in this example to 500):

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-7" --id "test-7" -f persistent --multiAckEvery 200 -q 500

Publisher confirms can be used with maximum of N outstanding publishes:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-8" --id "test-8" -f persistent -q 500 -c 500

PerfTest can publish only a certain number of messages instead of running until shut down:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-10" --id "test-10" -f persistent -q 500 -pmessages 100000

Publisher rate can be limited:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-11" --id "test-11" -f persistent -q 500 --rate 5000

Consumer rate can be limited as well to simulate slower consumers or create a backlog:

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-12" --id "test-12" -f persistent --rate 5000 --consumerRate 2000

PerfTest can be configured to run for a limited amount of time (in seconds):

bin/runjava com.rabbitmq.perf.PerfTest -x 1 -y 2 -u "throughput-test-13" --id "test-13" -f persistent -z 30
Note that the consumer rate limit is applied per consumer, so in the configuration above the limit is actually 2 * 2000 = 4000 deliveries/second.

Running PerfTest without consumers and with a limited number of messages can be used to pre-populate a queue, e.g. with 1M messages 1 kB in size each::

bin/runjava com.rabbitmq.perf.PerfTest -y0 -p -u “”throughput-test-14” -s 1000 -C 1000000 --id “test-14” -f persistent

To consume from a pre-declared and pre-populated queue without starting any publishers, use

bin/runjava com.rabbitmq.perf.PerfTest -x0 -y10 -p -u “”throughput-test-14” --id “test-15”

PerfTest is useful for establishing baseline cluster throughput with various configurations but does not simulate many other aspects of real world applications. It is also biased towards very simplistic workloads that use a single queue, which provides limited CPU utilisation on RabbitMQ nodes and is not recommended for most cases. Multiple PerfTest instances running simultaneously can be used to simulate more realistic workloads.

How It Works

If a queue name is defined (-u "queue-name"), PerfTest will create a queue with this name and all consumers will consume from this queue. The queue will be bound to the direct exchange with its name as the routing key. The routing key will be used by producers to send messages. This will cause messages from all producers to be sent to this single queue and all consumers to receive messages from this single queue.

If the queue name is not defined, PerfTest will create a random UUID routing key with which producers will publish messages. Each consumer will create its own anonymous queue and bind it to the direct exchange with this routing key. This will cause each message from all producers to be replicated to multiple queues (number of queues equals number of consumers), while each consumer will be receiving messages from only one queue.

PerfTest Results in HTML

The PerfTest HTML extension are a set of tools that can help you run automated benchmarks by wrapping around the PerfTest benchmarking framework. You can provide benchmark specs, and the tool will take care of running the benchmark, collecting results and displaying them in an HTML page. Learn more here.

Tracer

The tracer is a very basic, very simple AMQP 0-9-1 protocol analyzer, similar in purpose to Wireshark. Use it with the runtracer or runtracer.bat script:

runtracer listenPort connectHost connectPort
listenPort
port to listen for incoming AMQP connections on - defaults to 5673.
connectHost
hostname to use when making an outbound connection in response to an incoming connection - defaults to localhost.
connectPort
port number to use when making an outbound connection - defaults to 5672.

Download and source code

Releases

Source code