RabbitMQ


Frequently Asked Questions

See also these other sources of information:

If you have a question not answered here, please join our mailing list or alternatively, contact us directly at info@rabbitmq.com.

Background

Q. What is messaging?

A. Messaging is used to manage data in motion, unlike, say, databases, which manage data at rest. Use messaging to communicate between and scale applications, within your enterprise, across the web, or in the cloud.

Q. What is AMQP?

A. AMQP is a standard wire-level protocol and semantic framework for high performance enterprise messaging.

From the AMQP website:

AMQP is an Open Standard for Messaging Middleware.

By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner.

AMQP enables complete interoperability for messaging middleware; both the networking protocol and the semantics of broker services are defined in AMQP.

For more information on what AMQP is, please see the AMQP Working Group's overview page.

Q. Why use AMQP?

A. AMQP is specifically designed with modern messaging needs in mind, including the reduction of change and maintenance costs through separation of integration concerns, removal of silo dependency, and freedom from language and platform lock in, without compromise on user experience, security, scalability and consistently excellent performance.

For more details please see this page.

Q. Why use RabbitMQ?

A. RabbitMQ enables developers of messaging solutions to take advantage of not just AMQP but also one of the most proven systems on the planet. The Open Telecom Platform (OTP) is used by multiple telecommunications companies to manage switching exchanges for voice calls, VoIP and now video. These systems are designed to never go down and to handle truly vast user loads. And because the systems cannot be taken offline, they have to be very flexible, for instance it must be possible to 'hot deploy' features and fixes on the fly whilst managing a consistent user SLA.

Instead of creating a new messaging infrastructure, the RabbitMQ team selected the best one for the need, and built an AMQP layer on top. This combines the robustness and scalability of a proven platform with the flexibility of AMQP's messaging model.

Q. What is the Open Telecom Platform (OTP)?

A. The Open Telecom Platform (OTP) is a battle-tested library of management, monitoring, and support code for constructing extremely high-performance, reliable, scalable, available (nine nines!) distributed network applications. It is written in Erlang.

For good general introductions to both technological and business reasons supporting the use of Erlang and OTP, we recommend the following:

  • An overview of Erlang and OTP by Ericsson, the original developer of the technology and one of its major commercial users.
  • An interview with Joe Armstrong, Erlang and OTP's Chief Software Architect.
  • A discussion of concurrency and Erlang, with examples from general Erlang and OO programming, plus highlights from the online multiplayer game industry.
  • An introduction to Erlang with motivating examples followed by business proof-points.
  • A white paper from Ericsson and Cisco (see particularly page 15), discussing Ericsson's Engine Integral softswitch solution, which is implemented using OTP on Erlang.
  • A case study by Motorola comparing the use of C++ and Erlang for telecoms software.

For further details on the rationale for choosing Erlang, please see the FAQ entries on clustering technology and management interfaces. Finally, we've prepared a comprehensive assessment of Erlang's advantages at this page.

Q. What messaging scenarios are supported by AMQP and RabbitMQ?

A. AMQP is a very general system that can be configured to cover a great variety of messaging middleware use-cases. For example:

  • Point-to-point communication

    One of the simplest and most common scenarios is for a message producer to transmit a message addressed to a particular message consumer. AMQP covers this scenario by allowing queues to be named and to be bound to a "direct" exchange, which routes messages to queues by name.

  • One-to-many broadcasting (including multicast)

    In this scenario, the broadcasters publish messages to an AMQP "fanout" exchange, and subscribers create and subscribe to their own private AMQP queues, which forward published messages on to them, with one copy per queue.

    Multicast is addressed at the broker implementation level. AMQP clients need not be made aware of transport-level optimisations such as multicast: broker clusters are free to use whatever such low-level optimisations are available from configuration to configuration.

    Multiple optimisations are possible, since AMQP separates routing logic (exchanges and bindings) from message queueing (queues). Multicast relates only to routing from message publishers to message queues, and as a routing optimisation can be completely physically decoupled from AMQP's logical semantics. Further optimisations include physical separation of exchange from queue or even colocation of queue with a consumer application.

  • Transactional publication and acknowledgement

    AMQP supports transactional publication, where an AMQP channel is opened, transactional mode is selected, messages are published and acknowledged, and the transaction is committed. The system guarantees atomicity and durability properties for transactional message activity.

  • High-speed transient message flows

    Messages are individually flagged as transient or persistent in AMQP at the time of publication. By sending messages outside the transactional part of the protocol, in non-persistent mode, an application can achieve very high throughput and low latency.

  • Reliable persistent message delivery

    Messages that are published in persistent mode are logged to disk for durability. If the server is restarted, the system ensures that received persistent messages are not lost. The transactional part of the protocol provides the final piece of the puzzle, by allowing the server to communicate its definite receipt of a set of published messages.

  • Store-and-forward

    Store-and-forward is implemented by delivering messages marked as "persistent" to AMQP's durable queues. Published, persistent messages delivered to durable queues are stored on disk until a consumer retrieves and deletes them.

  • Wide area messaging

    Because routing logic is decoupled from message delivery, RabbitMQ is able to support extended broker clustering across WANs. Some of the approaches include AJAX-style access to AMQP resources, and spanning-tree pseudo-multicast implemented internally to a RabbitMQ cluster.

  • File streaming

    The AMQP protocol, version 0-8, supports file streaming by way of the file content class. Very large files are transferred to a temporary area on the broker before being routed to queues for download by consumers.

RabbitMQ Features

Q. Is RabbitMQ transactional?

A. Yes. RabbitMQ implements AMQP's "TX" message class, which provides atomicity and durability properties to those clients that request them.

Q. Is RabbitMQ persistent?

A. Yes. RabbitMQ uses Erlang's built-in logging and database support tools to provide high-speed, high-reliability durable message storage.

Q. Is RabbitMQ reliable and highly-available?

A. Yes. RabbitMQ is built atop the industry-leading OTP Erlang libraries, which provides a solid foundation for building reliable software. The underlying system supporting the Rabbit codebase has been used by Ericsson to achieve nine nines (99.9999999%) of availability.

Q. Does RabbitMQ support clustering, and high-availability through live failover?

A. Yes. RabbitMQ brokers can be made up of an arbitrary number of nodes, each of which is available for AMQP clients to connect to and interact with. RabbitMQ's routing tables are shared across the entire cluster, and delivery of messages published at one node to a queue residing at another node is seamless. For details see the clustering guide. We are currently developing further support for live failover of AMQP resources within a cluster.

Q. Will RabbitMQ use SMP, when it's available?

A. Yes. Erlang supports both single-node SMP configurations and configurations with multiple Erlang nodes running in a cluster within a single host.

Q. Does RabbitMQ support store-and-forward?

A. Yes, RabbitMQ, and AMQP in general, supports store-and-forward-style exchanges and queues.

Q. Do you think RabbitMQ, as a core module, could handle real time traffic of something like an SMSC (SMS foreward&store)?

A. Absolutely. Some large telcos currently deploy Erlang/OTP-based systems in their SMSC infrastructure. At present these systems are not based on AMQP/RabbitMQ, but there is no reason why they couldn't be.

Q. Can I create RabbitMQ applications using web frameworks (eg Spring, or Ruby-On-Rails)?

A. Yes. To begin with, Spring has good support for message access in Java: the RabbitMQ Java client is simply a POJO library. The same is true for other POJO-based systems, such as Mule. If you specifically wish to use JavaEE EJBs as message accessors, use a stateful session bean.

For frameworks in other languages, such as Ruby or in-browser Javascript, we plan on providing both RESTful HTTP access to AMQP features as well as AJAX- and web-services-style interfaces, allowing rapid development of web applications that take advantage of AMQP messaging.

RabbitMQ Architecture

Q. How is authentication and authorization supported in RabbitMQ?

A. AMQP uses SASL (see also Wikipedia on SASL) for authentication of AMQP clients. Our current support for SASL is limited to the PLAIN authentication mechanism.

Authorization is implemented in RabbitMQ using a distributed database table mapping users to virtual hosts.

Message authentication, for instance using cryptographic message authentication codes, is not defined by the AMQP specification, but as for encryption, could be very easily implemented within the RabbitMQ client library.

Q. What platforms will RabbitMQ run on?

A. RabbitMQ is developed on Debian Linux and Mac OS X. As the server component is written in Erlang, and the client component in Java, it is platform-neutral. Erlang runs on the following platforms (taken from Erlang's FAQ):

  • Solaris (including 64 bit)
  • BSD
  • Linux
  • OSX
  • TRU64
  • Windows NT/2000/2003/XP
  • Windows 95, 98
  • VxWorks

Q. What is shared session clustering?

A. For those familiar with typical tiered session/application architectures such as JavaEE, it might help to think of the AMQP exchanges as corresponding to a logical session tier, and AMQP queues as corresponding to a logical application tier. From this point of view, RabbitMQ's routing tables can be seen as clustered shared session state. RabbitMQ uses the OTP distributed database, Mnesia, to reliably and persistently replicate session state across all nodes in a cluster.

Q. How does clustering work?

A. RabbitMQ's current clustering mechanisms are built around Erlang's native support for reliable distributed programming. This is a deeply sophisticated framework, implemented at the language and virtual-machine level.

Some of the unique points about Erlang's networking and distribution model are:

  • it scales to hundreds of thousands of parallel processes ("green threads") within a single virtual machine, bounded only by available memory;
  • the failure model for local inter-process communication (IPC) is the same as that for distributed IPC, making the transition from non-distributed to distributed code very smooth;
  • binary pattern-matching constructs within the language ensure straightforward and efficient translation between wire-level encodings and internal Erlang data structures;
  • finally, the OTP libraries shipped with the Erlang distribution include the notion of a supervisor/worker relationship, where supervisor processes monitor and restart worker processes under their control, making management of the entire process hierarchy within a server deterministic and automatic.

Within an Erlang node cluster, Erlang's native high-speed messaging is used to provide an efficient way of distributing work across the cluster. Individual AMQP clients connect to machines within the cluster, and Erlang's distributed routing database routes AMQP messages to the appropriate endpoints.

For more detail on Erlang's benefits in an AMQP setting, please see this page.

Q. How does failover and HA work?

A. Currently, AMQP durable queues and their persistent messages are recovered from disk at each server restart. Exchange failover is handled by OTP's support for clustering (see above).

Future releases will support live failover using, for instance, a combination of the "known hosts" field in connection.open-ok and the connection.redirect message. Erlang's built-in fault-tolerant database will ensure that routing information is preserved, and the OTP supervisor architecture in combination with AMQP's transactional features will ensure that durable messages are not lost.

Q. How does addressing work in AMQP?

A. Like other messaging systems, AMQP and RabbitMQ provide internal addressing mechanisms based on notions such as routing keys, exchange addresses and queue names.

These are managed on a per-virtual-host basis. Please see the AMQP specification for details of the scoping of the various names in the network. We are actively investigating interoperation with other addressing schemes and systems.

Q. How is RabbitMQ scaled and the load balance handled?

A. At the core we are relying on Erlang/OTP to distribute one logical AMQP broker across several physical nodes, with the necessary broker state being maintained in an instance of OTP's distributed database Mnesia. There are certain aspects of the AMQP spec that are currently undergoing revision that impact this area, so watch out for associated new features and documentation in future releases of RabbitMQ.

Integration

Q. What clients does RabbitMQ support?

A. We maintain a list of clients, which is fairly complete. If you don't see one you want, please email our mailing list.

How-To

Q. "Last" message on Topic

A classic use-case for messaging systems is the example of subscribing to a topic of, say, 'market prices' for an asset. By that a client very often means, "send me the last published one and then subscribe me to any update". Sometimes it is not necessary if the asset is liquid (MSFT) but if it is illiquid, the client should receive the last available message, as it's possible there will be a long delay before the next update.

Does RabbitMQ support this?

A. This is not something AMQP provides out-of-the-box, although one can imagine a solution involving a trivial service (a simple hash table, in effect) answering requests for most-recent-price served via AMQP. The client would subscribe to the stream as usual, and in parallel ask the service for the most recent price.

Such a service could be implemented either inside the RabbitMQ broker, in Erlang, thus taking advantage of Erlang's distribution and high-availability properties, or it could be implemented outside the broker, as a regular AMQP client.

Q. Channel Encryption

Say I want to transfer a message across the AMQP network, without depending on the target being there, and that the 'channel' between the message source client and the message target client should be strongly encrypted and authenticated. Is this something RabbitMQ provides?

A. There are two kinds of encryption that could be supported by an AMQP implementation.

The first is encryption of the AMQP stream. There's no provision in the standard for that, yet, but wrapping the stream in a TLS layer is a straightforward extension (please see the section on SSL/TLS in our API guide). The notion of addressing for AMQP entities (ie. URLs for AMQP queues and exchanges) is under active development at the moment. It seems likely that once that settles down, it will quickly be extended to standardise a TLS layering for AMQP ("amqp:" -> "amqps:" ?).

The second kind of encryption is encryption of individual messages. AMQP is silent on this front as well, but using a layer of encryption in the clients is very easy. Adding support for this to RabbitMQ's client library would be a simple matter.

Q. How do I configure access control?

A. Please see the section on access control in the RabbitMQ admin guide.

Q. Instrumentation, logging and debugging

A. RabbitMQ uses AMQP internally to the broker to notify interested parties of significant events. Administration and management exchanges are provided for tools to bind to.

In addition, there is some discussion in the AMQP community of the need for replay as a built-in feature.

Q. How do I test RabbitMQ?

A. You can test the RabbitMQ broker by:

  • connecting to our demo server using one of the RabbitMQ example programs
  • connecting to our demo server using another AMQP 0-8 compatible client
  • Downloading the broker and experimenting with it on your own machine

Q. Can I test my AMQP client against RabbitMQ?

A. Yes: you can either run your client against our public demonstration server, or download and run a server of your own to experiment with.

Q. How do I tune the scope of replication? Will it affect performance?

A. Since replication is used mainly to propagate routing table information between Erlang nodes within a cluster, it has a low impact on performance to begin with; also, it is trivial to arrange for a "queue-only" Erlang node to join the cluster, only able to consume from queues, which can then avoid being part of the routing-table replication system.

Performance and Scalability

Q. How low can latency be?

A. We have seen latencies averaging less than a millisecond; for heavy, steady-state loads, latency hovers around the 4-millisecond mark (including two network hops).

Q. How fast is RabbitMQ in persistent mode?

A. From our testing, we expect easily-achievable throughputs of 4000 persistent, non-transacted one-kilobyte messages per second (Intel Pentium D, 2.8GHz, dual core, gigabit ethernet) from a single RabbitMQ broker node writing to a single spindle.

Please let us know how RabbitMQ performs for you!

Q. How do I switch off persistence/transactionality?

A. To disable transactionality, use an AMQP channel which is not in "TX" mode (ie. do not call tx.select). To disable message persistence, set the "delivery mode" field in the basic-class properties for the message to either absent or to one, as mode two enables persistence.

Q. How does RabbitMQ's performance scale with clustering?

A. Based on typical OTP application scalability, we expect close-to-linear scaling.

Q. How many virtual hosts can I run and where?

A. There is no hard limit to the number of virtual hosts that can be configured within a RabbitMQ cluster, save available disk and memory resources.

Virtual hosts within AMQP are decoupled from the physical network layout: one virtual host can be accessed from multiple clustered brokers, just as many virtual hosts can be accessed within a single broker. Virtual hosts act solely as a namespacing mechanism for AMQP resources.

General

Q. How is RabbitMQ priced?

A. RabbitMQ is free software in both the gratis and libre senses, licenced under the Mozilla Public License.

Q. What support options are available for RabbitMQ?

A. RabbitMQ will be supported, both commercially and as an open-source project, by both LShift and by CohesiveFT as virtual appliances. For inquiries about support, please email support@rabbitmq.com. For inquiries specifically about virtual appliances, contact support@cohesiveft.com.

Q. How is RabbitMQ licenced (for redistribution, etc.)?

A. RabbitMQ is distributed under the open-source Mozilla Public License.

If you would like to explore other licensing options, please get in touch with us by emailing info@rabbitmq.com.

Q. How do I contribute to the RabbitMQ project?

A. Welcome aboard! We love contributions - here are some suggestions for what you can work on:

  • packaging for other operating systems
  • integration with other messaging systems
  • integration with other management systems
  • integration with other user and permission database systems
  • improvements to the client, and work towards other clients
  • new services for running with RabbitMQ, either within the broker or adjacent to it
  • ... and so on!

We're happy to hear from anyone who wants to get involved or is curious about how we intend to manage the project.

Q. How should I interpret RabbitMQ's version numbering?

A. RabbitMQ is versioned with the scheme major.minor.patch.

  • major: This number indicates the major version of the software. It is only changed for significant alterations to the software, such as (for instance) a total rewrite.
  • minor: A change in this number indicates new or significantly altered features.
  • patch: This number changes to make each distinct packaging of the software uniquely numbered. Small changes such as bug-fixes or packaging alterations may cause a change in only the patch number.

We also label each release with a status. This is either alpha, beta, rc (for "release candidate") or final, and indicates the maturity of the release. The status is mentioned in the release announcement and on the download page.

Q. I have a question!

A. Please join our mailing list or send questions to info@rabbitmq.com. We'd love to hear from you.