RabbitMQ


Frequently Asked Questions

Please see also the documentation index page. If you have a question not answered here, why not join our mailing list or alternatively, contact us directly at info@rabbitmq.com.

Background

Q. What is AMQP?

A. AMQP is a standard wire-level protocol and semantic framework for high performance enterprise messaging.

From the AMQP website:

AMQP is an Open Standard for Messaging Middleware.

By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organisations in a timely manner.

AMQP enables complete interoperability for messaging middleware; both the networking protocol and the semantics of broker services are defined in AMQP.

For more information on what AMQP is, please see the AMQP Working Group's overview page.

Q. Why use AMQP?

A. AMQP is specifically designed with modern messaging needs in mind, including the reduction of change and maintenance costs through separation of integration concerns, removal of silo dependency, and freedom from language and platform lock in, without compromise on user experience, security, scalability and consistently excellent performance.

For more details please see this page.

Q. Why use RabbitMQ?

A. RabbitMQ enables developers of messaging solutions to take advantage of not just AMQP but also one of the most proven systems on the planet. The Open Telecom Platform (OTP) is used by multiple telecommunications companies to manage switching exchanges for voice calls, VoIP and now video. These systems are designed to never go down and to handle truly vast user loads. And because the systems cannot be taken offline, they have to be very flexible, for instance it must be possible to 'hot deploy' features and fixes on the fly whilst managing a consistent user SLA.

Instead of creating a new messaging infrastructure, the RabbitMQ team selected the best one for the need, and built an AMQP layer on top. This combines the robustness and scalability of a proven platform with the flexibility of AMQP's messaging model.

Q. What is the Open Telecom Platform (OTP)?

A. The Open Telecom Platform (OTP) is a battle-tested library of management, monitoring, and support code for constructing extremely high-performance, reliable, scalable, available (nine nines!) distributed network applications. It is written in Erlang.

For good general introductions to both technological and business reasons supporting the use of Erlang and OTP, we recommend the following:

  • An overview of Erlang and OTP by Ericsson, the original developer of the technology and one of its major commercial users.
  • An interview with Joe Armstrong, Erlang and OTP's Chief Software Architect.
  • A discussion of concurrency and Erlang, with examples from general Erlang and OO programming, plus highlights from the online multiplayer game industry.
  • An introduction to Erlang with motivating examples followed by business proof-points.
  • A white paper from Ericsson and Cisco (see particularly page 15), discussing Ericsson's Engine Integral softswitch solution, which is implemented using OTP on Erlang.
  • A case study by Motorola comparing the use of C++ and Erlang for telecoms software.

For further details on the rationale for choosing Erlang, please see the FAQ entries on clustering technology and management interfaces. Finally, we've prepared a comprehensive assessment of Erlang's advantages at this page.

Q. What messaging scenarios are supported by AMQP and RabbitMQ?

A. AMQP is a very general system that can be configured to cover a great variety of messaging middleware use-cases. For example:

  • Point-to-point communication

    One of the simplest and most common scenarios is for a message producer to transmit a message addressed to a particular message consumer. AMQP covers this scenario by allowing queues to be named and to be bound to a "direct" exchange, which routes messages to queues by name.

  • One-to-many broadcasting (including multicast)

    In this scenario, the broadcasters publish messages to an AMQP "fanout" exchange, and subscribers create and subscribe to their own private AMQP queues, which forward published messages on to them, with one copy per queue.

    Multicast is addressed at the broker implementation level. AMQP clients need not be made aware of transport-level optimisations such as multicast: broker clusters are free to use whatever such low-level optimisations are available from configuration to configuration.

    Multiple optimisations are possible, since AMQP separates routing logic (exchanges and bindings) from message queueing (queues). Multicast relates only to routing from message publishers to message queues, and as a routing optimisation can be completely physically decoupled from AMQP's logical semantics. Further optimisations include physical separation of exchange from queue or even colocation of queue with a consumer application.

  • Transactional publication and acknowledgement

    AMQP supports transactional publication, where an AMQP channel is opened, transactional mode is selected, messages are published and acknowledged, and the transaction is committed. The system guarantees atomicity and durability properties for transactional message activity.

  • High-speed transient message flows

    Messages are individually flagged as transient or persistent in AMQP at the time of publication. By sending messages outside the transactional part of the protocol, in non-persistent mode, an application can achieve very high throughput and low latency.

  • Reliable persistent message delivery

    Messages that are published in persistent mode are logged to disk for durability. If the server is restarted, the system ensures that received persistent messages are not lost. The transactional part of the protocol provides the final piece of the puzzle, by allowing the server to communicate its definite receipt of a set of published messages.

  • Store-and-forward

    Store-and-forward is implemented by delivering messages marked as "persistent" to AMQP's durable queues. Published, persistent messages delivered to durable queues are stored on disk until a consumer retrieves and deletes them.

  • Wide area messaging

    Because routing logic is decoupled from message delivery, RabbitMQ is able to support extended broker clustering across WANs. Some of the approaches include AJAX-style access to AMQP resources, and spanning-tree pseudo-multicast implemented internally to a RabbitMQ cluster.

  • File streaming

    The AMQP protocol, version 0-8, supports file streaming by way of the file content class. Very large files are transferred to a temporary area on the broker before being routed to queues for download by consumers.

RabbitMQ Features

Q. Is RabbitMQ transactional?

A. Yes. RabbitMQ implements AMQP's "TX" message class, which provides atomicity and durability properties to those clients that request them.

Q. Is RabbitMQ persistent?

A. Yes. RabbitMQ uses Erlang's built-in logging and database support tools to provide high-speed, high-reliability durable message storage. For storage in your own RDBMS, please see this FAQ entry.

Q. Is RabbitMQ reliable and highly-available?

A. Yes. RabbitMQ is built atop the industry-leading OTP Erlang libraries, which provides a solid foundation for building reliable software. The underlying system supporting the Rabbit codebase has been used by Ericsson to achieve nine nines (99.9999999%) of availability.

Q. Does RabbitMQ support clustering, and high-availability through live failover?

A. Yes. RabbitMQ brokers can be made up of an arbitrary number of nodes, each of which is available for AMQP clients to connect to and interact with. RabbitMQ's routing tables are shared across the entire cluster, and delivery of messages published at one node to a queue residing at another node is seamless. For details see the clustering guide. We are currently developing further support for live failover of AMQP resources within a cluster. Please see our roadmap.

Q. Will RabbitMQ support advanced transports such as RDMA and SCTP?

A. Yes. Erlang allows bindings to any C library; Erlang's foreign-function interface will be used to interface to existing implementations of non-TCP/IP AMQP transports.

Q. Will RabbitMQ use SMP, when it's available?

A. Yes. Erlang supports both single-node SMP configurations and configurations with multiple Erlang nodes running in a cluster within a single host.

Q. Does RabbitMQ support store-and-forward?

A. Yes, RabbitMQ, and AMQP in general, supports store-and-forward-style exchanges and queues.

Q. Can I create RabbitMQ applications using web frameworks (eg Spring, or Ruby-On-Rails)?

A. Yes. To begin with, Spring has good support for message access in Java: the RabbitMQ Java client is simply a POJO library. The same is true for other POJO-based systems, such as Mule. If you specifically wish to use JavaEE EJBs as message accessors, use a stateful session bean.

For frameworks in other languages, such as Ruby or in-browser Javascript, we plan on providing both RESTful HTTP access to AMQP features as well as AJAX- and web-services-style interfaces, allowing rapid development of web applications that take advantage of AMQP messaging.

RabbitMQ Architecture

Q. How is authentication and authorization supported in RabbitMQ?

A. AMQP uses SASL (see also Wikipedia on SASL) for authentication of AMQP clients. Our current support for SASL is limited to the PLAIN authentication mechanism; see our roadmap for our future SASL plans.

Authorization is implemented in RabbitMQ using a distributed database table mapping users to virtual hosts and realms.

Message authentication, for instance using cryptographic message authentication codes, is not defined by the AMQP specification, but as for encryption, could be very easily implemented within the RabbitMQ client library.

Q. What platforms will RabbitMQ run on?

A. RabbitMQ is developed on Debian Linux and Mac OS X. As the server component is written in Erlang, and the client component in Java, it is platform-neutral. Erlang runs on the following platforms (taken from Erlang's FAQ):

  • Solaris (including 64 bit)
  • BSD
  • Linux
  • OSX
  • TRU64
  • Windows NT/2000/2003/XP
  • Windows 95, 98
  • VxWorks

Q. What is shared session clustering?

A. For those familiar with typical tiered session/application architectures such as JavaEE, it might help to think of the AMQP exchanges as corresponding to a logical session tier, and AMQP queues as corresponding to a logical application tier. From this point of view, RabbitMQ's routing tables can be seen as clustered shared session state. RabbitMQ uses the OTP distributed database, Mnesia, to reliably and persistently replicate session state across all nodes in a cluster.

Q. How does clustering work?

A. RabbitMQ's current clustering mechanisms are built around Erlang's native support for reliable distributed programming. This is a deeply sophisticated framework, implemented at the language and virtual-machine level.

Some of the unique points about Erlang's networking and distribution model are:

  • it scales to hundreds of thousands of parallel processes ("green threads") within a single virtual machine, bounded only by available memory;
  • the failure model for local inter-process communication (IPC) is the same as that for distributed IPC, making the transition from non-distributed to distributed code very smooth;
  • binary pattern-matching constructs within the language ensure straightforward and efficient translation between wire-level encodings and internal Erlang data structures;
  • finally, the OTP libraries shipped with the Erlang distribution include the notion of a supervisor/worker relationship, where supervisor processes monitor and restart worker processes under their control, making management of the entire process hierarchy within a server deterministic and automatic.

Within an Erlang node cluster, Erlang's native high-speed messaging is used to provide an efficient way of distributing work across the cluster. Individual AMQP clients connect to machines within the cluster, and Erlang's distributed routing database routes AMQP messages to the appropriate endpoints.

For more detail on Erlang's benefits in an AMQP setting, please see this page.

Q. How does failover and HA work?

A. Currently, AMQP durable queues and their persistent messages are recovered from disk at each server restart. Exchange failover is handled by OTP's support for clustering (see above).

Future releases will support live failover using, for instance, a combination of the "known hosts" field in connection.open-ok and the connection.redirect message. Erlang's built-in fault-tolerant database will ensure that routing information is preserved, and the OTP supervisor architecture in combination with AMQP's transactional features will ensure that durable messages are not lost.

Q. How does addressing work in AMQP?

A. Like other messaging systems, AMQP and RabbitMQ provide internal addressing mechanisms based on notions such as routing keys, exchange addresses, realms and queue names.

These are managed on a per-virtual-host basis. Please see the AMQP specification for details of the scoping of the various names in the network. We are actively investigating interoperation with other addressing schemes and systems.

Integration

Q. What clients will RabbitMQ support?

A. At present, the RabbitMQ suite offers a client library for Java applications. Support for in-browser Javascript (via AJAX mechanisms, including JSON and JSON-RPC) and native Erlang client access is planned for the near future. We will develop client interfaces to RabbitMQ's features in response to demand.

Q. How do I use RabbitMQ from Java?

A. Please see our Java client API guide.

The RabbitMQ client library is supplied as a JAR file, rabbitmq-client.jar, with a minimal number of external dependencies (currently backport-util-concurrent-2.2.jar, commons-io-1.2.jar, and commons-lang-2.0.jar).

Q. I use .NET - what do I need to know to use RabbitMQ?

A. One strong possibility is to use IKVM to access the Java client from .NET. Other alternatives include using an ESB such as Mule to connect to .NET applications, or using AMQP-over-HTTP or over JSON-RPC to access AMQP functionality through web-service-style interfaces.

Q. Does RabbitMQ implement a JMS Interface?

A. We're tracking the development of the core AMQP standard, which is at present working on defining a good mapping from AMQP to the concepts used in JMS. If you want to use JMS with AMQP today, please email us.

Q. How do I integrate RabbitMQ with other systems?

A. We plan on exposing RabbitMQ to the world of services accessible via Mule - please see below.

Q. Does RabbitMQ interface with Mule?

A. Mule is an Enterprise Service Bus (ESB), which is an essential part of our roadmap for integration with other protocols and legacy networks.

Q. Does RabbitMQ interface to HermesJMS?

A. We plan on implementing an interface to allow HermesJMS to browse and manage queues on RabbitMQ brokers.

Q. Can RabbitMQ be used with existing market data systems? What about other asynchronous models like RSS, XMPP, ...?

A. Absolutely. Options for integration range from using AMQP clients natively to pass messages between your AMQP network and your existing market data system, through configuring routing between your various networks at a very high level using, for instance, an ESB such as Mule.

Q. How do I integrate with JavaEE transactions?

A. A bundled bridge between AMQP's transaction model and JavaEE's transaction model is on our roadmap.

Q. How do I integrate with LDAP and other directories?

A. Our current roadmap calls for integration with user and configuration databases besides the internal one. We are investigating options ranging from direct integration at the broker implementation level, through using AMQP itself to access other networked systems.

How-To

Q. "Last" message on Topic

A classic use-case for messaging systems is the example of subscribing to a topic of, say, 'market prices' for an asset. By that a client very often means, "send me the last published one and then subscribe me to any update". Sometimes it is not necessary if the asset is liquid (MSFT) but if it is illiquid, the client should receive the last available message, as it's possible there will be a long delay before the next update.

Does RabbitMQ support this?

A. This is not something AMQP provides out-of-the-box, although one can imagine a solution involving a trivial service (a simple hash table, in effect) answering requests for most-recent-price served via AMQP. The client would subscribe to the stream as usual, and in parallel ask the service for the most recent price.

Such a service could be implemented either inside the RabbitMQ broker, in Erlang, thus taking advantage of Erlang's distribution and high-availability properties, or it could be implemented outside the broker, as a regular AMQP client.

Our roadmap includes just such an integrated service.

Q. Channel Encryption

Say I want to transfer a message across the AMQP network, without depending on the target being there, and that the 'channel' between the message source client and the message target client should be strongly encrypted and authenticated. Is this something RabbitMQ provides?

A. There are two kinds of encryption that could be supported by an AMQP implementation.

The first is encryption of the AMQP stream. There's no provision in the standard for that, yet, but wrapping the stream in a TLS layer is a straightforward extension (please see the section on SSL/TLS in our API guide). The notion of addressing for AMQP entities (ie. URLs for AMQP queues and exchanges) is under active development at the moment. It seems likely that once that settles down, it will quickly be extended to standardise a TLS layering for AMQP ("amqp:" -> "amqps:" ?).

The second kind of encryption is encryption of individual messages. AMQP is silent on this front as well, but using a layer of encryption in the clients is very easy. Adding support for this to RabbitMQ's client library would be a simple matter.

Q. How do I do management, monitoring and control?

A. AMQP includes some management capability through the current protocol design, and further features are currently under development by the Working Group. RabbitMQ is tracking these, and in addition, RabbitMQ will make use of both HermesJMS and Erlang/OTP's existing proven management libraries to provide a complete management suite (or "HQ"). When used in a virtual appliance, additional management features will be bundled.

Q. How do I configure access control?

A. Please see the section on access control in the RabbitMQ admin guide.

Q. Instrumentation, logging and debugging

A. RabbitMQ uses AMQP internally to the broker to notify interested parties of significant events. Administration and management exchanges are provided for tools to bind to.

In addition, there is some discussion in the AMQP community of the need for replay as a built-in feature.

Q. How do I test RabbitMQ?

A. You can test the RabbitMQ broker by:

  • connecting to our demo server using one of the RabbitMQ example programs
  • connecting to our demo server using another AMQP 0-8 compatible client
  • trying the Java web-startable demo of Camstream
  • Downloading the broker and experimenting with it on your own machine

Q. Can I test my AMQP client against RabbitMQ?

A. Yes: you can either run your client against our public demonstration server, or download and run a server of your own to experiment with.

Q. How do I do Java client retry?

A. The AMQP "TX" message class, supporting transactional behaviour, is supported in the Java API.

Q. How do I archive to RDBMS?

A. There are two ways of achieving this:

  • create a consumer which retrieves messages transactionally from a queue, writing them transactionally to an RDBMS, or
  • integrate with RabbitMQ's internal transaction and persistence mechanism to provide a write-behind log of AMQP activity in your RDBMS.

Full support for this is on our roadmap.

Q. How do I tune the scope of replication? Will it affect performance?

A. Since replication is used mainly to propagate routing table information between Erlang nodes within a cluster, it has a low impact on performance to begin with; also, it is trivial to arrange for a "queue-only" Erlang node to join the cluster, only able to consume from queues, which can then avoid being part of the routing-table replication system.

Performance and Scalability

Q. How low can latency be?

A. We have seen latencies averaging less than a millisecond; for heavy, steady-state loads, latency hovers around the 4-millisecond mark (including two network hops).

Q. How fast is RabbitMQ in persistent mode?

A. From our testing, we expect easily-achievable throughputs of 4000 persistent, non-transacted one-kilobyte messages per second (Intel Pentium D, 2.8GHz, dual core, gigabit ethernet) from a single RabbitMQ broker node writing to a single spindle.

Please let us know how RabbitMQ performs for you!

Q. How do I switch off persistence/transactionality?

A. To disable transactionality, use an AMQP channel which is not in "TX" mode (ie. do not call tx.select). To disable message persistence, set the "delivery mode" field in the basic-class properties for the message to either absent or to one, as mode two enables persistence.

Q. How does RabbitMQ's performance scale with clustering?

A. Based on typical OTP application scalability, we expect close-to-linear scaling.

Q. How many virtual hosts can I run and where?

A. There is no hard limit to the number of virtual hosts that can be configured within a RabbitMQ cluster, save available disk and memory resources.

Virtual hosts within AMQP are decoupled from the physical network layout: one virtual host can be accessed from multiple clustered brokers, just as many virtual hosts can be accessed within a single broker. Virtual hosts act solely as a namespacing mechanism for AMQP resources.

General

Q. How is RabbitMQ priced?

A. RabbitMQ is free software in both the gratis and libre senses, licenced under the Mozilla Public License.

Q. What support options are available for RabbitMQ?

A. RabbitMQ will be supported, both commercially and as an open-source project, by both LShift and by CohesiveFT as virtual appliances. For inquiries about support, please email support@rabbitmq.com. For inquiries specifically about virtual appliances, contact support@cohesiveft.com.

Q. How is RabbitMQ licenced (for redistribution, etc.)?

A. RabbitMQ will be distributed under the open-source Mozilla Public License.

If you would like to explore other licensing options, please get in touch with us by emailing info@rabbitmq.com.

Q. What is the RabbitMQ roadmap?

A. A list of planned features for future releases of RabbitMQ can be found at our roadmap page.

Q. How do I contribute to the RabbitMQ project?

A. Welcome aboard! We love contributions - here are some suggestions for what you can work on:

  • any of the items from this FAQ and our roadmap
  • packaging for other operating systems
  • integration with other messaging systems
  • integration with other management systems
  • integration with other user and permission database systems
  • improvements to the client, and work towards other clients
  • new services for running with RabbitMQ, either within the broker or adjacent to it
  • ... and so on!

We're happy to hear from anyone who wants to get involved or is curious about how we intend to manage the project.

We plan on making the RabbitMQ project's source code available via a public Subversion server shortly, although of course we welcome plain old patches, as well.

Q. Will RabbitMQ be distributed with my Operating System?

A. We hope that as RabbitMQ develops, it will be included with various operating systems and distributions. We already have packages available for Debian GNU/Linux; we are developing packages for integrating smoothly with Fedora Linux, Microsoft Windows and Apple Mac OS X, and hope that eventually our packages will be accepted into core OS distributions.

Q. How should I interpret RabbitMQ's version numbering?

A. RabbitMQ is versioned with the scheme major.minor.patch.

  • major: This number indicates the major version of the software. It is only changed for significant alterations to the software, such as (for instance) a total rewrite.
  • minor: A change in this number indicates new or significantly altered features.
  • patch: This number changes to make each distinct packaging of the software uniquely numbered. Small changes such as bug-fixes or packaging alterations may cause a change in only the patch number.

We also label each release with a status. This is either alpha, beta, rc (for "release candidate") or final, and indicates the maturity of the release. The status is mentioned in the release announcement and on the download page.

Q. I have a question!

A. Please join our mailing list or send questions to info@rabbitmq.com. We'd love to hear from you.