Posts Tagged ‘broker’

High Availability in RabbitMQ: solving part of the puzzle

Tuesday, October 25th, 2011

In RabbitMQ 2.6.0 we introduced Highly Available queues. These necessitated a new extension to AMQP, and a fair amount of documentation, but to date, little has been written on how they work. (more…)

RabbitMQ and AMQP 0-9-1

Wednesday, August 4th, 2010

Since the beginning, RabbitMQ has implemented the 0-8 version of the AMQP specification. This was the first publicly-available version, but there's been plenty of development since then. In particular, we've wanted to support the 0-9-1 version of AMQP for a while now.

For around the last year we've maintained a 0-9-1-supporting branch of the broker and clients in Mercurial, and interested users have been running that version as it matures. Well, it's finally mature and we're now merging the 0-9-1 support into the default branch. This means it'll be in the next release.

At this point, you're probably wondering what the differences are between 0-8 and 0-9-1. Well, the good news is that the differences are not huge, and are entirely sensible. 0-9-1 mostly cleans up the 0-8 spec, explaining more clearly how the broker and clients are expected to behave in certain edge cases, and removing a whole load of ambiguity and half (or less) thought out features from 0-8.

In fact, if you're running 1.8.0 or later, you're already running a broker with most of the semantic changes from 0-9-1. But the wire protocol has changed a bit too, so it matters whether you're speaking 0-9-1 or 0-8.

For the Java and Erlang clients, we're intending to simply switch to supporting AMQP 0-9-1 exclusively in the next release. The .NET library already supports multiple protocols, so we're adding 0-9-1 as another option (and making it the default). For the broker we're going to support both AMQP 0-9-1 and AMQP 0-8. For any other client libraries we encourage you to speak to the library developer ;)

This means that this time it's safe to upgrade just your broker, or your broker and all your clients, but it's not safe to upgrade your clients without upgrading your broker!

Thank you for your attention.

Well, I’ll let you go … basic.reject in RabbitMQ

Tuesday, August 3rd, 2010

Support for AMQP's basic.reject just landed on default. It's taken this long because we couldn't agree on a single set of semantics that followed the specification, was actually useful, and wasn't too complicated to implement.

First up, this is what RabbitMQ's basic.reject will do: if you supply requeue=false it will discard the message (this is in lieu of dead-lettering it, until we have a dead letter feature); if you supply requeue=true, it will release it back on to the queue, to be delivered again.

The first is useful from a error handling point of view; if your application cannot process a particular message, you can get rid of it. At the minute, it's semantically the same as just acking the message; but, given a dead-letter mechanism, it will mean unprocessable messages can be picked up elsewhere for diagnosis.

The second, with requeue=true, is useful for example if your application needs a "message lock" semantics. In this scenario, a consumer can be delivered a message, then decide not to deal with it after all, and place it back on the queue. Note that RabbitMQ doesn't take care to stop the same consumer getting the message again -- see below.

The AMQP 0-9-1 specification says a number of seemingly incompatible things about basic.reject. For a start, it says in the method description

The client MUST NOT use this method as a means of selecting messages to process.

and in the specification XML (but not in the generated PDF),

The server MUST NOT deliver the message to the same client within the context of the current channel. The recommended strategy is to attempt to deliver the message to an alternative consumer, and if that is not possible, to move the message to a dead-letter queue. The server MAY use more sophisticated tracking to hold the message on the queue and redeliver it to the same client at a later stage.

This seems to suggest that the server has to take care not to deliver the message to the same consumer twice, but consumers are not allowed to rely on this prohibition. This means basic.reject could either redeliver the message or dead-letter it, which makes it useless for the "message lock" scenario given above.

So, we have chosen to implement the simplest thing that is useful, which is to re-enqueue the message and treat it as though it were completely new. This means the consumer can receive again a message it has rejected.