Breaking things with RabbitMQ 3.0

November 19th, 2012 by Simon MacMullen

RabbitMQ includes a bunch of cool new features. But in order to implement some of them we needed to change some things. So in this blog post I'm going to list some of those things in case you need to do anything about them. Read the rest of this entry »

MQTT Adapter

September 12th, 2012 by Emile Joubert

I've written a plugin for RabbitMQ that adds support for the MQTT 3.1 protocol. MQ Telemetry Transport is a light-weight PUB/SUB protocol designed for resource-constrained devices and limited bandwidth situations, making it ideally suited to sensors and mobile devices. The implementation is a protocol adapter  plugin, allowing MQTT clients to connect to a RabbitMQ broker simultaneously with clients implementing other protocols. We encourage projects that demand the combination of a low-overhead protocol on a robust, scalable broker with high reliability and enterprise features to consider this option. Read the rest of this entry »

Jason and Alvaro’s excellent Rabbit book

May 29th, 2012 by alexis

Here at Rabbit HQ we've been enjoying "RabbitMQ in Action", the introduction to RabbitMQ and messaging.  Part of the Manning series, the book is written by Jason Williams and Alvaro Videla, both well known for their many contributions to the Rabbit community.

Today we'd like to say thank-you to Jason and Alvaro.  Thank-you Jason and Alvaro!  You did an amazing job and infinite beers are on us.

But there's more...  Manning have kindly offered a promotional discount of 37% to readers of this blog.  All is revealed below, in a guest post by Jason Williams himself...

RabbitMQ in Action is here

Well, it's finally here. After 18 months of writing, re-writing and updating, RabbitMQ in Action is finished and in the flesh. It's hard to believe that when we started, RabbitMQ was at version 1.8.0 and now we're at 2.8.2. So much has changed in Rabbit that required rewrites of whole sections along the way, that it feels like we're really at 5 or 6.0. It's a testament to the Rabbit team members that helped us that the book kept pace with it all. So now that it's out why should you read it (besides the 37% discount code below)?

If you feel like you want a deeper understanding than the online tutorials offer, we wrote this for you. Whether it's figuring out clustering and mirrored queues, or just getting a better understanding of messaging fabrics (queues, bindings and routing exchanges, relays and federations.), our goal was to write the book we wished had existed when we started, and that we hope will help you. From the management console and API to building real world applications and plugins, we've tried to cover everything you need to get a good foundation of Rabbit under your belt…and hopefully that you can use as a desk reference too.

Lots of example code on Github to get you started

One thing we tried to focus on was using RabbitMQ to link together different applications written in completely different languages.  That's one of the main reasons we wrote the examples in Python and PHP. However, we had two other reasons also:

1.) Python reads almost like pseudo-code and produces incredibly readable programs…which makes it an excellent teaching language. You can focus on what the example program's doing, without a lot of class declarations and boiler plate clouding up the works.

2.) There are a ton of books on messaging targeted at Java and the old-line enterprise brokers. We wanted to write something different... something that was easier to read and more accessible to people without any background in messaging. RabbitMQ in Action is very much a book for people of all languages and backgrounds. Writing in Python and PHP helped us do that (there's appendices on using Rabbit with Java and .NET too).

With that last one in mind, we’ve done something a little different than other Manning books…all of our examples are in a public repo on Github.

We did this so that if you feel like converting the examples into the language of your choice to help those like you, you can. As long as the license on your contribution is BSD, we'll merge in your pull requests and hopefully build a huge library of RabbitMQ examples that can help everyone. There are already Ruby versions of the examples merged in!

So if those aren't reasons enough to give RabbitMQ in Action a shot…how about a 37% discount just because you read this blog? :)

Save 37% on RabbitMQ in Action with Promotional Discount Code 12rmqb when you checkout at the Manning web site.

Introducing RabbitMQ-Web-Stomp

May 14th, 2012 by Marek Majkowski

For quite a while here, at RabbitMQ headquarters, we were struggling to find a good way to expose messaging in a web browser. In the past we tried many things ranging from the old-and-famous JsonRPC plugin (which basically exposes AMQP via AJAX), to Rabbit-Socks (an attempt to create a generic protocol hub), to the management plugin (which can be used for basic things like sending and receiving messages from the browser).

Over time we've learned that the messaging on the web is very different to what we're used to. None of our attempts really addressed that, and it is likely that messaging on the web will not be a fully solved problem for some time yet.

That said, there is a simple thing RabbitMQ users keep on asking about, and although not perfect, it's far from the worst way do messaging in the browser: exposing STOMP through Websockets.

Read the rest of this entry »

Some queuing theory: throughput, latency and bandwidth

May 11th, 2012 by Matthew Sackman

You have a queue in Rabbit. You have some clients consuming from that queue. If you don't set a QoS setting at all (basic.qos), then Rabbit will push all the queue's messages to the clients as fast as the network and the clients will allow. The consumers will balloon in memory as they buffer all the messages in their own RAM. The queue may appear empty if you ask Rabbit, but there may be millions of messages unacknowledged as they sit in the clients, ready for processing by the client application. If you add a new consumer, there are no messages left in the queue to be sent to the new consumer. Messages are just being buffered in the existing clients, and may be there for a long time, even if there are other consumers that become available to process such messages sooner. This is rather sub optimal.

So, the default QoS prefetch setting gives clients an unlimited buffer, and that can result in poor behaviour and performance. But what should you set the QoS prefetch buffer size to? The goal is to keep the consumers saturated with work, but to minimise the client's buffer size so that more messages stay in Rabbit's queue and are thus available for new consumers or to just be sent out to consumers as they become free. Read the rest of this entry »

RabbitMQ Performance Measurements, part 2

April 25th, 2012 by Simon MacMullen

Welcome back! Last time we talked about flow control and latency; today let's talk about how different features affect the performance we see. Here are some simple scenarios. As before, they're all variations on the theme of one publisher and one consumer publishing as fast as they can.

Read the rest of this entry »

London Realtime hackweekend

April 17th, 2012 by Michael

Over the weekend, RabbitMQ co-sponsored London Realtime, two nights and two days of unadulterated hackery. It was all put on by the apparently indefatigable* crew at GoSquared, a very impressive debut effort.

As a co-sponsor we had one of the iPad prizes to award. We decided to allow hacks that used one or more of RabbitMQ, SockJS, or Cloud Foundry. This meant that about half of the twenty-seven hacks were eligible when it came to judging, making the choice rather difficult. Read the rest of this entry »

RabbitMQ Performance Measurements, part 1

April 17th, 2012 by Simon MacMullen

So today I would like to talk about some aspects of RabbitMQ's performance. There are a huge number of variables that feed into the overall level of performance you can get from a RabbitMQ server, and today we're going to try tweaking some of them and seeing what we can see.

Read the rest of this entry »

How to compose apps using WebSockets

February 23rd, 2012 by Marek Majkowski

Or: How to properly do multiplexing on WebSockets or on SockJS

As you may know, WebSockets are a cool new HTML5 technology which allows you to asynchronously send and receive messages. Our compatibility layer - SockJS - emulates it and will work even on old browsers or behind proxies.

WebSockets conceptually are very simple. The API is basically: connect, send and receive. But what if your web-app has many modules and every one wants to be able to send and receive data?

Read the rest of this entry »

AtomizeJS: Distributed Software Transactional Memory

February 21st, 2012 by Matthew Sackman

AtomizeJS is a JavaScript library for writing distributed programs, that run in the browser, without having to write any application specific logic on the server.

Here at RabbitMQ HQ we spend quite a lot of time arguing. Occasionally, it's about important things, like what messaging really means, and the range of different APIs that can be used to achieve messaging. RabbitMQ and AMQP present a very explicit interface to messaging: you very much have verbs send and receive and you need to think about what your messaging patterns are. There's a lot (of often quite clever stuff) going on under the bonnet but nevertheless, the interface is quite low-level and explicit, which gives a good degree of flexibility. Sometimes though, that style of API is not the most natural fit for the problem you're trying to solve - do you really reach an impasse and think "What I need here is an AMQP-message broker", or do you, from pre-existing knowledge, realise that you could choose to use an AMQP-message broker to solve your current problem?

AtomizeJS exists at the opposite end of the spectrum. There is lots of messaging involved, but you almost never get to see any of it. Instead, you write transactions in JavaScript that modify objects, and those objects are shared between all clients that are connected to the same AtomizeJS server. The API that you're given lets you do slightly more powerful things than you're used to from database transactions, in particular, retry allows you to abort a transaction but then restart it automatically once someone else has changed one of the variables you read. This means you have the observer-pattern, and from that you can then build any explicit messaging patterns you want. In most cases, I doubt you'll be building APIs that say send or receive, instead you'll be building richer data-structures - work queues, shared dictionaries etc. The question to pose then is: is it easier to build these things based on top of a transaction-like API such as offered by AtomizeJS, or on top of an explicit messaging API such as offered by RabbitMQ and AMQP brokers. There is no one solution and horses-for-courses etc, but please leave your thoughts below.

The gain that AtomizeJS provides is not so much in the use of STM from the browser, but the use of STM against a distributed object. This allows you to trivially share state between browsers, modify it safely in intuitive terms, and thus build your applications with little or no application-specific server-side code. Currently, it's a little clunky to use with browsers that don't support bleeding-edge JavaScript features (though I've provided some tooling to try and mitigate this), and everything does work with the latest versions of Chrome, Firefox, IE, Safari, and Opera. Please have a go and let us know what you think!