Menu

Archive for the ‘Uncategorized’ Category

What’s new in RabbitMQ 3.6.0

Monday, December 28th, 2015

We are pleased to announce the immediate availability of RabbitMQ 3.6.0, a new version of the broker that comes packed with lot of new features. Before we go on, you can obtain it here: http://www.rabbitmq.com/download.html.

This release brings many improvements in broker features, development environment for our contributors, and security. Let’s take a look at some of the most significant ones.

(more…)

Jason and Alvaro’s excellent Rabbit book

Tuesday, May 29th, 2012

Here at Rabbit HQ we've been enjoying "RabbitMQ in Action", the introduction to RabbitMQ and messaging.  Part of the Manning series, the book is written by Jason Williams and Alvaro Videla, both well known for their many contributions to the Rabbit community.

Today we'd like to say thank-you to Jason and Alvaro.  Thank-you Jason and Alvaro!  You did an amazing job and infinite beers are on us.

But there's more...  Manning have kindly offered a promotional discount of 37% to readers of this blog.  All is revealed below, in a guest post by Jason Williams himself...

RabbitMQ in Action is here

Well, it's finally here. After 18 months of writing, re-writing and updating, RabbitMQ in Action is finished and in the flesh. It's hard to believe that when we started, RabbitMQ was at version 1.8.0 and now we're at 2.8.2. So much has changed in Rabbit that required rewrites of whole sections along the way, that it feels like we're really at 5 or 6.0. It's a testament to the Rabbit team members that helped us that the book kept pace with it all. So now that it's out why should you read it (besides the 37% discount code below)?

If you feel like you want a deeper understanding than the online tutorials offer, we wrote this for you. Whether it's figuring out clustering and mirrored queues, or just getting a better understanding of messaging fabrics (queues, bindings and routing exchanges, relays and federations.), our goal was to write the book we wished had existed when we started, and that we hope will help you. From the management console and API to building real world applications and plugins, we've tried to cover everything you need to get a good foundation of Rabbit under your belt…and hopefully that you can use as a desk reference too.

Lots of example code on Github to get you started

One thing we tried to focus on was using RabbitMQ to link together different applications written in completely different languages.  That's one of the main reasons we wrote the examples in Python and PHP. However, we had two other reasons also:

1.) Python reads almost like pseudo-code and produces incredibly readable programs…which makes it an excellent teaching language. You can focus on what the example program's doing, without a lot of class declarations and boiler plate clouding up the works.

2.) There are a ton of books on messaging targeted at Java and the old-line enterprise brokers. We wanted to write something different... something that was easier to read and more accessible to people without any background in messaging. RabbitMQ in Action is very much a book for people of all languages and backgrounds. Writing in Python and PHP helped us do that (there's appendices on using Rabbit with Java and .NET too).

With that last one in mind, we’ve done something a little different than other Manning books…all of our examples are in a public repo on Github.

We did this so that if you feel like converting the examples into the language of your choice to help those like you, you can. As long as the license on your contribution is BSD, we'll merge in your pull requests and hopefully build a huge library of RabbitMQ examples that can help everyone. There are already Ruby versions of the examples merged in!

So if those aren't reasons enough to give RabbitMQ in Action a shot…how about a 37% discount just because you read this blog? :)

Save 37% on RabbitMQ in Action with Promotional Discount Code 12rmqb when you checkout at the Manning web site.

RabbitMQ Performance Measurements, part 2

Wednesday, April 25th, 2012

Welcome back! Last time we talked about flow control and latency; today let's talk about how different features affect the performance we see. Here are some simple scenarios. As before, they're all variations on the theme of one publisher and one consumer publishing as fast as they can.

(more…)

London Realtime hackweekend

Tuesday, April 17th, 2012

Over the weekend, RabbitMQ co-sponsored London Realtime, two nights and two days of unadulterated hackery. It was all put on by the apparently indefatigable* crew at GoSquared, a very impressive debut effort.

As a co-sponsor we had one of the iPad prizes to award. We decided to allow hacks that used one or more of RabbitMQ, SockJS, or Cloud Foundry. This meant that about half of the twenty-seven hacks were eligible when it came to judging, making the choice rather difficult. (more…)

Puka – rethinking AMQP clients

Friday, July 8th, 2011

I fundamentally disagree with the APIs exposed by our current AMQP client libraries.

There is a reason why they’re imperfect: we intentionally avoided innovation in APIs since the beginning. The purpose of our client libraries is to expose generic AMQP, not any one view of messaging. But, in my opinion, trying to map AMQP directly to client libraries APIs is just wrong and results in over-complication and abstractions hard to use.

There is no common ground: the client libraries blindly following AMQP model will be complex; easy to use client libraries must to be opinionated.

(more…)

RabbitMQ 2.5.0 released

Thursday, June 16th, 2011

The RabbitMQ team is delighted to announce the release of RabbitMQ 2.5.0.

This release fixes a number of bugs. In particular:

  • recovery has been simplified, improving startup times when many exchanges or bindings exist
  • bindings are recovered between durable queues and non-durable exchanges on restart of individual cluster nodes
  • better performance under high load and memory pressure
  • source compatibility with the new Erlang R14B03 release

New features include:

  • tracing facility for debugging incoming and outgoing messages, (see firehose)
  • improved inbound network performance
  • improved routing performance
  • new rabbitmqctl commands ('report', 'environment', and 'cluster_status')

For details see the release notes.

As always, we welcome any questions, bug reports, and other feedback on this release, as well as general suggestions for features and enhancements in future releases. Mail us via the RabbitMQ discussion list, or directly at info@rabbitmq.com.

Very fast and scalable topic routing – part 2

Monday, March 28th, 2011

In our previous blog post we talked about a few approaches to topic routing optimization and described the two more important of these in brief. In this post, we will talk about a few things we tried when implementing the DFA, as well as some performance benchmarking we have done on the trie and the DFA. (more…)

Ruby AMQP Benchmarks

Tuesday, March 1st, 2011

I decided to run some benchmarks of my AMQP encoder/decoder (AMQ Protocol gem) against the old one in the AMQP gem to see whether it performs better or not. So far I did only the most basic optimisations like storing reusable values in constants, nothing special (yet).

I did two sets of benchmarks: CPU time benchmarking using my fork of RBench with support for custom formatters (like writing results into a YAML file) and memory benchmarking using Object.count_objects (Ruby 1.9). (more…)

Broker vs Brokerless

Wednesday, September 22nd, 2010

The RabbitMQ team has been working with Martin Sustrik to provide code and documentation for using RabbitMQ and ZeroMQ together.  Why is this a good idea?  Because the broker and brokerless approaches are complementary.  We'll be posting more about this as the codebase evolves.  This post is introductory and can be seen as commentary on Ilya Grigorik's excellent introduction to ZeroMQ and the InfoQ summary of Ilya's article.

I like ZeroMQ and think it is useful - of which more below.  But I have seen some brash claims made on its behalf.  This can lead to confusion.

So what is the 'brokerless' model?  In the comments to Ilya's and the InfoQ post, ZeroMQ is compared to SCTP and to JGroups.  These are important technologies and form a helpful starting point for thinking about brokerless messaging patterns.  Let's look at what you might need if you combine messaging (like SCTP) with pubsub groups (like JGroups) to make arbitrary networks using 'brokerless' peers.

Some things you might need in a brokerless network

If you set up a brokerless messaging network, three things that you might need are: discovery, availability and management.

Discovery is the problem of maintaining a roster of peers that a system can send messages to, and who can join this roster.

Availability is the problem of dealing with peers disappearing from time to time.  For example if you have 50 subscribers to a feed, and only 40 of them are available to receive updates, should you keep a copy of their messages until they reappear?  That could mean "for a very long time".   And if you do keep messages and lists of "who has seen what", then where is it best to do this?

This is also a problem when message receivers do not respond quickly.  To quote from Martin Sustrik of ZeroMQ, "You can never differentiate between 'network error and 'no response received'. TCP in no better. You'll have accept that or keep with a single box."

Management is an interesting area for analysis too.  ZeroMQ's model aligns messaging closely with sockets.  This means that, like in TCP, 'any' communication network can be implemented in such a way that it provides some messaging capability.  But, networks can be arbitrarily complex.  For example unless you don't care about it (and you may not) management of "who is connected to who, and who can be connected to who" can get complicated.   This kind of management problem gets more difficult the more you scale.  Models like JGroups usually make this problem go away by making a simplifying assumption, i.e.: everyone in the group talks to everyone else in the group.  Easy :-)

I am not suggesting that you always need these things.  The ZeroMQ philosophy is to home right in on networking, and this creates focus.  But if you do need them then you might end up implementing them yourself.  Enter the broker...

How can a broker help to solve these problems

Brokers can provide solutions for discovery, availability and management.  They can also form reliable networks, e.g. for email delivery and instant messaging services.

First: what is a 'broker'?  It is both a leader, and an intermediary.

A broker is a leader.  In distributed computing, the problems of management, discovery and availability are typically solved by electing a leader among the set of distributed components.  In the world of "messaging", such a leader is usually known as a "broker".  Stating that in order to be a leader, you need to be a broker, makes it much easier to work out who is the leader, than in a completely brokerless system in which "anyone can lead, but nobody knows how".

A broker is also an intermediary.  For example, instead of having to connect everyone in the group directly, communicators simply connect to the broker (or brokers).  A broker may also be used to solve availability problems such as "offline consumer", by providing persistence and managing recovery on behalf of systems that cannot do it themselves.

Thus, brokers simplify network design by making reasonable assumptions.  Of course, when those assumptions don't hold, you may not want a broker.

Brokers are not 'centralized'

A commonly held misconception about brokers is that they are 'centralized'.  Brokers are NOT necessarily a 'centralized' solution.  Intermediaries can be decentralized.  You can have multiple brokers in a single network in order to increase throughput and availability.  Sometimes these networks of servers are called federations.  Note that individual brokers do not need to be 'highly available' in order to have a redundant network of servers.

This is, for example, how email (SMTP) and XMPP networks work.  Both email and instant messaging are brokered models, and both use multiple brokers in a simple and redundant way.  For example, mail transfer agents provide a delivery and routing network for email.  It would be difficult to come up with a design for this that was completely peer to peer, without reinventing 'special peers' - also known as brokers.

So what model is simplest?

Peer to peer models are not inherently more or less simple than brokered models.  If you do not need discovery, availability, management, or intermediation then it may be simpler to not use them.  But if you need them, it may be simpler to not implement them yourself.

Networks of servers (brokers) are not more or less redundant or decentralized than networks of clients (peers).  Both the broker and brokerless model have their pros and cons in terms of reliability, and other considerations eg latency.

The two models solve different problems.

For example, RabbitMQ and ZeroMQ are complementary.  From a RabbitMQ point of view ZeroMQ is a 'smart client' that can use its buffers like a queue.  That's useful in some cases.  From a ZeroMQ point of view, RabbitMQ is a network device that provides services that you would not necessarily want to have to implement yourself.

We want our customers and users to always have the best toolset available which is why we have provided the Github repo for you to play with.  Thanks again to Martin Sustrik for his work on this.

Watch this space for more on this interesting area of work and discussion.

RabbitMQ on github

Monday, September 20th, 2010

We've received quite a few requests recently for us to put the RabbitMQ code on github.

RabbitMQ is open source, and the Mercurial repositories where we work on the code are publicly accessible. But github is rapidly establishing itself as the Facebook of open-source development: It makes it easy to follow projects and participate in their development, all within a slick web-based UI.

So from today, we are mirroring our repositories to github. You can find them at http://github.com/rabbitmq. The repositories on github track our Mercurial repositories with a delay of a few minutes.

The main development of RabbitMQ will continue to take place on Mercurial. Converting our development workflow and infrastructure to git would take a lot of effort that we'd prefer to spend improving RabbitMQ. And besides, members of the team differ in their opinions about the relative merits of hg and git.

If you wish to contribute to RabbitMQ, we are happy to receive changes via github, or Mercurial hosting sites such as bitbucket, or even as old-fashioned patches!