Menu

This Month in RabbitMQ — May 2019

May 13th, 2019 by Michael Klishin

Couple of key public service announcements this month. First, the deadline for submitting a talk for RabbitMQ Summit 2019 (5 November in London UK) was May 10. We had a great line-up last year at the inaugural event and we’re looking forward to an even better event this fall.

Then, on May 23, we’ll be doing an overview of what’s new in RabbitMQ 3.8 (beta 4 of which has dropped recently). Whether you’re a couple versions behind, or on the latest 3.7.14 release, you’re going to want to learn about the latest features and changes.

 

Project updates

  • RabbitMQ 3.7.15-beta.1 is available for community testing
  • And so is 3.8.0-beta.4. See next.rabbitmq.com for documentation of 3.8-specific features.
  • Team RabbitMQ has published an overview of a new feature flag subsystem shipping in RabbitMQ 3.8. The purpose of this subsystem is to simplify rolling upgrades between releases that have incompatible or potentially incompatible changes.
  • RabbitMQ Docker image now ships RabbitMQ 3.7.14 and 3.7.15-beta.1 on latest Erlang and OpenSSL 1.1.1b
  • Java client 5.7.0 (for Java 8+) and 4.11.0 (for Java 6 & 7) have been released with usability improvements and dependency upgrades.
  • Reactor RabbitMQ 1.2.0 GA has been released, with a bug fix, dependency upgrades, and improvements in the publisher confirms support. Reactor RabbitMQ is a reactive API for RabbitMQ based on Reactor and RabbitMQ Java client. Reactor RabbitMQ goal is to enable messages to be published to and consumed from RabbitMQ using functional APIs with non-blocking back-pressure and very low overhead.
  • Debian and RPM packages of several latest Erlang and Elixir releases are now available in Team RabbitMQ's Erlang Bintray repository
 

Community writings and resources

 

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ

Simplifying rolling upgrades between minor versions with feature flags

April 23rd, 2019 by Jean-Sébastien Pédron

In this post we will cover feature flags, a new subsystem in RabbitMQ 3.8. Feature flags will allow a rolling cluster upgrade to the next minor version, without requiring all nodes to be stopped before upgrading.

Minor Version Upgrades Today: RabbitMQ 3.6.x to 3.7.x

It you had to upgrade a cluster from RabbitMQ 3.6.x to 3.7.x, you probably had to use one of the following solutions:

  • Deploy a new cluster alongside the existing one (this strategy is known as the blue-green deployment), then migrate data & clients to the new cluster
  • Stop all nodes in the existing cluster, upgrade the last node that was stopped first, then continue upgrading all other nodes, one-by-one

Blue-green deployment strategy is low risk but also fairly complex to automate. On the other hand, a cluster-wide shutdown affects availability. Feature flags are meant to provide a 3rd option by making rolling cluster upgrades possible and reasonably easy to automate.

The Feature Flags Subsystem

Feature flags indicate a RabbitMQ node's capabilities to its cluster peers. Previously nodes used versions to assess compatibility with cluster versions. There are many ways in which nodes in a distributed system can become incompatible, including Erlang and dependency versions. Many of those aspects are not reflected in a set of version numbers. Feature flags is a better approach as it can reflect more capabilities of a node, whether it is a particular feature or internal communication protocol revision. In fact, with some message protocols RabbitMQ supports clients have a mechanism for clients to indicate their capabilities. This allows client libraries to evolve and be upgraded independently of RabbitMQ nodes.

For example, RabbitMQ 3.8.0 introduces a new queue type, quorum queues. To implement them, an internal data structure and a database schema were modified. This impacts the communication with other nodes because the data structure is exchanged between nodes, and internal data store schema is replicated to all nodes.

Without the feature flags subsystem, it would be impossible to have a RabbitMQ 3.8.0 node inside a cluster where other nodes are running RabbitMQ 3.7.x. Indeed, the 3.7.x nodes would be unable to understand the data structure or the database schema from 3.8.0 node. The opposite is also true. That's why RabbitMQ today prevents this from happening by comparing versions and by denying clustering when versions are considered incompatible (the policy considers different minor/major versions to be incompatible).

New in RabbitMQ 3.8.0 is the feature flags subsystem: when a single node in a 3.7.x cluster is upgraded to 3.8.0 and restarted, it will not immediately enable the new features or migrate its database schema because the feature flags subsystem told it not to. It could determine this because RabbitMQ 3.7.x supports no feature flags at all, therefore new features or behaviours in RabbitMQ 3.8.0 cannot be used before all nodes in the cluster are upgraded.

After a partial upgrade of a cluster to RabbitMQ 3.8.0, all nodes are acting as 3.7.x nodes with regards to incompatible features, even the 3.8.0 one. In this situation, quorum queues are unavailable. Operator must finish the upgrade by upgrading all nodes. When that is done, the operator can decide to enable the new feature flags provided by RabbitMQ 3.8.0: one of them enables quorum queues. This is done using RabbitMQ CLI tools or management UI and supposed to be performed by deployment automation tools. The idea is that the operator needs to confirm that her cluster doesn't have any remaining RabbitMQ 3.7.x nodes that might rejoin the cluster at a later point.

Once a new feature flag is enabled, it is impossible to add a node that runs an older version to that cluster.

Demo with RabbitMQ 3.8.0

Let's go through a complete upgrade of a RabbitMQ 3.7.x cluster. We will take a look at the feature flags in the process.

We have the following 2-node cluster running RabbitMQ 3.7.12:

We now upgrade node A to RabbitMQ 3.8.0 and restart it. Here is what the management overview page looks like after the node is restarted:

We can see the difference of versions in the list of nodes: their version is displayed just below their node name.

The list of feature flags provided by RabbitMQ 3.8.0 is now available in the management UI on node A:

This page will not exist on node B because it is still running RabbitMQ 3.7.12.

On node A, we see that the quorum_queue feature flag is marked as Unavailable. The reason is that node B (still running RabbitMQ 3.7.12) does not known about quorum_queue feature flag, therefore node A is not allowed to use that new feature flag. This feature flag cannot be enabled until all nodes in the cluster support it.

For instance, we could try to declare a quorum queue on node A, but it is denied:

After node B is upgraded, feature flags are available and they can be enabled. We proceed and enable quorum_queue by clicking the Enable button:

Now, we can declare a quorum queue:

To Learn More

The Feature Flags subsystem documentation describes in greater details how it works and what operators and plugin developers should pay attention to.

Note that feature flags are not a guarantee that a cluster shutdown will never be required for upgrades in the future: the ability to implement a change using a feature flag depends on the nature of the change, and is decided on a case-by-case basis. Correct behaviour of a distributed system may require that all of its components behave in a certain way, and sometimes that means that they have to be upgraded in lockstep.

Please give feature flags a try and let us know what you think on the RabbitMQ mailing list!

This Month in RabbitMQ — April 3, 2019

April 3rd, 2019 by Michael Klishin

RabbitMQ 3.8 is coming! If you haven’t already played with the beta (version 3 is now available), it’s time to start familiarizing yourself with what’s coming. Karl Nilsson and I will present on a webinar in May to walk through what’s new, so please register and attend.

We are also starting to look forward to the next RabbitMQ Summit, once again in London this coming November. The call for talks is open until May 10, so please consider sharing how you are using RabbitMQ or something you have tried and learned and want to share with the community.

Project updates

Community writings and resources

Upcoming Courses and Webinars

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ  

This Month in RabbitMQ — March 7, 2019

March 8th, 2019 by Michael Klishin

Welcome back for another issue of This Month in RabbitMQ. Did you know that RabbitMQ was the seventh highest paying tech skill in 2018? AND, that average pay grew a healthy 5.3% since 2017. It’s no wonder that we keep seeing more folks in the community sharing how they are getting started—or getting better—with RabbitMQ. In that spirit, read on for the latest project updates, community writings, and upcoming trainings!

Project updates

Community writings and resources

 

Training

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ:

This Month in RabbitMQ — Feb 7, 2019

February 7th, 2019 by Michael Klishin

Welcome back for another issue of This Month in RabbitMQ. Hopefully you are finding this new series helpful to keep up with the latest project updates and community topics. As we look across the different articles published throughout the month, it’s clear that it truly has a polyglot community. From Spring and .NET, to Ruby and Node.js, there are active users of RabbitMQ out there writing in many different languages. It’s a polyglot world, and we’re connecting it all together!

Read the rest of this entry »

This Month in RabbitMQ — Jan 8, 2019

January 8th, 2019 by Michael Klishin

Happy New Year! Welcome back for another installment of This Month in RabbitMQ. Between running a webinar and publishing a new page, we made a lot of progress in promoting RabbitMQ “best practices” in December. Watch for more content to help everyone in the Rabbit community know how to run Rabbit smoothly.

There were plenty of other great developments from RabbitMQ engineering, including 1.0 of Reactor RabbitMQ, and great insights shared across the community. Read on!

Read the rest of this entry »

This Month in RabbitMQ, November 2018

December 4th, 2018 by Michael Klishin

Hello RabbitMQ friends! Welcome to the first installment of This Month in RabbitMQ, inspired by the wonderful and industrious Josh Long, who publishes monthly and weekly recaps for the Spring community. Our team was also inspired by the first ever RabbitMQ Summit that we held on November 12 in London. It was awesome to see an assembly of the community and the knowledge shared. Look out for videos from that event in a future issue of This Month in RabbitMQ.

Without further ado, let's take a look at a roundup of what happened in RabbitMQ land last month!

Read the rest of this entry »

RabbitMQ Java Client Metrics with Micrometer and Datadog

April 10th, 2018 by Arnaud Cogoluègnes

In this post we'll cover how the RabbitMQ Java client library gathers runtime metrics and sends them to monitoring systems like JMX and Datadog.

Read the rest of this entry »

New Configuration Format in RabbitMQ 3.7

February 22nd, 2018 by Michael Klishin

In this post we'll cover a new configuration format available in RabbitMQ 3.7.0.

Read the rest of this entry »

Peer Discovery Subsystem in RabbitMQ 3.7

February 12th, 2018 by Michael Klishin

In this blog post we're going to take a closer look at a new subsystem introduced in RabbitMQ 3.7.0.

Read the rest of this entry »