Menu

Archive for the ‘Updates’ Category

This Month in RabbitMQ, February 2020 Recap

Tuesday, March 10th, 2020

This Month in RabbitMQ — February 2020 Recap!

RabbitMQ Summit is coming again! This time, the gathering will be in Berlin on June 9 and the call for proposals (to speak at the event) is open until March 22.

Mark your calendars, brush up on your Deutsch, and buy your tickets for the next chance to immerse yourself in all things RabbitMQ. I’m sure there will be at least a couple of RabbitMQ influencers there, too 🙂

Project updates

Community Writings and Resources

Learn More

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ:

This Month in RabbitMQ, January 2020 Recap

Wednesday, February 12th, 2020

This Month in RabbitMQ, January 2020 Recap

Introducing TGI RabbitMQ! Inspired by TGI Kubernetes, RabbitMQ engineer, Gerhard Lazu has begun a series of tutorial videos. Tune in at the end of each month for the latest release. In January, Gerhard covered upgrading from 3.7 to 3.8. Star and watch the repository for future episode updates.

Also, be sure to check out the dashboards we’ve published to Grafana. These are a great way to get started with the new Prometheus and Grafana support in 3.8.

Project Updates

Community Writings and Resources

Ready to learn more?

This Month in RabbitMQ, December 2019 Recap

Thursday, January 9th, 2020

This Month in RabbitMQ — December Recap!

Happy new year! 3.8.x has been available for over three months now and we’re seeing a lot of great uptake. This is good news, since the upgrade process is even easier with the addition of feature flags. Keep up the upgrading!

Over at the CloudAMQP blog, you’ll now find videos transcripts of all the RabbitMQ Summit talks. Those are useful if you didn’t make it to the event and want to know what’s in the talk before watching the full 30 minute replay.

Take a look at Observe and Understand RabbitMQ, for example.

We also published a new case study about LAIKA, the animation company that brought you Coraline, The BoxTrolls, and Missing Link. If you are interested in having your use case for RabbitMQ profiled on rabbitmq.com, drop a note in the mailing list or email info@rabbitmq.com.

Project updates

Community Writings and Resources

Training

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ

This Month in RabbitMQ, November 2019 Recap

Saturday, December 7th, 2019

Last month was a big one for the RabbitMQ community because RabbitMQ Summit happened in London! If you missed the event, or if you were at the event, but missed a session in the other track, all the recordings are now available. Also, be sure to check out our overview blog for an easy-to-digest summary of what’s new in RabbitMQ 3.8.

More new 3.8 features and lessons learned will be covered in an upcoming webinar, by RabbitMQ core team member Gerhard Lazu. Tune in on December 12th!

Project Updates

Community Writings and Resources

Ready to learn more?

Check out these upcoming opportunities to learn more about RabbitMQ

This Month in RabbitMQ: November 2019

Wednesday, November 13th, 2019

This Month (and the month before) in RabbitMQ — October and September recap!

We’re a little behind this month! At the beginning of October, we shipped RabbitMQ 3.8. That’s right, folks, RabbitMQ 3.8 is finally out!

Headline features include:

You’ll find some early reviews from folks in the community who have been kicking the tires in the community updates section below. Make sure you are all over the upgrades best practices to avoid potential hazards of upgrading to RabbitMQ 3.8.

Oh, and there were some other rather meaningful ecosystem announcements out there:

SpringOne Platform 2019 talks that highlighted RabbitMQ:

Project updates

Several updates to 3.7.x with bug fixes:

Community Writings and Resources

Webinars and Training

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ

RabbitMQ 3.8 Release Overview

Monday, November 11th, 2019

RabbitMQ 3.8 has just been released and has some major new features which focus on reliability, operations, and observability.

You can find the new 3.8 release on the GitHub releases page which includes information about what is included in the release as well as various installation assets. See our upgrade guide for more information about upgrading to 3.8.0.

Our team dedicates this release to Joe Armstrong, the creator of Erlang. Joe’s work in the fields of concurrent and distributed systems benefits RabbitMQ to this day. Equally importantly, Joe was a rare example of a brilliant engineer who was also very humble and kind.

Let’s take a quick look at the new features in this release.

Quorum Queues

For years, RabbitMQ has offered mirrored queues, also known as HA queues, as a solution for both high availability and data safety. Messages are replicated from a queue master to one or more mirrors so that in the event of the loss of a broker, a mirror can be promoted to master and the queue continues to be available without loss of confirmed messages.

Quorum queues are the next generation of replicated queue and offer both better performance and solve some of the pain points of mirrored queues. Quorum queues use the well established Raft protocol which has now been implemented in countless data systems as a means of achieving reliable and fault tolerant state replication.

Shows a quorum queue consisting of one leader and two followers

One of the main pain points around mirrored queues was blocking synchronization coupled with throwing away data on leaving and rejoining a cluster. This made applying OS patches difficult if queues were large in size as the administrator was forced to choose between lower redundancy or a period of unavailability. Quorum queues completely avoid this issue by not throwing away data and making replication to a single node non blocking. Quorum queues also avoid split-brain scenarios that could provoke message loss and always favour consistency over availability.

From now on we will be referring to classic and quorum queues.

Read more about quorum queues in documentation guides.

Feature Flags

Prior to the new feature flag sub-system, upgrades to RabbitMQ required cluster-wide shutdown. Feature flags allow for rolling upgrades of a cluster enabling continued availability.

As Jean-Sébastien Pédron described in  this blog: “New in RabbitMQ 3.8.0 is the feature flags subsystem: when a single node in a 3.7.x cluster is upgraded to 3.8.0 and restarted, it will not immediately enable the new features or migrate its database schema because the feature flags subsystem told it not to. It could determine this because RabbitMQ 3.7.x supports no feature flags at all, therefore new features or behaviours in RabbitMQ 3.8.0 cannot be used before all nodes in the cluster are upgraded.”

Multi-step process of upgrading and enabling feature flags

Read more about feature flags in documentation guides.

Prometheus and Grafana Monitoring Support

Many systems come with their own custom monitoring visualization solution, the Management Plugin has has been this solution in RabbitMQ for years. The new paradigm is for applications and infrastructure to expose metrics to external observability platforms and delegate the storing, indexing and alerting to those specialized tools. Both Prometheus and Grafana have become an industry standard in the systems observability space and provide powerful visualization and exploratory data analysis capabilities.

RabbitMQ 3.8 comes with new support for exposing its metrics via a Prometheus endpoint. Additionally, many more metrics are now available, vastly improving the overall observability of RabbitMQ. Visualizing these metrics is now as simple as importing pre-built dashboards into Grafana.

The RabitMQ overview Grafana dashboard

Prometheus and Grafana support has a dedicated documentation guide.

OAuth 2.0 Support

RabbitMQ 3.8 allows clients to use JWT access tokens for authentication and authorization. Clients obtain an access token from an OAuth2.0 provider, through any grant type they wish, and use that token to gain access to RabbitMQ. OAuth 2.0 tokens use scopes to communicate what set of permissions a particular client has been granted and RabbitMQ permissions are mapped onto these scopes.

Read more about OAuth2.0 support in the docs.

Additional CLI Tools

You can perform various levels of health checks with the rabbitmq-diagostics CLI tool. The checks range from basic pings to checking queues and vhosts are running to in-depth runtime information.

We have a new CLI tool, rabbitmq-queues, which gives us the ability to modify quorum queue memberships but also gives us new master/leader rebalancing functionality for both quorum and mirrored queues.

One of the pain points of performing a rolling upgrade to the servers of a RabbitMQ cluster was that queue masters would end up concentrated on one or two servers. The new rebalance command will automatically rebalance masters across the cluster. 

rabbitmq-queues has a man page.

Single Active Consumer (SAC)

SAC is also the next generation of an existing feature - exclusive consumers. The objective of exclusive consumers is to ensure that only a single consumer can consume a given queue at a time. The consumer uses the “exclusive” flag when registering itself, and the registration only succeeds if no other consumer is already registered.

SAC improves on this by making exclusivity a feature of the queue itself and making the process transparent to clients. If a second consumer registers itself, the registration succeeds and the consumer sits idle ready to become active if the currently active consumer shuts down or crashes. This gives us an automatic active-backup consumer strategy for when we want only a single consumer, but a secondary to take over quickly in the event the active goes away.

Shows two consumers on a queue where only one is active

Read more about Single Active Consumer in the Consumers guide.

And…

  • Messages can now be deadlettered from the tail of a classic queue with the new queue overflow configuration reject-publish-dlx.
  • High queue creation/deletion rates (queue churn) are now less costly.
  • Maximum message size is now configurable.
  • Quorum queues come with a new poison message feature that allows you to configure messages to be dropped after a given number of redeliveries by setting the delivery-limit policy.
  • RabbitMQ for Kubernetes is coming. Sign up to the beta.
 

If you’re more of a classroom learner, I recommend watching the webinar, “What's new in RabbitMQ 3.8?

Please give 3.8 a try and let us know what you think on the RabbitMQ mailing list!

This Month in RabbitMQ — September 2019

Monday, September 9th, 2019

Welcome back for another edition of This Month in RabbitMQ! Exciting news is that the first release candidate for RabbitMQ 3.8 is now available!

Be sure to catch up on what is new in 3.8 by reading the release notes and watching this webinar replay.

We are starting to countdown until RabbitMQ Summit in London on November 4. The RabbitMQ team is looking forward to sharing updates on the project, but we’re also looking forward to hearing from end-users like Bloomberg, WeWork, Softonic, and Zalando. Be sure to register and snag a spot in one of the training add-on courses.

Project updates

Community writings and resources

Events and Training

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ

This Month in RabbitMQ — August 2019

Monday, August 12th, 2019

Welcome back for another edition of This Month in RabbitMQ! Some big news last month was Pivotal announced a forthcoming alpha of Pivotal RabbitMQ for Kubernetes.

You can inquire about the alpha here. As part of that announcement, RabbitMQ was mentioned in coverage in Business Insider, Container Journal, SiliconANGLE, Storage Review, The New Stack, and ZDNet. Pretty cool!

Before we move on to the update from the core team and our wonderful community, a reminder that prices for RabbitMQ Summit go up on August 22, so get your tickets now!

You can add on RabbitMQ training to your ticket—basic and advanced courses are available. Great talks planned from Bloomberg, WeWork, Softonic, the Erlang Solutions and CloudAMQP teams, as well as the core RabbitMQ engineers, of course.

Project updates

Community writings and resources

We also came across this article in the International Journal of Research Studies in Computer Science and Engineering (IJRSCSE) on Distributing Messages Using Rabbitmq with Advanced Message Exchanges

Events and Training Courses

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ

This Month in RabbitMQ — July 2019

Tuesday, July 9th, 2019

Welcome back for another edition of This Month in RabbitMQ! In June, we saw the RabbitMQ Summit agenda start to go live, featuring some great returning speakers as well as new faces. There are also a couple of training sessions offered to add onto your ticket. It’s a great way to immerse yourself in all things RabbitMQ for a couple of days. Registration is open, so book your tickets now before the prices go up in August!

Project updates

  • RabbitMQ 3.7.16 has been released with bug fixes, usability improvements and new rabbitmq-diagnostics commands
  • PerfTest 2.8.1 was released with a couple of bug fixes
  • Reactor RabbitMQ 1.3.0.M1 was released as part of the Reactor Dysprosium-M2 release train. More goodies to come in the next few weeks!
  • March Hare 4.0 was released, now based on the 5.7.x series of the RabbitMQ Java client

Community writings and resources

June 5: Vermaden (@vermaden) wrote about setting up RabbitMQ cluster on FreeBSD Jails

June 5: Josh Smeaton (@jarshwah) of Kogan published about monitoring Celery queue length with RabbitMQ

June 7: Emre Tiryaki (@emrtryki) from Hepsiburada published about event ordering with RabbitMQ using the consistent hash exchange (in Turkish)

June 8: Cleison Ferreira Melo (Cleison Ferreira Melo) wrote another installment of his series on building a microservices application, focused on the RabbitMQ container and connection

June 8: Jose Alonso Romero Matias published a four-part video series (in Spanish) showing how to create a messaging project that emulates the sending of invoices through a service, using RabbitMQ: part 1, part 2 on dependency injection with RabbitMQ, part 3 on creating and testing the invoice handler, part 4

June 9: Gilles Robert (@ask4gilles) released v2.0.3 of @opentracing Spring RabbitMQ with a bunch of new instrumented methods on AmqpTemplate and documentation improvements

June 11: Marco Behler (@MarcoBehler) published a video on How to Consume RabbitMQ Messages From Queues With Java

June 13: Maksim Martianov wrote about Kubernetes worker autoscaling based on RabbitMQ queue depth

June 14: Bartha Bela Tibor published about RabbitMQ in Docker with Alpine Linux

June 17: Rafael Capuano published (in Portuguese) a three-part series on the external configuration store pattern, using RabbitMQ for  configuration change propagation: part 1 on contextualizing, part 2 on creating the API, and part 3 on creating the client

June 18: Ram N. published a video and resource links on how to send and receive product objects to or from a queue

June 19: IBM published a tutorial on invoking serverless functions through a message broker

June 23: Karol Galanciak (@Azdaroth) published the third part in a series on Messages on Rails, this one focused on RabbitMQ

June 25: Dhananjay Singh wrote about Spring Cloud Stream with RabbitMQ: Message-Driven Microservices

June 27: Ranga Karanam () published on DZone about Asynchronous Communication With Queues and Microservices: A Perfect Combination?

June 28: Davide Guida (@DavideGuida82) published the first in a series on using message queues in .NET Core (in Italian)

June 29: Teerapong Singthong (@iamgoangle) wrote about Go Messaging System with RabbitMQ and RabbitMQ client for Go (in Thai)

June 30: Md. Al-Amin published about Solving RabbitMQ High CPU/Memory Usages Problem With Celery

Ready to learn more?

Check out these upcoming opportunities to learn more about RabbitMQ:

This Month in RabbitMQ — June 2019

Thursday, June 6th, 2019

Welcome back for another edition of This Month in RabbitMQ! Keep sharing your war stories and lessons learned out there, and tweet them with #rabbitMQ to get them on our radar for inclusion in these write-ups.

As we march towards RabbitMQ 3.8 going GA, be sure to catch the replay of the webinar we did last month on what’s new in RabbitMQ 3.8. Jack Vanlightly also did some coverage on the Single Active Consumer feature in 3.8, adding to his earlier coverage on Quorum Queues.

Project updates

  • RabbitMQ 3.7.15 was released, includes initial support for Erlang 22
  • Erlang 22 is now GA with a new inter-node communication implementation and initial TLSv1.3 server support
  • Erlang 22.0.2 and 21.3.8.3 were released, addressing ERL-934 and ERL-938, an issue that affected RabbitMQ environments that used TLS and had a high data ingestion volume.
  • RabbitMQ Docker image has transitioned to use Erlang 22
  • PerfTest 2.8.0 has been released with lots of goodies: new options to vary message size and publishing rate, optional polling consumers (instead of asynchronous consumers by default), optionally nack messages instead of acking them, and dependency upgrades
  • Java client 5.7.1 (for Java 8+) and 4.11.1 (for Java 6 & 7) have been released with a bug fix
 

Community writings and resources

May 1: Denis Orehovsky (@apirobotme) published about Distributed systems with RabbitMQ.

May 1: Sam Bently (@sambentley00) published about Building An External Rabbitmq Service For VCloud Director, with particular attention to SSL requirements.

May 1: Nerengen Babu published Integrating RabbitMQ with SpringBoot Application (Receiver Part).

May 5: Chetan Khatri (@khatri_chetan)wrote about How to Setup Airflow Multi-Node Cluster with Celery & RabbitMQ

May 6: mastanggt wrote about migrating RabbitMQ to/within Kubernetes (in Russian).

May 8: Nikita wrote about RabbitMQ Fetching Remote Data, particularly in Rails.

May 10: Jacques Roussel published about Using the Ansible Operator-sdk To Build A RabbitMQ Operator for Kubernetes.

May 10: Aamer Mohammed compares several different messaging technologies, including RabbitMQ, in the context of Asynchronous communication in Microservices.

May 13: Jind?ich Hrabal (@Backglite) published on Dead Letter Queue Reprocessing with Spring Integration and RabbitMQ.

May 14: Prashant Vats published on Kubernetes pod autoscaling in response to the change in the RabbitMQ queue.

May 14: Francisco Cardoso published about what’s different in AMQP 1.0 compared to AMQP 0-9-1 (in Portugese).

May 17: Lakmini Wathsala published How to integrate WSO2 EI with RabbitMQ without passing credentials from address URI.

May 20: Lukasz Lenart (@lukaszlenart?) published on How to configure RabbitMQ via definitions.

May 24: Jeroen Jacobs (@jeroen1205) wrote about troubleshooting sync issues with classic mirrored queues.

May 24: Lovisa Johansson (@lillajja) answered the question “What is the message size limit in RabbitMQ?”

May 24: Ryan Gunn (@Icidis) published about Blazor, RabbitMQ and MQTT using Paho with JSInterop.

May 27: Marcela Sisiliani (@ma_sisiliani) wrote an introduction to the world of queues (in Portugese).

May 28: Hervé Beraud (@4383hberaud) published about How To Play With RabbitMQ And Python Quickly.

May 28: Jack Vanlightly (@vanlightly) wrote about Maintaining Long-Lived Connections with AMQProxy and about Publishing Throughput - Asynchronous vs Synchronous.

May 28: Tomas Henriquez (@Hassek85) published about how he sets up RabbitMQ clusters to scale, noting using the consistent hash plugin and mirrored queues .

May 29: Mohamed Elhachmi published about Efficient design for daemons tasks.

May 29: Denis Setianto published How to Install RabbitMQ Server on Ubuntu 18.04 & 16.04 LTS.

 

Ready to learn more? Check out these upcoming opportunities to learn more about RabbitMQ

6 June 2019, online: Boosting Microservice Performance with Messaging and Spring.

4 November 2019: London: RabbitMQ Summit 2019.

On-demand, online: LearnFly: Learn RabbitMQ Asynchronous Messaging with Java and Spring.

On-demand, online: Udemy: RabbitMQ: Messaging with Java, Spring Boot And Spring MVC.