Parameters and Policies

Introduction

While much of the configuration for RabbitMQ lives in the configuration file, some things do not mesh well with the use of a configuration file:

  • If they need to be the same across all nodes in a cluster
  • If they are likely to change at run time

RabbitMQ calls these items parameters. Parameters can be set by invoking rabbitmqctl or through the management plugin's HTTP API. There are 2 kinds of parameters: vhost-scoped parameters and global parameters. Vhost-scoped parameters are tied to a virtual host and consist of a component name, a name and a value. Global parameters are not tied to a particular virtual and they consist of a name and value.

One special case of parameters usage is policies, which are used for specifying optional arguments for groups of queues and exchanges, as well as plugins such as Federation and Shovel. Policies are vhost-scoped.

Global and Per-vhost Parameters

As stated above, there are vhost-scoped parameters and global parameters. An example of vhost-scoped parameter is a federation upstream: it targets a component (federation-upstream), it has a name that identifies it, it's tied to a virtual host (federation links will target some resources of this virtual host), and its value defines connection parameters to an upstream broker. Vhost-scoped parameters can be set, cleared and listed:

rabbitmqctl rabbitmqctl set_parameter {-p vhost} component_name name value
rabbitmqctl clear_parameter {-p vhost} component_name name
rabbitmqctl list_parameters {-p vhost}
HTTP API PUT /api/parameters/component_name/vhost/name
DELETE /api/parameters/component_name/vhost/name
GET /api/parameters

Global parameters is the other kind of parameters. An example of a global parameter is the name of the cluster. Global parameters can be set, cleared and listed:

rabbitmqctl rabbitmqctl set_global_parameter name value
rabbitmqctl clear_global_parameter name
rabbitmqctl list_global_parameters
HTTP API PUT /api/global-parameters/name
DELETE /api/global-parameters/name
GET /api/global-parameters

Since a parameter value is a JSON document, you will usually need to quote it when creating one on the command line with rabbitmqctl. On Unix it is usually easiest to quote the whole document with single quotes, and use double quotes within it. On Windows you will have to escape every double quote. We give examples for both Unix and Windows for this reason.

Parameters reside in the database used by RabbitMQ for definitions of virtual hosts, exchanges, queues, bindings, users and permissions. Parameters are exported along with other object definitions by the management plugin's export feature.

Vhost-scoped parameters are used by the federation and shovel plugins. Global parameters are used by the MQTT plugin.

Policies

Why Policies Exist

Before we explain what policies are and how to use them it would be helpful to explain why they were introduced to RabbitMQ.

In addition to mandatory properties (e.g. durable or exclusive), queues and exchanges in RabbitMQ have optional parameters (arguments), sometimes referred to as x-arguments. Those are provided by clients when they declare queues (exchanges) and control various optional features, such as mirroring or TTL.

Client-controlled properties in some of the protocols RabbitMQ supports generally work well but they can be inflexible: updating TTL values or mirroring parameters that way required application changes, redeployment and queue re-declaration (which involves deletion). In addition, there is no way to control the extra arguments for groups of queues and exchanges. Policies were introduced to address the above pain points.

A policy matches one or more queues by name (using a regular expression pattern) and appends its definition (a map of optional arguments) to the x-arguments of the matching queues. In other words, it is possible to configure x-arguments for multiple queues at once with a policy, and update them all at once by updating policy definition.

In modern versions of RabbitMQ the set of features which can be controlled by policy is not the same as the set of features which can be controlled by client-provided arguments.

How Policies Work

Key policy attributes are

  • name: it can be anything but ASCII-based names without spaces are recommended
  • pattern: a regular expression that matches one or more queue (exchange) names. Any regular expression can be used.
  • definition: a set of key/value pairs (think a JSON document) that will be injected into the map of optional arguments of the matching queues and exchanges
  • priority: see below

Policies automatically match against exchanges and queues, and help determine how they behave. Each exchange or queue will have at most one policy matching (see Combining Policy Definitions below), and each policy then injects a set of key-value pairs (policy definition) on to the matching queues (exchanges).

Policies can match only queues, only exchanges, or both. This is controlled using the apply-to flag when a policy is created.

Policies can change at any time. When a policy definition is updated, its effect on matching exchanges and queues will be reapplied. Usually it happens instantaneously but for very busy queues can take a bit of time (say, a few seconds).

Policies are matched and applied every time an exchange or queue is created, not just when the policy is created.

Policies can be used to configure the federation plugin, mirrored queues, alternate exchanges, dead lettering, per-queue TTLs, and maximum queue length.

An example of defining a policy looks like:

rabbitmqctl
rabbitmqctl set_policy federate-me "^amq\." '{"federation-upstream-set":"all"}' --priority 1 --apply-to exchanges
rabbitmqctl (Windows)
rabbitmqctl set_policy federate-me "^amq\." "{""federation-upstream-set"":""all""}" --priority 1 --apply-to exchanges
HTTP API
PUT /api/policies/%2f/federate-me
                    {"pattern": "^amq\.",
                     "definition": {"federation-upstream-set":"all"},
                     "priority": 1,
                    "apply-to": "exchanges"}
Web UI
  • Navigate to Admin > Policies > Add / update a policy.
  • Enter "federate-me" next to Name, "^amq\." next to Pattern, and select "Exchanges" next to Apply to.
  • Enter "federation-upstream-set" = "all" in the first line next to Policy.
  • Click Add policy.

This matches the value "all" with the key "federation-upstream-set" for all exchanges with names beginning with "amq.", in the virtual host "/".

The "pattern" argument is a regular expression used to match exchange or queue names.

In the event that more than one policy can match a given exchange or queue, the policy with the greatest priority applies.

The "apply-to" argument can be "exchanges", "queues" or "all". The "apply-to" and "priority" settings are optional, in which case the defaults are "all" and "0" respectively.

Combining policy definitions

In some cases we might want to apply more than one policy definition to a resource. For example we might need a queue to be federated and mirrored. At most one policy will apply to a resource at any given time, but we can apply multiple definitions in that policy.

A federation policy definition would require an upstream set to be specified, so we would need the federation-upstream-set key in our definition. On the other hand to define some queues as mirrored, we would need the ha-mode key to be defined as well for the policy. Since the policy definition is just a JSON object, we can have both keys in the same policy definition.

Here's an example:

rabbitmqctl
rabbitmqctl set_policy ha-fed "^hf\." '{"federation-upstream-set":"all","ha-mode":"all"}' \
--priority 1 --apply-to queues
rabbitmqctl (Windows)
rabbitmqctl set_policy ha-fed "^hf\." "{""federation-upstream-set"":""all"", ""ha-mode"":""all""}" ^
--priority 1 --apply-to queues
HTTP API
PUT /api/policies/%2f/ha-fed
{"pattern": "^hf\.",
 "definition": {"federation-upstream-set":"all", "ha-mode": "all"},
 "priority": 1,
 "apply-to": "queues"}
Web UI
  • Navigate to Admin > Policies > Add / update a policy.
  • Enter "ha-fed" next to Name, "^hf\." next to Pattern, and select "Queues" next to Apply to.
  • Enter "federation-upstream-set" = "all" in the first line next to Policy.
  • Enter "ha-mode" = "all" on the next line.
  • Click Add policy.

By doing that all the queues matched by the pattern "^hf\." will have the "federation-upstream-set" and the "ha-mode" definitions applied to them.