Parameters and Policies


While much of the configuration for RabbitMQ lives in the configuration file, some things do not mesh well with the use of a configuration file:

  • If they need to be the same across all nodes in a cluster
  • If they are likely to change at run time

RabbitMQ calls these items parameters. Parameters can be set by invoking rabbitmqctl or through the management plugin's HTTP API. There are 2 kinds of parameters: vhost-scoped parameters and global parameters. Vhost-scoped parameters are tied to a virtual host and consist of a component name, a name and a value. Global parameters are not tied to a particular virtual and they consist of a name and value.

One special case of parameters usage is policies, which are used for specifying optional arguments for groups of queues and exchanges, as well as plugins such as Federation and Shovel. Policies are vhost-scoped.

Global and Per-vhost Parameters

As stated above, there are vhost-scoped parameters and global parameters. An example of vhost-scoped parameter is a federation upstream: it targets a component (federation-upstream), it has a name that identifies it, it's tied to a virtual host (federation links will target some resources of this virtual host), and its value defines connection parameters to an upstream broker. Vhost-scoped parameters can be set, cleared and listed:

rabbitmqctl rabbitmqctl set_parameter {-p vhost} component_name name value
rabbitmqctl clear_parameter {-p vhost} component_name name
rabbitmqctl list_parameters {-p vhost}
HTTP API PUT /api/parameters/component_name/vhost/name
DELETE /api/parameters/component_name/vhost/name
GET /api/parameters

Global parameters is the other kind of parameters. An example of a global parameter is the name of the cluster. Global parameters can be set, cleared and listed:

rabbitmqctl rabbitmqctl set_global_parameter name value
rabbitmqctl clear_global_parameter name
rabbitmqctl list_global_parameters
HTTP API PUT /api/global-parameters/name
DELETE /api/global-parameters/name
GET /api/global-parameters

Since a parameter value is a JSON document, you will usually need to quote it when creating one on the command line with rabbitmqctl. On Unix it is usually easiest to quote the whole document with single quotes, and use double quotes within it. On Windows you will have to escape every double quote. We give examples for both Unix and Windows for this reason.

Parameters reside in the database used by RabbitMQ for definitions of virtual hosts, exchanges, queues, bindings, users and permissions. Parameters are exported along with other object definitions by the management plugin's export feature.

Vhost-scoped parameters are used by the federation and shovel plugins. Global parameters are used by the MQTT plugin.


Why Policies Exist

Before we explain what policies are and how to use them it would be helpful to explain why they were introduced to RabbitMQ.

In addition to mandatory properties (e.g. durable or exclusive), queues and exchanges in RabbitMQ have optional parameters (arguments), sometimes referred to as x-arguments. Those are provided by clients when they declare queues (exchanges) and control various optional features, such as queue length limit or TTL.

Client-controlled properties in some of the protocols RabbitMQ supports generally work well but they can be inflexible: updating TTL values or mirroring parameters that way required application changes, redeployment and queue re-declaration (which involves deletion). In addition, there is no way to control the extra arguments for groups of queues and exchanges. Policies were introduced to address the above pain points.

A policy matches one or more queues by name (using a regular expression pattern) and appends its definition (a map of optional arguments) to the x-arguments of the matching queues. In other words, it is possible to configure x-arguments for multiple queues at once with a policy, and update them all at once by updating policy definition.

In modern versions of RabbitMQ the set of features which can be controlled by policy is not the same as the set of features which can be controlled by client-provided arguments.

How Policies Work

Key policy attributes are

  • name: it can be anything but ASCII-based names without spaces are recommended
  • pattern: a regular expression that matches one or more queue (exchange) names. Any regular expression can be used.
  • definition: a set of key/value pairs (think a JSON document) that will be injected into the map of optional arguments of the matching queues and exchanges
  • priority: see below

Policies automatically match against exchanges and queues, and help determine how they behave. Each exchange or queue will have at most one policy matching (see Combining Policy Definitions below), and each policy then injects a set of key-value pairs (policy definition) on to the matching queues (exchanges).

Policies can match only queues, only exchanges, or both. This is controlled using the apply-to flag when a policy is created.

Policies can change at any time. When a policy definition is updated, its effect on matching exchanges and queues will be reapplied. Usually it happens instantaneously but for very busy queues can take a bit of time (say, a few seconds).

Policies are matched and applied every time an exchange or queue is created, not just when the policy is created.

Policies can be used to configure the federation plugin, mirrored queues, alternate exchanges, dead lettering, per-queue TTLs, and maximum queue length.

An example of defining a policy looks like:

rabbitmqctl set_policy federate-me "^amq\." '{"federation-upstream-set":"all"}' --priority 1 --apply-to exchanges
rabbitmqctl (Windows)
rabbitmqctl.bat set_policy federate-me "^amq\." "{""federation-upstream-set"":""all""}" --priority 1 --apply-to exchanges
PUT /api/policies/%2f/federate-me
                    {"pattern": "^amq\.",
                     "definition": {"federation-upstream-set":"all"},
                     "priority": 1,
                    "apply-to": "exchanges"}
Web UI
  • Navigate to Admin > Policies > Add / update a policy.
  • Enter "federate-me" next to Name, "^amq\." next to Pattern, and select "Exchanges" next to Apply to.
  • Enter "federation-upstream-set" = "all" in the first line next to Policy.
  • Click Add policy.

This matches the value "all" with the key "federation-upstream-set" for all exchanges with names beginning with "amq.", in the virtual host "/".

The "pattern" argument is a regular expression used to match exchange or queue names.

In the event that more than one policy can match a given exchange or queue, the policy with the greatest priority applies.

The "apply-to" argument can be "exchanges", "queues" or "all". The "apply-to" and "priority" settings are optional, in which case the defaults are "all" and "0" respectively.

Combining policy definitions

In some cases we might want to apply more than one policy definition to a resource. For example we might need a queue to be federated and mirrored. At most one policy will apply to a resource at any given time, but we can apply multiple definitions in that policy.

A federation policy definition would require an upstream set to be specified, so we would need the federation-upstream-set key in our definition. On the other hand to define some queues as mirrored, we would need the ha-mode key to be defined as well for the policy. Since the policy definition is just a JSON object, we can have both keys in the same policy definition.

Here's an example:

rabbitmqctl set_policy ha-fed "^hf\." '{"federation-upstream-set":"all","ha-mode":"all"}' \
--priority 1 --apply-to queues
rabbitmqctl (Windows)
rabbitmqctl set_policy ha-fed "^hf\." "{""federation-upstream-set"":""all"", ""ha-mode"":""all""}" ^
--priority 1 --apply-to queues
PUT /api/policies/%2f/ha-fed
{"pattern": "^hf\.",
 "definition": {"federation-upstream-set":"all", "ha-mode": "all"},
 "priority": 1,
 "apply-to": "queues"}
Web UI
  • Navigate to Admin > Policies > Add / update a policy.
  • Enter "ha-fed" next to Name, "^hf\." next to Pattern, and select "Queues" next to Apply to.
  • Enter "federation-upstream-set" = "all" in the first line next to Policy.
  • Enter "ha-mode" = "all" on the next line.
  • Click Add policy.

By doing that all the queues matched by the pattern "^hf\." will have the "federation-upstream-set" and the "ha-mode" definitions applied to them.

Operator Policies

Difference From Regular Policies

Sometimes it is necessary for the operator to enforce certain policies. For example, it may be desirable to force queue TTL but still let other users manage policies. Operator policies allow for that.

Operator policies are much like regular ones but their definitions are used differently. They are merged with regular policy definitions before the result is applied to matching queues.

Because operator policies can unexpectedly change queue attributes and, in turn, application assumptions and semantics, they are limited only to a few arguments:

  • expires
  • message-ttl
  • max-length
  • max-length-bytes
The arguments above are all numeric. Thie reason for that is explained in the following section.

Conflict Resolution with Regular Policies

An operator policy and a regular one can contain the same keys in their definitions. When it happens, the smaller value is chosen as effective. For example, if a matching operator policy definition sets max-length to 50 and a matching regular policy definition uses the value of 100, the value of 50 will be used. If, however, regular policy's value was 20, it would be used. Operator policies, therefore, don't just overwrite regular policy values. They enforce limits but try to not override user-provided policies where possible.

Defining Operator Policies

Operator policies defined in a way very similar to regular (user) policies. When rabbitmqctl is used, the command name is set_operator_policy instead of set_policy. In the HTTP API, /api/policies/ in request path becomes /api/operator-policies/:

rabbitmqctl set_operator_policy transient-queue-ttl "^amq\." '{"expires":1800000}' --priority 1 --apply-to queues
rabbitmqctl (Windows)
rabbitmqctl.bat set_operator_policy transient-queue-ttl "^amq\." "{""expires"": 1800000}" --priority 1 --apply-to queues
PUT /api/operator-policies/%2f/transient-queue-ttl
                        {"pattern": "^amq\.",
                         "definition": {"expires": 1800000},
                         "priority": 1,
                         "apply-to": "queues"}
Web UI
  • Navigate to Admin > Policies > Add / update an operator policy.
  • Enter "transient-queue-ttl" next to Name, "^amq\." next to Pattern, and select "Queues" next to Apply to.
  • Enter "expires" = 1800000 in the first line next to Policy.
  • Click Add policy.