Configuring Dynamic Shovels

Information about dynamic shovels is stored in the RabbitMQ database, along with users, permissions, queues, etc. Every shovel is defined by a corresponding named parameter.

Parameters can be defined using rabbitmqctl, through the management HTTP API, or (with the rabbitmq_shovel_management plugin installed) through the management plugin's web UI.

A simple example

Here we will consume messages from the queue my-queue on the local broker, and push them out to the queue another-queue on a remote broker.

rabbitmqctl rabbitmqctl set_parameter shovel my-shovel \
'{"src-uri": "amqp://", "src-queue": "my-queue", \
"dest-uri": "amqp://remote-server", "dest-queue": "another-queue"}'
rabbitmqctl (Windows) rabbitmqctl set_parameter shovel my-shovel ^
"{""src-uri"":""amqp://"", ""src-queue"": ""my-queue"", ^
""dest-uri"": ""amqp://remote-server"", ""dest-queue"": ""another-queue""}"
HTTP API PUT /api/parameters/shovel/%2f/my-shovel
{"value":{"src-uri":  "amqp://",              "src-queue":  "my-queue",
          "dest-uri": "amqp://remote-server", "dest-queue": "another-queue"}}
Web UI Navigate to Admin > Shovel Management > Add a new shovel. Enter "my-shovel" next to Name, "amqp://" and "my-queue" next to Source, and "amqp://remote-server" and "another-queue" next Destination Expiry. Click Add shovel.

Configuration reference

Source

src-uri

The AMQP URI(s) for the source. Mandatory. See the AMQP URI reference for information on how RabbitMQ treats AMQP URIs in general, and the query parameter reference for the Erlang client's extensions (including those for SSL and SASL) which are available to the shovel.

Note that this field can either be a string, or a list of strings. If more than one string is provided, the shovel will randomly pick one URI from the list. This can be used to connect to a cluster and ensure the link will eventually find another node in the event that one fails. It's probably not a great idea to use multiple URIs that do not point to the same cluster.

src-queue

The queue from which to consume. Either this or src-exchange (but not both) must be set.

If the source queue does not exist on the source broker, it will be declared as a durable queue with no arguments.

src-exchange

The exchange from which to consume. Either this or src-queue (but not both) must be set.

The shovel will declare an exclusive queue and bind it to the named exchange with src-exchange-key before consuming from the queue.

If the source exchange does not exist on the source broker, it will be not declared; the shovel will fail to start.

src-exchange-key

Routing key when using src-exchange

Destination

dest-uri

The AMQP URI(s) for the destination. Mandatory. See src-uri above.

dest-queue

The queue to which messages should be published. Either this or dest-exchange (but not both) may be set. If neither is set then messages are republished with their original exchange and routing key.

If the destination queue does not exist on the source broker, it will be declared as a durable queue with no arguments.

dest-exchange

The exchange to which messages should be published. Either this or dest-queue (but not both) may be set.

If the destination exchange does not exist on the source broker, it will be not declared; the shovel will fail to start.

dest-exchange-key

Routing key when using dest-exchange. If this is not set, the original message's routing key will be used.

Shovel

prefetch-count
The maximum number of unacknowledged messages copied over a shovel at any one time. Default is 1000.
reconnect-delay
The duration (in seconds) to wait before reconnecting to the brokers after being disconnected at either end. Default is 1.
publish-properties
Properties to overwrite when shovelling messages. Setting headers this way is not currently supported. Default is {}.
add-forward-headers
Whether to add x-shovelled headers to the shovelled messages indicating where they have been shovelled from and to. Default is false.
ack-mode

Determines how the shovel should acknowledge messages. If set to on-confirm (the default), messages are acknowledged to the source broker after they have been confirmed by the destination. This handles network errors and broker failures without losing messages, and is the slowest option.

If set to on-publish, messages are acknowledged to the source broker after they have been published at the destination. This handles network errors without losing messages, but may lose messages in the event of broker failures.

If set to no-ack, message acknowledgements are not used. This is the fastest option, but may lose messages in the event of network or broker failures.

delete-after

Determines when (if ever) the shovel should delete itself. This can be useful if the shovel is being treated as more of a move operation - i.e. being used to move messages from one queue to another on an ad hoc basis.

The default is never, meaning the shovel should never delete itself.

If set to queue-length then the shovel will measure the length of the source queue when starting up, and delete itself after it has transfered that many messages.

If set to an integer, then the shovel will transfer that number of messages before deleting itself.