Securing Cluster (Inter-node) and CLI Tool Communication with TLS (SSL)

Sometimes is desirable to make the Erlang nodes talk to each other using TLS (SSL), and thus make the whole RabbitMQ cluster communication via TLS. To achieve that we need to make the Erlang distribution mechanism to use TLS. In this document we are going to review the steps to make this possible.


First we need to create the TLS certificate that's going to be used by the Erlang distribution mechanism. We assume you have done that already, otherwise follow the main TLS guide. Once we have our certificates ready we need to concatenate the server certificate and key into one file, for example, assuming we have the files server_certificate.pem and server_key.pem we can do the following:

cat server_certificate.pem server_key.pem > rabbit.pem

Then we have to tell Erlang where to find the ssl library during startup. We can create a variable like this:

# NOTE: these commands ensure that ERL_SSL_PATH is the first line in
# /etc/rabbitmq/rabbitmq-env.conf and will preserve the existing
# contents of that file if it already exists

erl -noinput -eval 'io:format("ERL_SSL_PATH=~p~n",[code:lib_dir(ssl, ebin)]),halt().' > /tmp/ssl-path.txt
cat /tmp/ssl-path.txt /etc/rabbitmq/rabbitmq-env.conf > /tmp/new-rabbitmq-env.conf
mv -f /tmp/new-rabbitmq-env.conf /etc/rabbitmq/rabbitmq-env.conf

First we find where Erlang has the ssl library, and then the variable ERL_SSL_PATH is set with the result from that command.

By using the previous information now is time to craft the SERVER_ADDITIONAL_ERL_ARGS environment variable so RabbitMQ is able to start Erlang using TLS for distribution. We do that by setting the proto_dist argument to inet_tls and then telling Erlang what certificate to use (in our case that's the rabbit.pem file we just created). Finally we set secure renegotiation to true. Here is the line that should be added to /etc/rabbitmq/rabbitmq-env.conf following the ERL_SSL_PATH line that the above commands added. Please note that the double quotes must be present:

-proto_dist inet_tls \
-ssl_dist_opt server_certfile /path/to/rabbit.pem \
-ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

Once our initial ("seed") node has inter-node connection configured with TLS, CLI tools such as rabbitmqctl also must use TLS to talk to the node. This means we have to do what we just did for SERVER_ADDITIONAL_ERL_ARGS but this time for the environment variable CTL_ERL_ARGS. Here is the complete /etc/rabbitmq/rabbitmq-env.conf file:

# NOTE: the following path is **system dependent**

-proto_dist inet_tls \
-ssl_dist_opt server_certfile /path/to/rabbit.pem \
-ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

-proto_dist inet_tls \
-ssl_dist_opt server_certfile /path/to/rabbit.pem \
-ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true"

Now that we have this in place, it's just a matter of starting RabbitMQ as we usually do to get the Erlang distribution to use TLS for internode communication. Just like CLI tools, all other nodes in the cluster that want to join our initial node must have inter-node connection TLS configured using SERVER_ADDITIONAL_ERL_ARGS in their /etc/rabbitmq/rabbitmq-env.conf file.

All nodes and CLI tools must use certificate/key pairs and TLS settings that allow inter-node TCP connections perform TLS handshake and peer verification successfully. For example, certificate/key pairs used by other nodes and CLI tools must be signed by the same CA as the initial node or a different CA that is trusted on the initial node's machine. This is no different from how client and plugin TLS connections work.

It is possible to reuse a single certificate/key pair for all nodes and CLI tools if it uses a wildcard Common Name, e.g. *.rabbitmq.example.local and all host names RabbitMQ nodes plus CLI tools are started on match the wildcard pattern.


If you are using the OSX Standalone Release, then you need to add some extra arguments in order to run the erl command to find the path of Erlang's TLS library. Assuming you are inside the folder where you installed the standalone release, the commands will look like these:

echo erts-6.1/bin/erl -boot releases/3.4.3/start_clean \
-eval 'io:format("~p", [code:lib_dir(ssl, ebin)]),halt().' -noshell
export ERL_SSL_PATH=/path/to/erl/lib/ssl-5.3.5/ebin

The difference is that we need to specify the path to the erl executable and also provide a path to a boot file, which in our case is inside the releases folder of our standalone installation.

Once you have ran the previous commands, then you can proceed to create the environment variables as explained above on the Linux section



Getting Help and Providing Feedback

If you have questions about the contents of this guide or any other topic related to RabbitMQ, don't hesitate to ask them on the RabbitMQ mailing list.

Documentation feedback is also very welcome on the list. If you'd like to contribute an improvement to the site, its source is available on GitHub.