Backup and Restore RabbitMQ Data & Configurations

Every RabbitMQ node has a data directory that stores all the information that resides on that node.
A data directory contains two types of data: definitions (metadata, schema/topology) and message store data.
Nodes and clusters store information that can be thought of schema, metadata or topology. Users, vhosts, queues, exchanges, bindings, runtime parameters all fall into this category.

Definitions are stored in an internal database and replicated across all cluster nodes. Every node in a cluster has its own replica of all definitions. When a part of definitions changes, the update is performed on all nodes in a single transaction. In the context of backups this means that in practice definitions can be exported from any cluster node with the same result.
Messages instead are stored in a message store. A “message store” can be easy and incompletely defined as an internal store for messages, a single entity that’s transparent to the user.

Each node has its own data directory and stores messages for the queues that have their master hosted on that node. Messages can be replicated between nodes using queue mirroring. Messages are stored in subdirectories of the node’s data directory.

Definitions are usually mostly static, while messages are continuously flowing from publishers to consumers when performing a backup, first step is deciding whether to back up only definitions or the message store as well. Because messages are often short-lived and possibly transient, a backup of them is hight recommended from a stopped system to avoid inconsistencies.
Definitions can only be backed up from a running node.

Using rabbitmqadmin

This type of backup backup doesn’t include Messages since they are stored in a separate message store. It will only backup RabbitMQ users, vhosts, queues, exchanges, and bindings. The backup file is a JSON representation of RabbitMQ metadata. We will do a backup using rabbitmqadmincommand line tool.

First check if rabbitmq_managment is enabled on one of your RabbitMQ nodes:

rabbitmq-plugins enable rabbitmq_management

After that you will be able to download rabbitmqadmin on the RabbitMQ node with the following command:

wget https://{rabbitmq-node-hostname}:15672/cli/

Once downloaded, make the file executable and move it to the /usr/local/bin directory on the RabbitMQ node:

chmod +x rabbitmqadmin 
sudo mv rabbitmqadmin /usr/local/bin

To backup the RabbitMQ instance configuration, use the following command:

rabbitmqadmin export <configuration-backup-file-name.json>

To restore the RabbitMQ instance configuration, user the following command:

rabbitmqadmin import <configuration-backup-file-name.json>

Using Rabbitmq data directory

RabbitMQ Definitions and Messages are stored in an internal database located in the node’s data directory. To get the directory path, run the following command against a running RabbitMQ node:
rabbitmqctl eval 'rabbit_mnesia:dir().'

This directory contains many files:

# ls /var/lib/rabbitmq/mnesia/[email protected]
cluster_nodes.config  nodes_running_at_shutdown    rabbit_durable_route.DCD       rabbit_user.DCD             schema.DAT
DECISION_TAB.LOG      rabbit_durable_exchange.DCD  rabbit_runtime_parameters.DCD  rabbit_user_permission.DCD  schema_version
LATEST.LOG            rabbit_durable_exchange.DCL  rabbit_serial                  rabbit_vhost.DCD
msg_stores            rabbit_durable_queue.DCD     rabbit_topic_permission.DCD    rabbit_vhost.DCL

In RabbitMQ versions starting with 3.7.0 all messages data is combined in the msg_stores/vhosts directory and stored in a subdirectory per vhost. Each vhost directory is named with a hash and contains a .vhost file with the vhost name, so a specific vhost’s message set can be backed up separately.

In order to make a backup copy or archive this folder after rabbitmq has been stopped.

sudo systemctl stop rabbitmq-server.service

The example below will create an archive:

tar cvf rabbit-backup.tgz /var/lib/rabbitmq/mnesia/[email protected]

To restore from Backup, extract the files from backup to the data directory.
Internal node database stores node’s name in certain records. Should node name change, the database must first be updated to reflect the change using the following rabbitmqctl command:

rabbitmqctl rename_cluster_node <oldnode> <newnode>

When a new node starts with a backed up directory and a matching node name, it should perform the upgrade steps as needed and proceed to boot.

Leave a Reply

Your email address will not be published. Required fields are marked *