Restic is a backup program intended to be easy, fast, verifiable, secure, efficient, and free. Restic supports a range of backup targets, including local disk, SFTP, S3 (or compatible APIs like Minio), Backblaze B2, Azure, Google Cloud Storage, and zillions of others via rclone.
Restic is one of the more popular open-source backup solutions, and is often compared favorable to "freemium" products by virtue of its licence.
/var/data/restic/restic.exclude details which files / directories to exclude from the backup. Per our data layout, runtime data such as database files are stored in /var/data/runtime/[recipe], and excluded from backups, since we can't safely backup/restore data-in-use. Databases should be backed up by taking dumps/snapshots, and backing up these dumps/snapshots instead.
Prepare Restic environment
Create /var/data/config/restic/restic-backup.env, and populate with the following variables:
# run on startup, otherwise just on cronRUN_ON_STARTUP=true# when to run (TZ ensures it runs when you expect it!)BACKUP_CRON=001***
TZ=Pacific/Auckland
# restic backend/storage credentials# see https://restic.readthedocs.io/en/stable/040_backup.html#environment-variables#AWS_ACCESS_KEY_ID=xxxxxxxx#AWS_SECRET_ACCESS_KEY=yyyyyyyyy#B2_ACCOUNT_ID=xxxxxxxx#B2_ACCOUNT_KEY=yyyyyyyyy# will initialise the repo on startup the first time (if not already initialised)# don't lose this password otherwise you WON'T be able to decrypt your backups!RESTIC_REPOSITORY=<repo_name>
RESTIC_PASSWORD=<repo_password>
# what to backup (excluding anything in restic.exclude)RESTIC_BACKUP_SOURCES=/data
# define any args to pass to the backup operation (e.g. the exclude file)# see https://restic.readthedocs.io/en/stable/040_backup.htmlRESTIC_BACKUP_ARGS=--exclude-file/restic.exclude
# define any args to pass to the forget operation (e.g. what snapshots to keep)# see https://restic.readthedocs.io/en/stable/060_forget.htmlRESTIC_FORGET_ARGS=--keep-daily7--keep-monthly12
Create /var/data/config/restic/restic-prune.env, and populate with the following variables:
# run on startup, otherwise just on cronRUN_ON_STARTUP=false# when to run (TZ ensures it runs when you expect it!)PRUNE_CRON=004***
TZ=Pacific/Auckland
# restic backend/storage credentials# see https://restic.readthedocs.io/en/stable/040_backup.html#environment-variables#AWS_ACCESS_KEY_ID=xxxxxxxx#AWS_SECRET_ACCESS_KEY=yyyyyyyyy#B2_ACCOUNT_ID=xxxxxxxx#B2_ACCOUNT_KEY=yyyyyyyyy# will initialise the repo on startup the first time (if not already initialised)# don't lose this password otherwise you WON'T be able to decrypt your backups!RESTIC_REPOSITORY=<repo_name>
RESTIC_PASSWORD=<repo_password>
# prune will remove any *forgotten* snapshots, if there are some args you want# to pass to the prune operation define them here#RESTIC_PRUNE_ARGS=
Why create two separate .env files?
Although there's duplication involved, maintaining 2 files for the two services within the stack keeps it clean, and allows you to potentially alter the behaviour of one service without impacting the other in future
Restic Docker Swarm config
Create a docker swarm config file in docker-compose syntax (v3) in /var/data/config/restic/restic.yml , something like this:
Fast-track with premix! 🚀
"Premix" is a git repository which includes necessary docker-compose and env files for all published recipes. This means that you can launch any recipe with just a git pull and a docker stack deploy 👍.
🚀 Update: Premix now includes an ansible playbook, enabling you to deploy an entire stack + recipes, with a single ansible command! (more here)
Setup unique static subnets for every stack you deploy. This avoids IP/gateway conflicts which can otherwise occur when you're creating/removing stacks a lot. See my list here.
Serving
Launch Restic stack
Launch the Restic stack by running docker stack deploy restic -c <path -to-docker-compose.yml>, and watch the logs by running docker service logs restic_backup - you should see something like this:
Of note above is "Repository successfully initialized" - this indicates that the repository credentials passed to Restic are correct, and Restic has the necessary access to create repositories.
Restoring data
Repeat after me : "It's not a backup unless you've tested a restore"
The simplest way to test your restore is to run the container once, using the variables you're already prepared, with custom arguments, as per the following example:
The example above restores the entire/var/data folder (minus any exclusions). To restore just a subset of data, add the -i <regex> argument, i.e. -i plex
Chef's notes 📓
The /var/data/restic/restic.exclude exists to provide you with a way to exclude data you don't care to backup. ↩
A recent benchmark of various backup tools, including Restic, can be found here. ↩
Did you receive excellent service? Want to compliment the chef? (..and support development of current and future recipes!) Sponsor me on Github / Ko-Fi / Patreon, or see the contribute page for more (free or paid) ways to say thank you! 👏
Employ your chef (engage) 🤝
Is this too much of a geeky PITA? Do you just want results, stat? I do this for a living - I'm a full-time Kubernetes contractor, providing consulting and engineering expertise to businesses needing short-term, short-notice support in the cloud-native space, including AWS/Azure/GKE, Kubernetes, CI/CD and automation.
Want to know now when this recipe gets updated, or when future recipes are added? Subscribe to the RSS feed, or leave your email address below, and we'll keep you updated.