Incremental backups with Restic

This guide shows how to use restic for incremental backups. While it supports several storage system, we focus here on a the rclone remote described in a previews post. Updated: Adjusted systemd times (2021-04-11).

Start by installing restic. This guide was test with version 0.12.0 on GNU/Linux.

Repository creation 🔗

Since restic backup are encrypted, no need to use the crypt remote. We will use directly the onedrive remote.

Of course, the $REMOTE value below should match your setting:

$ export REMOTE=rclone:onedrive-ug:backup
$ restic -r $REMOTE init
enter password for new repository: 
enter password again: 
created restic repository d2XXXXX867 at rclone:onedrive-ug:backup

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

As stressed by restic: don’t loose your password (for instance use [safe paper backups]https://www.nadazero.ch/posts/long-term-preservation-for-passwords-via-qr-codes/)

Backuping files 🔗

Restic will ask for your password before each operation. You can just cache it in an environement variable. It will be forgotten when you exit the shell.

$ export RESTIC_PASSWORD=XXXXXXXXXXXX

Then you launch some backup:

$ restic -v -r $REMOTE --exclude='.git' --exclude-if-present='.nobackup' -v backup ~/docs

Here, I exclude .git/ repositories from backup and all directories that contained an hidden file named .nobackup (useful to avoid backuping cache directories). More info about exclusion in the doc .

In this example I backup a single directory, but you could try with your whole home.

Backups are incremental, only modification will be transferred, no data will be lost. Preceding versions and deleted files will be kept until you purge the snapshots (see below).

Restoring backups 🔗

You can display the available snapshots:

$ restic -r $REMOTE snapshots
repository XXXXXXXX opened successfully, password is correct
ID        Time                 Host            Tags        Paths
-----------------------------------------------------------------------------
be01ad57  2021-04-06 16:38:17  falcone-laptop              /home/falcone/docs
831eef3f  2021-04-06 17:00:45  falcone-laptop              /home/falcone/docs
-----------------------------------------------------------------------------
2 snapshots

There are many options to filter or group this list: man restic-snapshots.

To restore the latest snapshot:

$ restic -r $REMOTE restore e862ccf7 --target /tmp/restored/

Where e862ccf7 is the snapshot ID as shown in the snapshots command output above.

The --target directory will be created if it does not exist. The absolute path of the archived content will be recreated inside.

If you want to control what file are restored, use the --exclude and --include options.

Purging extra snapshot 🔗

Two different are required to purge unwanted snapshots:

1. Delete the snapshot with restic forget (but actual data will still be present).
2. Delete the data itself with restic prune

In programming language terms, the first operation destroys the references and the second one calls the garbage collector.

You can combine both steps with restic forget --prune.

You can set a policy by providing the number of snapshots you want to keep according to a period.

For instance I want to keep the last 7 daily snapshot, the last 5 weekly snapshots nd the last 25 yearly snapshots ¹:

restic -r $REMOTE forget --dry-run --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 25 --prune

The --dry-run flags allow me to check the policy. Restic will detail which snapshot will be kept and why.

When I a convinced everything’s fine we can run the same command without --dry-run.

As you will notice, the forget step is very fast, while the prunestep is slower.

Unattended operations 🔗

We could perform the command above manually, but in case of backups I prefer to have them running automatically. There are two possible routes: using a traditional and orthodox cronjob; or embracing heresy with systemd. Since I love bloated systems I have chosen the latter (I use emacs on a daily basis after all).

The following lines are inspired by a nice paper from Fedora Magazine .

Unattended backup 🔗

Start by creating a directory that will host the systemd units:

$ mkdir -p ~/.config/systemd/user/

Then copy the following data in the file ~/.config/systemd/user/restic.backup.

[Unit]
Description=Restic backup service
[Service]
Type=oneshot
ExecStart=restic backup --verbose --one-file-system  $BACKUP_EXCLUDES $BACKUP_PATHS
EnvironmentFile=%h/.config/restic.conf

Then create a configurationn file named ~/.config/restic.conf with the data I need:

BACKUP_PATHS="/home/falcone/docs /home/falcone/perso/docs /home/falcone/perso/homepage"
BACKUP_EXCLUDES="--exclude='.git' --exclude-if-present='.nobackup'"
RETENTION_POLICY="--keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 25"
RESTIC_REPOSITORY="rclone:onedrive-ug:backup"
RESTIC_PASSWORD=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Replace the data with yours. The password will be in cleartext. So chmod unwated permissions off. It may sound like a super dumb things to do, but if someone gains access to this file, the same person will gain also access to all your things…

You must reload systemd daemon before giving it a try. Check the logs and the resulting snapshots:

$ systemctl --user daemon-reload
$ systemctl --user start restic
$ journalctl --user-unit restic
$ restic -r $REMOTE snapshots

If everything is fine, it’s time to write the timer unit. In a file called ~/.config/systemd/user/restic.timer, copy the following lines:

[Unit]
Description=Restic backup (4 times a day)
[Timer]
OnCalendar=08/4:00:00
Persistent=true
[Install]
WantedBy=timers.target

Here I backup it at 8:00, 12:00, 16:00 and 20:00, to be sure I’ll have at least relevant daily snapshot per workday. If you just want a single daily backups you can use instead OnCalendar=daily.

The setting Persistent=true means that the task will be executed at boot if last scheduling was missed (because the computer was not running).

Enable the time with:

$ systemctl --user enable --now restic.timer
$ systemctl --user status restic.timer

Unattended pruning 🔗

Here the strategy is the same, we will create an unit and a timer. First the unit, in a file called ~/.config/systemd/user/restic-purge.service:

[Unit]
Description=Restic backup service (data pruning)
[Service]
Type=oneshot
ExecStart=restic forget $RETENTION_POLICY --prune
EnvironmentFile=%h/.config/restic.conf

Run, and check the output:

$ systemctl --user start restic-purge
$ journalctl --user-unit restic-purge
$ restic -r $REMOTE snapshots

Then write the timer in the file ~/.config/systemd/user/restic-purge.timer with the content:

[Unit]
Description=Purge data from the restic repository (daily)
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target

Again, enable it and check:

$ systemctl --user enable --now restic-purge.timer
$ systemctl --user status restic-purge.timer

You may notice that restic.service and restic-prune.service can be scheduled at the same time. However, it should not cause problem because restic uses exclusive locks.