Backup: rsnapshot

[VarrinS]

Greetings,

I’ve been using yunohost for a couple years now. My original use case was a Nextcloud server. I wanted the underlying files to remain easily available in the event of a problem with yunohost, Nextcloud, or my server itself. Additionally, at that time, yunohost’s backup was considered experimental.

In order to meet my objectives, I developed my own backup and restore scripts using the method I already use in other environments: rsnapshot. Rsnapshot is available in Debian, uses rsync to cut down on data transfer, and keeps multiple full backups in an ordinary file system for barely more than the (size) cost of an incremental backup. IMO, this is a superior method to the current tarball backup scheme.

I wonder if anyone has considered the merits of rsnapshot vs the current scheme or (what appears to be) the preference of the ‘next’ scheme (borg?)?

I’m not really much of a programmer. I hacked the stuff together, it’s not 100% automated (eg. on restore to a fresh install, I have to manually fix a thing or two to get the database working right), and it’s not really integrated into the yunohost system at all. I’m sure someone with some skill beyond mine could come up with a more elegant backup/restore implementation based on rsnapshot than I have made.

I suppose I’m simply stopping by to say: consider rsnapshot for backups

And thank you again to everyone who keeps this project going!

V-

[charly]

Hi V.

It’s funny, I just wrote a post on the same topic.

[TUTORIAL] How to backup and restore your entire system + data like a snapshot with raspBackup.sh

Hi, First of all, thanks to he Yunohost team. Yunohost is great and allows me to do great stuff and learn a lot. Bravo ! (version française ici) So, I’m using Yunohost for more than a year now with the following apps installed : Nextcloud (and some of its app like calendar, the phone tracker, notes, contacts) Wallabag ttrss Emails (that are pushed to my phone, that’s cool!) Etherpad Matrix-Synapse (in order to replace Whatsapp) The Wifi Hotspot But I also have some specific needs that ar…

How does work your script exactly ?
What do you need to fix in the database after a restore ?

[VarrinS]

Charly,

I actually wrote two sets of scripts, one set to automate some of the installation tasks and a second for the backup and restore processes. The install scripts are:

1: First step on the client: create SSH keys, copy remaining install scripts and backup scripts to the server.
2: Second step on the server: set up my drives correctly, install usbmount (I depend on this to keep my backup system working right. This includes some hacks to make it work right. In the future, I may need to come up with a different solution), install rsnapshot, copy backup script files to their correct locations, configure armbian, install yunohost.
3: Third step is run after yunohost is installed, users are added, nextcloud is installed, and some networking changes are accomplished: get the database password from the nextcloud configuration files for use by the backup system, make some more network changes, secure ssh, done.
One step I still need to add to the initial setup is configuring cron to run the backup scripts periodically (nightly in my case). As it is, I do that manually.

The backup script is then configured by running (one time) a configuration script which sets various script location variables, database password, secures files, etc.

Finally, having done all of the above, the backup script itself is quite simple which I will paste here:

#!/bin/bash

# Call this script to run the most frequent interval backup
# This must be run as root and may be scheduled with cron
#
# For higher level interval backups (eg weekly, monthly, etc.),
# call rsnapshot directly. It is not necessary to enter maintenance
# mode and dump the database. For example:
# rsnapshot -c/path/to/rsnapshot.ynrb monthly

sudo -u nextcloud php /var/www/nextcloud/occ maintenance:mode --on
sudo rsnapshot -c /root/ynrb/rsnapshot.ynrb daily
sudo mysqldump --defaults-file=/root/ynrb/nextcloudsql.cnf --single-transaction -h localhost -u nextcloud nextcloud > /media/usb/ynrb.backup/daily.0/nextcloud-sqlbkp.bak
sudo -u nextcloud php /var/www/nextcloud/occ maintenance:mode --off

I do not configure this to backup the entire system, just Nextcloud, but you could setup the include and exclude rules in rsnapshot to backup whatever you want, including everything. My strategy, instead, is to restore by rebuilding the box with a fresh install, and then restoring the nextcloud data. This does, however, create some problems. The script below contains instructions for how to solve the most important problem (database password doesn’t match):

#!/bin/bash

# WARNING - This script overwrites most of the data and
# configuration for a complete Yunohost / Nextcloud install.
# However, the backup script and this restore script does not
# backup any database data other than Nextcloud. 
# Using this restore script assumes the following:
#   1: Yunohost and Nextcloud are already (re)installed and working
#   2: Other apps' databases are recovered using another means
#   3: Users are configured as before
# If Nextcloud is the only app, this should restore a Yunohost/NC
# instance to a working condition, including some system config
# (eg. /etc).
#

#Turn maintenance mode on
sudo -u nextcloud php /var/www/nextcloud/occ maintenance:mode --on

# Edit the directories below to restore folders
sudo rsync -Aaxv --del --force /media/usb/ynrb.backup/daily.0/home/ /home/
sudo rsync -Aaxv --del --force /media/usb/ynrb.backup/daily.0/var/www/ /var/www/
sudo rsync -Aaxv --del --force /media/usb/ynrb.backup/daily.0/var/mail/ /var/mail/
sudo rsync -Aaxv --del --force /media/usb/ynrb.backup/daily.0/var/log/ /var/log/

# Start with a clean database
sudo mysql --defaults-file=/root/ynrb/nextcloudsql.cnf -h localhost -u nextcloud -e "DROP DATABASE nextcloud"
sudo mysql --defaults-file=/root/ynrb/nextcloudsql.cnf -h localhost -u nextcloud -e "CREATE DATABASE nextcloud"

# Restore the database
sudo mysql --defaults-file=/root/ynrb/nextcloudsql.cnf -h localhost -u nextcloud nextcloud < /media/usb/ynrb.backup/daily.0/nextcloud-sqlbkp.bak

# Cleanup and restart Nextcloud
sudo -u nextcloud php /var/www/nextcloud/occ maintenance:data-fingerprint
sudo -u nextcloud php /var/www/nextcloud/occ maintenance:mode --off

# Tell the user to fix password errors on new installs
echo If there are database errors, make sure the nextcloud config
echo has the database password from nextcloudsql.cnf

When restoring, it will overwrite the config file from the one which was backed up, however the database password will not match (due to a new one being created). The last two lines suggest how to fix that problem.

I’ve used this to restore a complete rebuild on a couple occasions and, though a bit kludgy, it does work. Caveat: it’s important to keep Yunohost and Nextcloud up to date (and whatever else you might use this concept for). Otherwise, when you rebuild the system, the versions may not match which could create all sorts of strange problems. Alternately, just set it up to backup the whole system.

Advantages of this method are: 1) the backup can be used as is (it’s a regular file system), 2) history is retained by using rsnapshot. Disadvantages are: 1) maybe the backup size is larger than if it was compressed (not too concerned about that personally), 2) not maintained by the Yunohost team, 3) requires some work to keep operating, 4) my configuration depends on usbmout which is going away (it doesn’t have to be that way, but that’s how I use it).

Hope that’s helpful info…

V-