Edit: AN APP SIMPLIFY NOW THIS TUTORIAL, SEE HERE: https://github.com/YunoHost-Apps/borg_ynh/blob/master/README.md
I let this tuto for people who want to know the general step the app made (not exactly the same)
Context
We want to backup Camille’s server on Sam one’s.
Camille : lachenille.tld
Sam: thedriver.tld
You need to setup borg from backport and initialize a repo.
Create a private key on Camille’s server
Camille creates a private and a public key:
$ ssh-keygen -t ed25519
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/root/.ssh/id_ed25519):
Don’t specify password
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/root/.ssh/id_ed25519.
Your public key has been saved in /home/root/.ssh/id_ed25519.pub.
The key fingerprint is:
SHA256:RDaEw3tNAKBGMJ2S4wmN+6P3yDYIE+v90Hfzz/0r73M root@lachenille.tld
The key's randomart image is:
+--[ED25519 256]--+
|o*...o.+*. |
|*o+. +o .. |
|o++ o.o |
|o+ ... . |
| + .S |
|+ o . |
|o+.o . . o |
|oo+o. . . o ....E|
| oooo. ..o+=*|
+----[SHA256]-----+
Then Camille send the public key to Sam. For example by email
$ cat /root/.ssh/id_ed25519.pub | mail sam@thedriver.tld
Prepare the way for the remote repository on Sam’s server
Sam creates a specific user for camille:
$ adduser camille
Sam adds Camille’s public key with borg limitation
$ echo "command=\"borg serve --restrict-to-path /home/camille/backup\",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc ssh-ed25519 AAAAC3NzaC1l[...]YRPujWoRyeVhhFZ0M root@lachenille.tld" >> /home/camille/.ssh/authorized_keys
Next Sam install borg-backup
$ apt-get install python3-pip python3-dev libacl1-dev libssl-dev liblz4-dev
$ pip3 install setuptools --upgrade
$ pip3 install borgbackup
If Sam’s server is a YunoHost, you need to avoid to backup camille’s backuped data. It’s very important if Sam’s server is backuped to Camille’s server.
$ touch /home/camille/.nobackup
$ mkdir -p /etc/yunohost/hooks.d/backup
$ cat > /etc/yunohost/hooks.d/backup/17-data_home <<EOF
#!/bin/bash
# Exit hook on subcommand error or unset variable
set -eu
# Source YNH helpers
source /usr/share/yunohost/helpers.d/filesystem
# Backup destination
backup_dir="\${1}/data/home"
# Backup user home
for f in \$(find /home/* -type d -prune | awk -F/ '{print \$NF}'); do
if [[ ! "\$f" =~ ^yunohost|lost\+found ]]; then
if [ ! -e "/home/\$f/.nobackup" ]; then
ynh_backup "/home/\$f" "${backup_dir}/\$f" 1
fi
fi
done
EOF
Setup borg on Camille’s server
$ apt-get install python3-pip python3-dev libacl1-dev libssl-dev liblz4-dev
$ pip3 install borgbackup
Create the remote repository from Camille’s server
$ borg init -e repokey camille@lachenille.tld:backup
You need to conserver the passphrase you set
Patch Camille’s Yunohost server
(Edit by Alex : this step is not necessary with Yunohost >= 2.7.x)
Currently, you need a patch to support custom backup methods
$ cd /usr/lib/moulinette/yunohost
$ rm backup.py
$ wget https://raw.githubusercontent.com/YunoHost/yunohost/ead685c455939c1c830a426e171f3e9c6266b5ad/src/yunohost/backup.py
$ cd /usr/share/moulinette/actionsmap/
$ rm yunohost.yml
$ wget https://raw.githubusercontent.com/YunoHost/yunohost/ead685c455939c1c830a426e171f3e9c6266b5ad/data/actionsmap/yunohost.yml
And run
mkdir -p /etc/yunohost/hooks.d/backup_method
mkdir -p /usr/share/yunohost/backup_method
Add the myborg hooks on Camille’s server
This hooks is a custom backup method. Don’t forget to replace repo and BORG_PASSPHRASE. If the passphrase contains double quote or dollar don’t forgot to escape those caracters with \
/etc/yunohost/hooks.d/backup_method/05-myborg
#!/bin/bash
set -e
BORG_PASSPHRASE="XXXXXXXX"
repo=camille@thedriver.tld:backup #$4
do_need_mount() {
true
}
do_backup() {
export BORG_PASSPHRASE
work_dir=$1
name=$2
repo=$3
size=$4
description=$5
LOGFILE=/var/log/backup_borg.log
ERRFILE=/var/log/backup_borg.err
current_date=$(date +"%d_%m_%y_%H:%M")
pushd $work_dir
borg create $repo::${name}_${current_date} ./ >> $LOGFILE 2>> $ERRFILE
popd
borg prune $repo -P ${name} --keep-daily=7 --keep-weekly=8 --keep-monthly=12 >> $LOGFILE 2>> $ERRFILE
}
do_mount() {
export BORG_PASSPHRASE
work_dir=$1
name=$2
repo=$3
size=$4
description=$5
LOGFILE=/var/log/backup_borg.log
ERRFILE=/var/log/backup_borg.err
borg mount $repo::$name $work_dir >> $LOGFILE 2>> $ERRFILE
}
work_dir=$2
name=$3
size=$5
description=$6
case "$1" in
need_mount)
do_need_mount $work_dir $name $repo $size $description
;;
backup)
do_backup $work_dir $name $repo $size $description
;;
mount)
do_mount
;;
*)
echo "hook called with unknown argument \`$1'" >&2
exit 1
;;
esac
exit 0
Test
$ yunohost backup create --ignore-apps --system conf_ldap -n test --methods myborg --debug --verbose
Run the backup each night
If it’s ok you can create a cron to run the command each night
/etc/cron.daily/yunohost-99-backup
#!/bin/bash
if yunohost -v | grep "version: 2." > /dev/null; then
ignore_apps="--ignore-apps"
ignore_system="--ignore-system"
else
ignore_apps=""
ignore_system=""
fi
filter_hooks() {
ls /usr/share/yunohost/hooks/backup/ /etc/yunohost/hooks.d/backup/ | grep "\-$1_" | cut -d"-" -f2 | uniq
}
# Backup system part conf
yunohost backup create $ignore_apps -n auto_conf --methods myborg --system $(filter_hooks conf)
# Backup system data
yunohost backup create $ignore_apps -n auto_data --methods myborg --system $(filter_hooks data)
# Backup all apps independently
for app in $(yunohost app list --installed -b | grep id: | cut -d: -f2); do
backup_methods=$(yunohost app setting $app backup_methods)
if [ -z "$backup_methods" ]; then
backup_methods=myborg
fi
if [ "$backup_methods" != "none" ]; then
yunohost backup create $ignore_system -n auto_$app --methods $backup_methods --apps $app
fi
done
Don’t forget to give rights to execute !
$ chmod a+x /etc/cron.daily/yunohost-99-backup
Disable the backup for a particular app
yunohost app setting strut backup_methods -v none
Backup a particular app with another methods
Create your specific methods
cp /etc/yunohost/hooks.d/backup_method/05-myborg /etc/yunohost/hooks.d/backup_method/15-mynextcloudborg
vi /etc/yunohost/hooks.d/backup_method/10-mynextcloudborg
Specify the method to use for the targeted app
yunohost app setting nextcloud backup_methods -v mynextcloudborg
[EDIT] LIMITATIONS
This tuto works if you are able to transfer your initial data within a maximum of 24 h.
So if you have an ADSL and more than 20G to transfer (at the first time) you should be careful.
How to restore
On a new server
If you are not on the same yunohost instance, you should copy the private key to access to camille server or recreate a private key and ask camille to put the new one.
Next, do:
borg list camille@lachenille.tld:backup
You need to identify recent backup. Conf, data and app have there own backup archives.
Begin by restoring conf
borg export-tar camille@lachenille.tld:backup::auto_confXXXX /home/yunohost.backup/archives/auto_confXXXX.tar.gz
yunohost backup restore auto_confXXXX
And next, restore data and each apps with the same method.
On the same server
do:
borg list camille@lachenille.tld:backup
You need to identify recent backup. Conf, data and app have there own backup archives.
Eventually remove the app you want restore.
borg export-tar camille@lachenille.tld:backup::autoXXXX /home/yunohost.backup/archives/autoXXXX.tar.gz
yunohost backup restore autoXXXX
[Optionnal] Backup on 2 servers (with a big remote backup space and a small one)
Add a new backup method and edit Passphrase and repo destination
cp /etc/yunohost/hooks.d/backup_method/05-myborg /etc/yunohost/hooks.d/backup_method/10-mysmallerborg
vi /etc/yunohost/hooks.d/backup_method/10-mysmallerborg
Change /etc/cron.daily/yunohost-99-backup with this content and edit size limit as you want:
#!/bin/bash
if yunohost -v | grep "version: 2." > /dev/null; then
ignore_apps="--ignore-apps"
ignore_system="--ignore-system"
else
ignore_apps=""
ignore_system=""
fi
# Size limit in MB per apps above which the script just backup on one server
declare -A limit
limit[nonessential]=10
limit[important]=150
limit[critical]=1024
# List of backup profiles
declare -A backup_profiles
backup_profiles[simple]="myborg"
backup_profiles[double]="mysmallerborg myborg"
backup_profiles[nobackup]="none"
if [ ! -e /etc/yunohost/hooks.d/backup_method/*-mysmallerborg ]; then
backup_profiles[double]="myborg"
fi
# Select backup methods to apply
get_backup_methods() {
importance=$1
size_max=$2 # In MB or "unlimited"
if [ "$size_max" == "unlimited" ]; then
size_max=$(expr $(df --output=size / | sed -n '2 p') / 1000)
fi
if (( $size_max > ${limit[$importance]} )); then
echo ${backup_profiles[simple]}
else
echo ${backup_profiles[double]}
fi
}
filter_hooks() {
ls /usr/share/yunohost/hooks/backup/ /etc/yunohost/hooks.d/backup/ | grep "\-$1_" | cut -d"-" -f2 | uniq
}
# Backup system part conf
yunohost backup create $ignore_apps -n auto_conf --methods $(get_backup_methods important 50) --system $(filter_hooks conf)
# Backup system data
yunohost backup create $ignore_apps -n auto_data --methods $(get_backup_methods important unlimited) --system $(filter_hooks data)
# Backup all apps independently
for app in $(yunohost app list --installed -b | grep id: | cut -d: -f2); do
backup_methods=$(yunohost app setting $app backup_methods)
if [ -z "$backup_methods" ]; then
size_max=$(yunohost app setting $app max_size)
size_max=${size_max:-unlimited}
importance=$(yunohost app setting $app importance)
importance=${importance:-important}
backup_methods=$(get_backup_methods $importance $size_max)
fi
if [ "$backup_methods" != "none" ]; then
yunohost backup create $ignore_system -n auto_$app --methods $backup_methods --apps $app
fi
done
Add max_size and importance to an app:
yunohost app setting wordpress max_size -v 500
yunohost app setting wordpress importance -v critical
yunohost app setting strut max_size -v 100
yunohost app setting strut importance -v nonessential
yunohost app setting nextcloud max_size -v 250000
yunohost app setting nextcloud importance -v important
)
