How to backup your YunoHost server on another server

Hi everybody, @ljf ,

Do you think this work could be packaged or merged in some way in Yunohost? There is a huge interest to integrate Borg in the Yunohost backup process. I’m ready to discuss and help to do it.

Tell me how far you’ve come now!

@klorydryk
Yes it could be directly in the Yunohost core.

See here the roadmap THE Backup roadmap

Since the custom backup method, I hadn’t had time to work on it, but we could discuss it on mumble or on the dev XMPP chat if you want :slight_smile:

1 Like

Hello!

Since Yunohost 3.0, it seems that --ignore-system option is not available.
Can I just remove it from the CRON job script?

Thanks,

Léo

Yes, idem for ignore-apps . Thanks for your report, I have edited the initial post with a condition to detect if we are on 2.7 or 3.0.

1 Like

Hi ljf,

It worked fine, I just had to update the borg vers using pip3 install borgbackup again and run again mkdir -p /usr/share/yunohost/backup_method.
Once done, with your edits, that worked fine.

Thanks,

Léo

An idea for Nextcloud :
Be able do 2 backups for nextcloud : one for the data and the other for the rest via yunohost backup method & backup_core.
The data path can be specify by admin (or setup by the hook script ?)
It will be activated or disactivated by a commented line.

By this way, you let the admin restore only the config and/or the data. The restore operation is faster --> not necessary to make a big fat tar.gz with all data. Just a tar.gz with config and an restore for the data

Or it’s a bad idea ?

I am understandable ?

That’s the kind of things I’d like to do too.
As I understood, for a backup, everything is extracted/copied to a tmp folder, and then the backup is made (tar.gz, bord, whatever you want) but for nextcloud (or maybe any other app with files that are directly files and can take a huge amount of data) the copy part of the backup is (I think) a waste of time and the backup could work directly from the real files.

Maybe that’s something already done and I didn’t read enough, but that is what I understood.

No, but maybe.

To create the tar.gz yunohost use directly the original file to add it to the tar.gz.

With borg, it’s quite different, because we need to organize the directory before. So Yunohost make mount bind in readonly mode and hardlinks to achieve this. In some case, it’s not possible, so the files/dir we are not able to link are copied. It could be the case with files on another disk.

There is an option for backup only configuration of an app.

It’s already possible, but only in cli mode:

BACKUP_CORE_ONLY=1 yunohost backup create --apps nextcloud

Hi @Aleks,
I’m currently stuck with a SSH authentification issue (all details in this thread Borgserver sur une machine non yunohost) and I wish to have more details please.

  • wich permissions are set for Camille on Sam’s server?
  • did you mean you add explicitly AllowUsers camille in etc/ssh/sshd_config? Or other things?
  • I’m not sure I understand the last point…

My problem is that I can backup with borg Camille datas on Sam’s server manually (ie: send in a shell sudo borg init [...] BUT it doesn’t work when I launch the Yunohost backup command: Sam’s server always ask for a password and ignore the authentification keys previously set up :frowning:

Thanks for your help.

Finally I came to avoid all the traps and found an (almost) clean and up to date solution to backup Camille’s server on Sam’s server (which is NOT a yunohost machine)

My home setup is:

  • a Yunohost server (Camille)
  • a borg backup server (Sam), which is in charge to backup everything on all my devices into encrypted borg repositories. This server is a raspberry pi and is NOT a yunohost machine.
  • then, after the borg operations all the backups are sent to my personal OVH cloud with https://rclone.org/

On Camille’s server:

Indicate the server where you want put your backups: sam.local
Indicate the ssh user to use to connect on this server: camille
Indicate a strong passphrase, that you will keep preciously if you want to be able to use your backups: N0tAW3akp4ssw0rdYoloMacN!guets
Would you like to backup your YunoHost configuration ? [0 | 1] (default: 1):
Would you like to backup mails and user home directory ? [0 | 1] (default: 1):
Which apps would you backup (list separated by comma or 'all') ? (default: all):
Indicate the backup frequency (see systemd OnCalendar format) (default: Daily):
  • Print the Alice’s public key with sudo cat /root/.ssh/id_borg_ed25519.pub

Then, on Sam’s server:

  • adduser camille --quiet --gecos ",,," --shell /bin/bash --disabled-password. Of course, replace camille by the user set previously :wink:
  • Allow Camille to connect on Sam’s server
sudo mkdir /home/camille/.ssh
sudo touch /home/camille/.ssh/authorized_keys
sudo echo "command=\"borg serve --storage-quota <the_quota_you_want> --restrict-to-repository <the_path_you_want>",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-user-rc <camille's publickey>" >> /home/camille/.ssh/authorized_keys

And that’s all!
On camille server, you can test the system with the command sudo yunohost backup create -n test --methods borg_app --debug

Now, some tips which could help:

  • if during the backup a password is requested to connect on Sam’s server, it means there is an issue with the command command in the authorized_keys file. In this case, you can edit the file with sudo nano /home/camille/.ssh/authorized_keys, delete the line and simply put the Camille’s key.
    I don’t know why it doesn’t work for me. Even the simplest command available everywhere on internet as an example like command=date [...] fails…:frowning: If you have an idea please tell me! :pray:
    :warning: By removing the command, you allow Camille to execute any command on Sam’s server, and not only a borg command. It’s safe if both machines are yours, otherwise you have to highly trust your mates :wink: :warning:
  • In my case the Camille’s home repository is located on the SD card of my raspberrypi. Because you can’t change the location of the setup when you install borg_ynh, you can do a mount bind in your /etc/fstab or add a symlink on an external usb drive (example: sudo ln -s /media/usbdisk/data/YunohostBackup /home/camille/backup.
    :warning: In this case you have to be careful about the owner of the backup folder, it must be camille (and not root or someone else) or you will have a backtrace like :warning:
47820 ERROR Échec de l’exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Traceback (most recent call last):
  File "/usr/lib/moulinette/yunohost/hook.py", line 283, in hook_callback
    no_trace=no_trace, raise_on_error=True)[1]
  File "/usr/lib/moulinette/yunohost/hook.py", line 397, in hook_exec
    raise YunohostError('hook_exec_failed', path=path)
YunohostError: Échec de l’exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app

Polochon :tropical_fish:

3 Likes

Hi all,

It seems that the .nobackup file trick (to avoid a directory from being backed up by ynh) does not work anymore (on my 3.8 raspian). Any clue from the team ?

I can provide logs as needed.

Hello all,
thank you for this tutorial and apps which look amazing.

I’m considering following that path but before I’d love to understand how does it compare to scheduling a regular copy of latest local backup to a remote storage (s3) please :slight_smile:

Bonjour, si j’ai bien compris, je peux sauvegarder avec ce tuttoriel les backups d’un serveur Yunohost hébergé sur un serveur dédié sur un serveur Yunohost hébergé chez moi, les deux serveurs ayant chacun des configurations différentes, c’est bien ça ?

oui, mais il est fortement recommandé d’utiliser les apps borg et borgserver.

Salut,
J’essaie de configurer borg pour backuper un serveur A vers un disque dur externe branché sur le serveur B. Et je bloque sur le choix du repository :

  • Si je laisse ssh://user@domain.tld:port/~/backup/ -» ça marche, mais ça fait le backup sur le DD interne
  • J’ai essayé ssh://user@domain.tld:port/mnt/DDexterne/backup -» marche pas, même avec un chown -R user:user /mnt/DDexterne.
  • J’ai essayé de monter le disque externe directement sur /home/user/backup, mais dans ce cas je voudrais backuper dans un sous-dossier : ssh://user@domain.tld:port/~/backup/borg. C’est pour éviter que le backup ne remplisse mon disque au cas où le disque externe ne serait pas monté. -» ça ne marche pas.
    Si j’ai bien compris, l’utilisateur créé par borg server est restreint au dossier /~/backup. Et en même temps, quand on installe borg sur le serveur A, il est bien indiqué :

With borgserver_ynh apps you can’t specify another repo path than ~/backup.

Comment je peux faire ? Est-ce qu’il ne faudrait pas pouvoir indiquer le path à l’installation de borgserver, pour qu’il donne le bon accès à user ?

Avec borgserver, je pense que tu peux personnaliser ça en utilisant un mount --bind pour monter le dossier de ton dd externe sur le dossier ~/backup de ton dd interne. Il faut aussi l’ajouter dans fstab du coup.

Les autres solutions consiste à ajouter cette fonctionnalités dans borgserver ou coder avec moi ce mois ci sur la PR de backup…

1 Like

Is there an English explanation on this that works with nextcloud 24 as currently on yunohost? I understood nextcloud kind of advertises this backing up to another nextcloud as a feature. But how does it work? If I have one nextcloud 24 on a cloud vps with yunohost and one nextcloud 24 in Docker on an rpi at home, could I backup them to each other completely? It’s hundreds of GB.

Here, it’s not NextCloud <-> NextCloud, but Borg <-> Borg (using YunoHost’s backup scripts).

I think the tutorial is outdated, maybe the link in the first post is up to date.

Basically, if you have 2 servers, with at least the server you want to backup running YunoHost, and it’s easier if both.

  1. Install borg app on your server
  2. Note everything you define for the installation
  3. Install borg_server on the distant server
  4. Run the first backup, it will be REALLY LONG (can be days, depending on the quantity of data and bandwidth)

Normally, there will be more details in the borg documentation

(Seems outdated too, but it should work, but it seems important to rewrite it so the same questions will not be asked again and again :sweat_smile:)

Thanks outdated or not I will have a look into it some of these… Checks calendar… Months, I’m afraid.

1 Like