How does .nobackup work?

:uk:
Let’s say i don’t want to backup a user’s home folder, i just have to put an empty .nobackup file at the root of it, right ? Now if i don’t want to backup this user’s photos, but its music, this other user’s movies but not this other subfolder and so on, how do i proceed ?

In other words : is the .nobackup technique useable in other folders than home ? Is any folder containing such a file excluded from the backup archive ? I’m not sure it is, but why not ?

Real life exampe : I have a big music library in yunohost.multimedia/shared and a big completed torrents Transmission folder too. Those are mounted as external storages in nextcloud for conveniency. I don’t want to backup those whenever i backup my whole server (as it’s really heavy and subject to change). For now the workaround i’ve found (as .nobackup doesn’t seem to do the job) is :

  • untick transmission and navidrome on the backup creation page
  • delete the external storages before each backup
  • backup stuff
  • put back external storages

If i don’t follow those steps, save archive is over 9000 tons, takes ages to complete and puts the server on its knees, hence the .nobackup question…


:fr:
Si je veux exclure un dossier home de ma sauvegarde, j’ajoute un fichier vide .nobackup à sa racine, c’est bon ? Maintenant si je veux sauvegarder la musique d’un user mais pas ses photos, ou les flims et pas un autre truc, comment puis-je faire ?

En d’autres termes : le coup du .nobackup est-il utilisable ailleurs que dans home ? Est-ce chaque sous dossiers qui comporte un tel fichier est ignoré ? J’ai pas l’impression, mais pourquoi ne le serait-il pas ?

Cas concret : j’ai une grosse librairie de musique dans yunohost.multimedia/shared et un gros dossier completed torrents pour Transmission. Les deux sont montés en stockages externes dans nextcloud parce que c’est pratique et qu’il n’y a pas d’autre moyen pour y accéder simplement (à ma connaissance). Je ne veux pas inclure ces énormes dossiers dans mes backups donc la combine que j’ai trouvée c’est :

  • virer mes stockages externes
  • décocher transmission et navidrome avant de lancer le backup
  • lancer le backup
  • remettre mes stockages externes quand c’est fini

Si je fais pas ça l’archive est gigantesque et la sauvegarde prend des années pendant lesquelles le serveur est à l’agonie, d’où ma question sur le .nobackup

up it goes

.nobackup is only tested for at the root of the user directory. I don’t know if you can expand that, actualmente. Pinging @ljf, who added the feature. :wink:

Regarding Transmission, that’s a lack of documentation sorry. The download directory is marked is_big in the backup script, so you can use the feature evoked there: in /etc/yunohost/transmission/settings.yml, set do_not_backup_data to 1.
Hopefully we will get a nice config panel for that in the next versons of YunoHost. :wink:

1 Like

That’s what i thought and it doesn’t seem to work when it is “deeper” in the directory tree.

Thanks that’s a great start !

I’m still concerned that even if transmission save file would not include its own data, the completed torrents folder being linked into nextcloud as an external storage, nextcloud save would include it… But i don’t really want to test it as it drives the whole server crazy when the save file is too big. Any assessment about that ? I’ll try to ask the nextcloud packager…

Yes it’s quite limited… At the beginning this feature was quite underground and uses by borgserver_ynh.

See here for the hook that does that : https://github.com/YunoHost/yunohost/blob/dev/data/hooks/backup/17-data_home#L15

We have no way in current yunohost backup process to exclude a subdirectory from a backuped directory… Borg can do it, but the yunohost backup create command can’t.

So for example for a tree like this

dirA
  subdirB
  subdirC
  fileD
  fileE
  ...

If you want to exclude subdirC, you need to do that like this in your backup hook:

ynh_backup "dirA/subdirB"
ynh_backup "dirA/fileD"
ynh_backup "dirA/fileE"
...

So it’s a bit difficult to do this proprely with an unknown tree (it’s possible, but not really simple)…

However, you can create your own backup hooks, to implement specific things related to your setup.

You cand do it like this:

mkdir -p /etc/yunohost/hooks.d/backup
mkdir -p /etc/yunohost/hooks.d/restore
touch /etc/yunohost/hooks.d/backup/99-data_custom
touch /etc/yunohost/hooks.d/restore/99-data_custom

Example for backup/99-data_custom

#!/bin/bash

# Source YNH helpers
source /usr/share/yunohost/helpers

ynh_backup_dest (){
    YNH_CWD="${YNH_BACKUP_DIR%/}/$1"
    mkdir -p $YNH_CWD
    cd "$YNH_CWD"
}

# Exit hook on subcommand error or unset variable
ynh_abort_if_errors

# Openvpn
ynh_backup_dest "data/custom/openvpn"
ynh_backup "/etc/sysctl.d/openvpn.conf"
ynh_backup "/etc/openvpn"
ynh_backup "/etc/fail2ban/jail.d/openvpn.conf"
ynh_backup "/etc/fail2ban/filter.d/openvpn.conf"

# Samba
ynh_backup_dest "data/custom/samba"
ynh_backup "/etc/samba"
ynh_backup "/var/lib/samba"
ynh_backup "/etc/yunohost/hooks.d/post_user_create/99-samba"
ynh_backup "/etc/yunohost/hooks.d/post_user_delete/99-samba"
ynh_backup --src_path="/etc/yunohost/hooks.d/post_user_update/99-samba" --not_mandatory
ynh_backup "/etc/cron.daily/clean-trash"

...

Example for restore/99-data_custom

#!/bin/bash

# Source YNH helpers
source /usr/share/yunohost/helpers

ynh_restore_dest (){
    YNH_CWD="${YNH_BACKUP_DIR%/}/$1"
    cd "$YNH_CWD"
}

# Exit hook on subcommand error or unset variable
ynh_abort_if_errors

# Openvpn
app="custom_openvpn" # Cette variable est importante pour le helper suivant
ynh_install_app_dependencies "openvpn openvpn-auth-ldap samba" 

ynh_restore_dest "data/custom/openvpn"
ynh_restore_file "/etc/sysctl.d/openvpn.conf"
ynh_restore_file "/etc/openvpn"
ynh_restore_file "/etc/fail2ban/jail.d/openvpn.conf"
ynh_restore_file "/etc/fail2ban/filter.d/openvpn.conf"

# Samba
app="custom_samba" # Cette variable est importante pour le helper suivant
ynh_install_app_dependencies "samba" 

ynh_restore_dest "data/custom/samba"
ynh_restore_file "/etc/samba"
ynh_restore_file "/var/lib/samba"
ynh_restore_file "/etc/yunohost/hooks.d/post_user_create/99-samba"
ynh_restore_file "/etc/yunohost/hooks.d/post_user_delete/99-samba"
ynh_restore_file --src_path="/etc/yunohost/hooks.d/post_user_update/99-samba" --not_mandatory
ynh_restore_file "/etc/cron.daily/clean-trash"

Important: You need to adapt it to indicate directories or files you want to backup

You can also decide to rewrite 17-data_home system hook, in this case replace 99-data_custom by 17_data_home.

1 Like