[Borg & Borg Server] Deduplicated, encrypted and remote backups

I only use it as a local backup (for now)

New release:

  • 02/12/2020 - 1.1.13~ynh2

    • Hotfix to avoid to send first email at each run of borg…
1 Like

Is there a way to change the parameters ?
There are a few changes that I would like to do :

  • Change the repository max size (I upgraded the hard drive)
  • Change the frequency of backups
  • Retrieve the server SSH key to add another client (some apps will need a daily backup, some other only a weekly one)

You could use restic for deduplicated backups. Easy to setup.

https://restic.net/

New testing:

  • 05/04/2021 - 1.1.16~ynh16
    • [fix] Passphrase issues (space, dollar…)
    • [fix] Setup several borg/borgserver
    • [fix] Don’t prune wordpress__2 app if we prune wordpress app
    • [fix] Major yunohost version change (stretch → buster, buster → bullseye)
    • [enh] Allow to setup a local repo (for example on an external drive)
    • [enh] Support “exclude:” instructions to exclude an app from backup
    • [enh] Send a backup success or a backup failed mail
    • [enh] Allow to specify a custom dir on a remote server (and not force the usage of backup dir)
    • [enh] Add fuse support to be able to mount an archive

Feel free to report any bug on this testing. Don’t miss to indicate the version of the package.

On my side i have validated what i wanted to validate. I think this new version is better than production one;

New stable release:

  • 06/04/2021 - 1.1.16~ynh16
    • [fix] Passphrase issues (space, dollar…)
    • [fix] Setup several borg/borgserver
    • [fix] Don’t prune wordpress__2 app if we prune wordpress app
    • [fix] Major yunohost version change (stretch → buster, buster → bullseye)
    • [enh] Allow to setup a local repo (for example on an external drive)
    • [enh] Support “exclude:” instructions to exclude an app from backup
    • [enh] Send a backup success or a backup failed mail
    • [enh] Allow to specify a custom dir on a remote server (and not force the usage of backup dir)
    • [enh] Add fuse support to be able to mount an archive
3 Likes

It seems that the upgrade broke something.
Borg is not installed anymore on my server :
https://paste.yunohost.org/mikuzucove.bash
(and restoration failed)
((and repository password should NOT be logged, this is really dangerous, I trust borg to crypt my data, but if anyone stealing my disk can just go read a log file to have the password, this is bad))

As I want to use a local repo, what are the steps to do ?
Can I remove the borg server ? Will it delete the existing repo ?
(For now I have a local repo, but using the borg server on the same computer)

Your bug is due to a connectivity issue or a problem with your proxad repo:

Could not connect to debian.proxad.net:80 (2a01:e0c:1:1598::2), connection timed out Could not connect to debian.proxad.net:80 (212.27.32.66), connection timed out

Fix this issues and rerun upgrade should fix your repo.

Restore operation failed for the same reason.

If someone steals the root storage of the server which is backupped (and where are the logs), this person can simply read your password in borgserver app settings. And also a lot of other things in your server, until you configure an encrypted luks volume for your root partition.

However, could you explain in which log you see the password ? In the yunohost journal of upgrade ? In borg log ?

I am currently writing a documentation about that. Steps should be something like:

  1. Mount your external disk in a dir (for example /backup ). You have to configure /etc/fstab by yourself too.
  2. Install the borg app and answer the path “/backup” to the first question

Yes borgserver is not needed anymore in this situation. Theoretically, the user is removed but repo is keeped.

The password is visible in the upgrade log, chich can be automatically shared here when an upgrade fails.
(My root partition is secured, I forgot that the password was stored to be able to do the backup, so at least it should be removed from the logs).

For now, I managed to fix my problem with apt, and reinstalled borg twice (one for a daily backup, one for a weekly one, with more apps saved).

I’ll uninstall the borg server app after I’ll make a backup of the backup, just in case :grin:

I published a pull request on yunohost core for this issue.

1 Like

Hi, it seems like a lot of us are working on our backups setups - maybe the fire at OVH DC is somehow related :slight_smile: . First of all thanks for this app. I have a question which would probably be better to ask to the Borg community directly, but who knows, maybe somebody here could help me.

I would like to have my main computer acting as a distant repo for my Yunohost instance (which is installed on a VPS). Because my main computer is not always on, I would like it to be the one who triggers the backup. Do you think that’s possible ?

Your question is about pull or push mode for backup. Currently the borgbackup is in push mode, i mean the server to backup send its file to the remote repo.

In pull mode, that’s the machine where is the repo that trigger the backup. There is several advantages to this in security. Cause with a pull mode if the server to backup is infected, the attackers can’t delete the backup on the other machine. However having backups onto a personal computer destroy a bit the advantages of this approch in security matter.

For working in pull mode, i think Borg community will advice you to setup a sshfs from your server onto your personal computer. The issue with this solution is it can’t simply work with the yunohost backup system.

Here is steps describing how borg_ynh works with yunohost backup system:

  1. A timer systemd run backup-with-borg script
  2. This script run yunohost backup create --method borg_app command on each part/apps of the system you have selected during borg_ynh setup
    1. Yunohost ask the app or the system to list files to backup
    2. YunoHost transmit the list to the custom “Backup Method” borg created by borg_ynh apps
    3. This custom “Backup Method” use borg create command with the list of files to transmit files onto remote machine

So if you just do a sshfs you can’t use the yunohost mechnism.

However, you could do another things, to trigger the push mode from your machine.

  • Install borg_ynh with a “yearly” frequency.
  • Create a private/public keys onto your personal computer
  • Create a specific user backup_trigger and add ssh keys into authorized_keys with Forcecommand
  • In the force command call the backup-with-borg borg (or systemctl start borg) command

Now to trigger a backup from your personnal computer you just need to run ssh -i .ssh/yourkeys backup_trigger@YOURDOMAIN.tld . You can create a launcher or a script that launch at startup on your personal computer.

2 Likes

Thanks for your very detailed answer and advice.

Now I understand way better your Borg Ynh app. It’s really cool to have Yunohost backup system triggering Borg create command with the correct list of files associated.

One question : if I decide to go the standard way, and use a friend’s server or another server for my distant repo. If my whole Yunohost is 300GBs, do I need 300GB locally + 300GB remotely or 300GB remotely are enough ?

Thanks

If you have a server with 300G of files and 20G free, and a remote server with 320G free, you will be able to do your backup. You don’t need to have 50% of free space on the server you backup to be able to do the backup (there are optimization in unohost backup command and in borg command to avoid that).

If you define a daily backup, i suggest you to provision at least 10% more, so a quota with 330G on the borgserver app.

Borg works with deduplication, but if you keep several copy from each month you probably will have variation. In some cases, it could be the double or just 10% in more or ??? . It depends of what you backup and variation in thje list of files. If on your nextcloud you replace a compressed file of 2G by another of the same size but completly different, you will backup 2G each days… So if you 14 copy, you need 28G.

1 Like

Clear, thanks !

New release for borgserver_ynh:

  • 2021/04/11 - 1.1.16~ynh3
    • [enh] Check regularly the backup has been made (it checks only the repo has changed, not if the backup is well formed)
    • [fix] Setup several borg/borgserver
    • [fix] Major yunohost version change (stretch → buster, buster → bullseye)
    • [fix] Missing settings for ssh_user/public keys

New release for borgserver_ynh:

  • 2021/04/14 - 1.1.16~ynh3 (same version as the old one couldn’t be installed)
    • [fix] Install process failed but succeed in CI /o\

New testing release for borg_ynh:

  • 06/04/2021 - 1.1.16~ynh18
    • [fix] Prune not working anymore correctly
    • [fix] Warning in diagnosis
    • [fix] Undetected failure on borg create due to date log system
1 Like

New release for borg_ynh:

  • 26/04/2021 - 1.1.16~ynh20
    • [fix] Backup and upgrade for local repo

New release for borgserver_ynh:

  • 09/05/2021 - 1.1.16~ynh5
    • Support of yunohost 4.2 (you won’t be able to install borgserver on yunohost <4.2)

Does it mean I have to uninstall the borg app and re-install the new release after upgrading to yunohost 4.2? Thanks.