[Borg & Borg Server] Deduplicated, encrypted and remote backups

ljf · July 15, 2021, 9:45am

The /opt is on HDD or SDcard ? Which kind of partition have you (ext4, btrfs ?)

Jaxom99 · July 15, 2021, 10:20am

I have a setup with berryboot on the SD card, and all raspbian on the HDD. It’s on ext4. (I think we’re digging too far, though )

ljf · July 15, 2021, 10:22am

I ask that after reading partially `borg init` failing on Raspbian (1.1.9) with 64-bit kernel · Issue #4891 · borgbackup/borg · GitHub

Jaxom99 · July 15, 2021, 10:40am

sad news, thanks for finding it. I don’t get how they closed/fixed it, installing libxxhash-dev on my rpi did not. Borg init still fails with Remote: Fatal Python error: Bus error…

Edit : I seem to have fixed it by manually installing a more recent version of borg, mentionned in the thread as per : https://borg.bauerj.eu/ . which borg indicated “/user/local/bin/borg” so I changed that one. Then the backup ran successfully

Edit2 : this change is overwritten with a 2nd install of borgserver app on the same machine (and maybe on upgrades as well)… To be continued…

Maybe we should split those few posts and I could mark it as ~solved ? It could be helpful to other Raspi users !

ljf · September 7, 2021, 11:57am

New release for borg_ynh:
- 07/09/2021 - 1.1.16~ynh23
  [fix] Borgbase change the location of repo

Mamie · September 13, 2021, 7:39am

Is this normal that I do not have any success/failure mails since this upgrade ?

ljf · September 13, 2021, 8:04am

A lot of people ask for a mail only if it fails. So a specific settings has been created and the default behaviour has changed, if you was satisfied of the old behaviour you can write that:

yunohost app setting borg mailalert -v "always"

SiM · September 13, 2021, 10:05pm

Hi,
I just installed Borg on one YNH instance (4.2.8.3), but I cannot find the public_key needed to set up the borgserver:

The ssh key was not displayed at the end of install
I have not received a mail after the install
There is no setting “public_key” on Borg app.
There isn’t any id_borg_ed25519 on my /root/.ssh folder

My install log seems ok:
https://paste.yunohost.org/raw/kanoyupini

What am I missing?

ljf · September 15, 2021, 8:25am

You have configured your borg app with the repo /home/yunohost.backup/borg/ , so it’s normal you don’t get a public key, cause it doesn’t need to backup on this location of your server.
If you want to make a remote backup, please uninstall borg, and reinstall with the Webadmin and read the help on the first question to configure correctly the repo.

SiM · September 15, 2021, 8:56am

Thanks for the answer and I realized my mistake at the same time by reading the manifest.json… Indeed I thought I had to specify a local path even if I wanted to use a remote server.
Wouldn’t it be possible to consider displaying the help in CLI too?

ljf · September 15, 2021, 9:15am

It has already been fixed in 4.3. This version will be released soon (before october).

Mamie · October 5, 2021, 7:18pm

I just created a borg server for a friend (for a backup not from YunoHost)
I asked the system to send me an email fi there were no update for 8 days.
And I received an email a few hours later (a backup had already be done), which seems like a bug :

Subject : Cron root@myserver : Monitor weblate backup ; ALERT_DELAY="$(yunohost app setting borgserver__2 alert_delay)"; [[ $(find /home/weblate/backup -follow -mtime -${ALERT_DELAY} -ls | wc -l) > 0 ]] || ( echo “No file has been backuped in /home/weblate since ${ALERT_DELAY} days” | mail -s “[YNH] Backup missing : weblate” $(yunohost app setting borgserver__2 alert_mails))
Content : Uhoh, somehow the lock /var/run/moulinette_yunohost.lock did not exist …

It seems quite wrong

Just in case, the folder with the backups is a symlink to an external drive (in case the test command can have problem with this)

wbk · November 7, 2021, 10:07pm

After reading the topic I am not quite sure whether I can use one borg_server for more than 1 borg-backup-app.

I thought to have multiple Yunohosts write backups to a central Yunohost, but I am under the impression that there can only be 1 borg_server per Yunohost, and only 1 borg-backup-app-client per borg_server.

If so, I think to use the borg-backup-app-client standalone, and have a SSHFS directory from the remote backup server mounted at the backup location.

Does one of the two solutions have a benefit over the other?

Mamie · November 7, 2021, 10:30pm

Just for info, on my server there are 2 borg clients, making local backups, and one borg server for someone else to use.
And soon, a 3rd client for distant backups.

And everything works great

But for multiple clients, it is best to havi multiple servers, just in case.

For example, my 2 clients save to the same folder and I have to be 100% sure that they never try to backup at the same time.

And with one server with multiples clients, the server is made for one, so for it, both will be the same and one can see/delete the other user backups.

wbk · November 8, 2021, 10:49pm

Wohoo, mine as well!

The situation is like:

homeYuno
- has space for backup
- create users and target dir here
VPSYunoA
- no space for backup
- add sshfs to fstab and install borg here
VPSYunoB
- no space for backup
- add sshfs to fstab and install borg here

VPSYuno* needs to do a remote backup, but I am not sure how to do it with Borg Server. Instead, I lure Borg Backup into thinking it is doing a local backup, while it is actually a remote backup via SSHFS.

On homeYuno:

add user vpsassh
- add to (new) SSH-group in users/manage users and groups, with SSH-rights
create the backup destination
- # su vpsassh -
- $ mkdir -p ~/backup/vpsyunoa.tld/borg/

On VPSYunoA:

Prepare the SSH connection
- `$ sudo su -
- # ssh-keygen
- # ssh-copy-id vpsassh@homeyuno.tld
- (actually, because of an IPv6 problem on my side, # ssh-copy-id -o AddressFamily=inet vpsassh@homeyuno.tld )
Add a line to /etc/fstab and check mount/umount for errors
- # mkdir /backup/borgremote/homeyuno/
- # echo 'sshfs#vpsassh@homeyuno:/home/vpsassh/backups/vpsyunoa.tld/borg /backup/borgremote/homeyuno fuse _netdev,reconnect,allow_other, inet 0 0' >> /etc/fstab
- The backup is written to root; I’m not sure whether it will recursively back up when I put it in a users’ directory in /home/*
- # mount -a
- # umount /backup/borgremote/homeyuno
- # mount -a
- # mount|grep backup
Install Borg Backup
- I installed it via the web GUI
- In which borg repository location do you want to backup your files ?
- in: /backup/borgremote/homeyuno
- change options to taste

The first backup ran with no problem. Checking contents from VPSYunoA:

# borg list /backup/borgremote/homeyuno
Enter passphrase for key /backup/borgremote/homeyuno:
_auto_conf-2021-11-08_00:00          Mon, 2021-11-08 00:00:12 [fabf407a6327e0fb941c7dbb43c901afda88837ca3c802eb2a19a9fcd76f00d]
_auto_data-2021-11-08_00:00          Mon, 2021-11-08 00:00:24 [4de8d16d1e3c925b5ab1e57ada3c21b25dd2254518fa6dc8bfd8f22ec0d4f9e]
_auto_borg-2021-11-08_00:01          Mon, 2021-11-08 00:01:07 [133de98e8b107411150d32d96aeb8bf5811805d7cb8f1a73ceba059972f783e]
_auto_element-2021-11-08_00:01       Mon, 2021-11-08 00:01:16 [e39d1b48910d50ed4340b4dc046865ac06c5362a0a078f9dcea22c3b4eb58ce]
_auto_hextris-2021-11-08_00:01       Mon, 2021-11-08 00:01:32 [1508fdb2bf91de8dd275381ba28ee5fa0d9c86746179174b23a8981a3d2ba2d]
...

I’ll be adding the other servers with different target directories on homeYuno, and install Borg Backup with slightly different windows to spread the load (this one took, according to timestamps, ~6 minutes for ~600M of backup, so things are not that wild).

So far I’m happy with the outcome!

ljf · November 30, 2021, 11:52pm

New release for borg_ynh:

01/12/2021 - 1.1.16~ynh26
[enh] Add config panel in order to be able to change some backup settings, list archives and show backup status
[fix] Improve upgrade performance (by avoiding to rebuild borg each time)
[enh] Allow to exclude data multimedia backup

Melchisedech · December 1, 2021, 1:52pm

Hi, since the last update of borg (yesterday), I’ve got this error :

my.domain : Nov 30 21:26:15 : admin : /etc/sudoers.d/borg is owned by uid 992, should be 0 ; TTY=pts/0 ; PWD=/home/yunohost.transmission/completed ; USER=root ;

Am I alone in this case?
What can I do?

Thanks in advance

ljf · December 1, 2021, 2:02pm

chown root:root /etc/sudoers.d/borg

but you should not have this issue, as this line is in the script:

github.com

YunoHost-Apps/borg_ynh/blob/testing/scripts/upgrade#L121


      
          #=================================================
          ynh_script_progression --message="Setting up backup method..." --weight=1
          ynh_add_config --template="backup_method" --destination="/etc/yunohost/hooks.d/backup_method/05-${app}_app"
          chmod go=--- "/etc/yunohost/hooks.d/backup_method/05-${app}_app"
          
          ynh_add_config --template="backup-with-borg" --destination="/usr/local/bin/backup-with-$app"
          chmod u+x "/usr/local/bin/backup-with-$app"
          chown $app:$app "/usr/local/bin/backup-with-$app"
          
          ynh_add_config --template="sudoer" --destination="/etc/sudoers.d/$app"
          chown root:root "/etc/sudoers.d/$app"
          
          #=================================================
          # SETUP SYSTEMD
          #=================================================
          ynh_script_progression --message="Upgrading systemd configuration..." --weight=1
          
          # Create a dedicated systemd config
          ynh_add_systemd_config
          # Disable the service, this is to prevent the service from being triggered at boot time
          systemctl disable $app.service --quiet

Melchisedech · December 1, 2021, 2:03pm

Thanks, I’ll do that right away. That’s strange indeed…

Melchisedech · December 2, 2021, 8:26pm

I’ve seen this error in the twice a day mail I receive from Yunohost :

[ERROR] An issue was found for app Borg Backup App

This app’s installed version still uses some super-old deprecated packaging practices. You should really consider upgrading it.

Is that an expected behavior?