Restic eats disk capacity of my main drive? But choosing to make a symbolic link from the YH Backup service to a second drive instead?! Now that's a long story but follow it to get te same ;-)

Support apps
1

Hardware: AMD64
YunoHost version: 4.3.6.3 (stable).
I have access to my server : Yes
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : yes
If yes, please explain: it’s behind a VPN exactly like: Homemade WireGuard VPN on a VPS server

Description:

I use restic for local backups on sdb but I found that it takes up the disk space of sda, why does it do this, and can I prevent this from happening?

2

I had a similar problem years ago (not with restic) and it was due to a default configuration and an error on my side.

What I remember is that even if you use a rpecific backup app, the default YunoHost batkup process is used first, then this backup is pushed to the app.

This backup contains many things, some generated files (database export for example), and also symlinks to real files (to avoid too much work).

This backup is, if I remember, in /home/yunohost.backup

There are 3 folders :
archives : where the default process puts the final archives, can be moved to another disk with symlink (maybe not the best idea, but it’s what I did)
premigration : no idea
tmp : I had huqe problems with this one when I first tried to move /home/yunohost.backup to another drive, so even if it temporary gets bigger, it should be emptied at the end of the process.

So in conclusion :
Wait for the end of the backup to check the used space.
Ajways remember that backup use space during the process, even if awesome work have been made to limit the used tmp space.
If lisk space is still used on the wnong disk, find where (and check if a backup on the right disk have been made)

3

Okay I would love to wait, but it makes the server useless and shutdown processes to save on resources, then mail fails cause there is no disk space left.

So making symlinks for those 3 folders to somewhere on the backup drive?

What I saw was that temp folders and files were created on the sdb drive. So how about that?

4

From what I remember, the tmp folder should NOT be moved (but I had this problem years ago, no idea of if it is still existing).
The system created stuff inside, and was not able to remove it, which failed each other backup.

So the backup process was using 100% of your main drive ?

Can you give some details about the disk size, space usually used, folder that fulled your drive, and stuff that can help understand ?

5

What do you mean “takes up disk space on /dev/sda”?

Can you show us the commands you are running?

I have a daily Restic backup going to local RAID and to an offsite RAID with Wireguard and my server doesn’t experience these problems.

6

That it takes up disk space of the MAIN YH server disk aka SDA (like the C drive is on Windows, is SDA usually the main disk of Linux).

Or if I run the backup command, after a few hours SDA is out of space.

This is the command i use,

sudo restic backup / --verbose --tag yunohost -r /mnt/backup/

As you can see; /mnt/backup is the mounting point of /dev/sdb (the second disk drive is usually SDB, which would be like the D drive on Windows).

Hope that I made it a little more clear for you!

7

holy moly. you are using restic to backup the whole disk? i can see why the disk might get full. i am not sure how much data is on your drive but your drive must not be big enough for the swap/cache files it needs. if you are really set on doing this sort of backup, i would recommend checking out the restic forums. somebody might be able to help you there.

also, i am not sure restic is the right tool for this type of backup. it seems like you need a snapshot type of tool. you might have a corrupt backup if files change while restic is backing them up. i’m not to familiar with full system snapshots but maybe timeshift could work?

1 Like
8

Or maybe use an app which use the YunoHost backups mechanisms, usually most of the system do not need to be backuped, only apps data and conf (and user files/mails).

1 Like
9

So if I use restic, which exact folder pad’s should I back up

10

Yes I did that on purpose.

Almost 1 TiB, since I’ve got a very huge website. But the drive is also 1 TiB both are actually the main drive and the backup drive.

Ah that can be a issue indeed, timeshift sounds also legit.

Now I do have that other tool (forgot its name) to clone the drive, but that’s an offline tool, therefore I need to boot to that system to do a backup, which means that the server would be offline. The best solution would be a snapshot based back up for which I do not have to take the server ‘offline’.
So i thought restic would be great but not, then which are better to use? Timeshift and?

11

if you just want to backup Yunohost stuff, use the built in backup.

check out this page for more information

1 Like
12

probably need to do a deep-dive on the internet. check out rsync, rsnapshot or rclone.

for me, i just use restic to backup my nextcloud data directory, and my home directory. if i lose my yunohost server, i would reinstall yunohost and just send my data back into my nextcloud.

13

I have absolutely no idea, and it will depends on the apps.
That’s why I use YunoHost’s backup mechanisms and backup apps using them.

On my side I’m using borg backup and in the conf panel I can choose “server conf”, “user data/mail” and/or “apps” (can be “all”, a list to include or a list to exclude).

And I do not care what to backup, it’s all determined by the apps mainteners :slightly_smiling_face:

(And I praise them every days :star_struck:)

1 Like
14

But my issue is that I cannot back up the ‘big’ site using the YH mechanism. Unless there is a way to change the backup folder to the other drive maybe? Then i don’t have to use restic at all…

15

i don’t remember how to use “yunohost backup create” to make a backup that isn’t on the main drive. i am not sure it is possible.

you could make a symbolic link from /home/yunohost.backup/archives to your other hard drive.

look up “ln -s” or “symbolic links”.

2 Likes
16

@Aleks can you confirm this please? Just to be sure.

17

Yes …

1 Like
18

Purr

Command used:
ln -s /mnt/backup /home/yunohost.backup/archives

One more thing… Can this also be done for all the other backup folders or for /home/yunohost.backup as a whole?

And also;

Info: Creating a backup archive from the collected files...
Info: The archive will contain about 452.0GiB of data.
Info: The operation 'Create a backup archive' could not be completed. Please share the full log of this operation using the command 'yunohost log share 20220809-182949-backup_create' to get help
Error: Not enough free space on '/home/yunohost.backup/archives

more on this: https://paste.yunohost.org/raw/ricexiduse

I don’t get, it look:

root@ecz-srv-01:~# fdisk -l
Disk /dev/nvme0n1: 477 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: WDC PC SN520 SDAPNUW-512G-1006          
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 5B68EDCC-FFD5-4670-AC26-A6F78075A12B

Device             Start        End   Sectors   Size Type
/dev/nvme0n1p1      2048     206847    204800   100M EFI System
/dev/nvme0n1p2    206848     239615     32768    16M Microsoft reserved
/dev/nvme0n1p3    239616  999176191 998936576 476.3G Microsoft basic data
/dev/nvme0n1p4 999178240 1000214527   1036288   506M Windows recovery environment


Disk /dev/sda: 894.3 GiB, 960197124096 bytes, 1875385008 sectors
Disk model: TEAM T253X5960G 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 1E199B49-C4BF-4991-9C81-CB919500E9F7

Device          Start        End    Sectors   Size Type
/dev/sda1        2048    1050623    1048576   512M EFI System
/dev/sda2     1050624 1873385471 1872334848 892.8G Linux filesystem
/dev/sda3  1873385472 1875384319    1998848   976M Linux swap


Disk /dev/sdb: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: WDC WDS100T2G0A-
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x3947100b

Device     Boot Start        End    Sectors   Size Id Type
/dev/sdb1        2048 1953525167 1953523120 931.5G 83 Linux

There must be plenty of space :woozy_face:

19

Hmm ok. From your log.

2022-08-09 20:31:44,840: INFO - The archive will contain about 452.0GiB of data.
2022-08-09 20:31:44,840: DEBUG - Creating the backup TAR archive...
2022-08-09 20:31:44,840: DEBUG - Not enough space at /home/yunohost.backup/archives (free: 181394104320 / needed: 485303156881)

Says you only have 181 GB free but it needs 485 GB.

Please post the output of:

df -H

also these commands:

cd /home/yunohost.backup/
ls -la
20 
Filesystem      Size  Used Avail Use% Mounted on
udev             16G     0   16G   0% /dev
tmpfs           3.2G  221M  3.0G   7% /run
/dev/sda2       943G  713G  183G  80% /
tmpfs            16G   29k   16G   1% /dev/shm
tmpfs           5.3M     0  5.3M   0% /run/lock
tmpfs            16G     0   16G   0% /sys/fs/cgroup
/dev/sda1       536M  3.5M  533M   1% /boot/efi
tmpfs           3.2G     0  3.2G   0% /run/user/0

total 16
drwxr-x---  4 admin root 4096 Aug  9 20:28 .
drwxr-xr-x 40 root  root 4096 Jul 10 12:19 ..
lrwxrwxrwx  1 root  root   11 Aug  9 20:28 archives -> /mnt/backup
drwxr-xr-x  2 root  root 4096 Jun  1  2021 premigration
drwxr-xr-x  2 root  root 4096 Aug  9 20:31 tmp