No disk space left on device

Support
1

What type of hardware are you using: VPS bought online
What YunoHost version are you running: 12.0.17
How are you able to access your server: SSH
Are you in a special context or did you perform specific tweaking on your YunoHost instance ?: no

Describe your issue

It looks dramatic: no Yunohost Server App is working: No Nextcloud app is working, no Matrix client can connect.. Yunohost diagnosis showed 0 Bytes (0%) free disk space on /.

The last actions according to the list of logs have been:

  • updating system packages
  • nextcloud configuration. Error log is included below.

SSH console is OK. What can I do to free some disk space , e.g.cache?

Thank you in advance!

Share relevant logs or error messages

https://paste.yunohost.org/raw/otusakatap

df -H
output:
Filesystem Size Used Avail
Use% Mounted on
udev 4.2G 0 4.2G
0% /dev
tmpfs 833M 84M 750M
11% /run
/dev/sda3 164G 164G 0
100% /
tmpfs 4.2G 0 4.2G
0% /dev/shm
tmpfs 5.3M 0 5.3M
0% /run/lock
/dev/sda2 1.1G 91M 860M
10% /boot
46.38.248.210:/voln353211a2 269G 41M 269G
1% /mnt/plusdisk
46.38.248.210:/voln353211a1 269G 85M 269G
1% /mnt/storagespace
46.38.248.210:/voln353211a1/home/yunohost.backup 269G 85M 269G
1% /home/yunohost.backup
tmpfs 833M 0 833M
0% /run/user/1007

2

sudo yunohost tools basic-space-cleanup && sudo apt install ncdu -y then sudo ncdu / to understand where to clean up

3

ncdu / shows those largest directories:
11.9 GiB /var
11.1 GiB /usr
7.8. GiB /home
…
Total disk usage: 36.1 GiB
Apparent sitze: 128.0 TiB

That (128TiB) looks rather strange.

4

The next run of ncdu / shows sizeof /var is 13.8GiB, sizeof file /var/log/syslog is 12.2 GiB.

If the syslog can be reduced, I might save some space.

5

Output of ncdu /, total usage: 39.8 GiB, which is still not much. syslog is 15.0GiB:

.  16.6 GiB [###########] /var
.  11.1 GiB [#######    ] /usr
.   7.8 GiB [#####      ] /home
    2.7 GiB [#          ] /opt
  871.4 MiB [           ] /go
.  86.4 MiB [           ] /boot
.  12.1 MiB [           ] /etc
.   7.7 MiB [           ] /tmp
. 676.0 KiB [           ] /run
  468.0 KiB [           ] /.wp-cli
.  44.0 KiB [           ] /mnt
!  16.0 KiB [           ] /lost+found
    8.0 KiB [           ] /media
e   4.0 KiB [           ] /srv
!   4.0 KiB [           ] /root
!   4.0 KiB [           ] /log
.   0.0   B [           ] /proc
.   0.0   B [           ] /sys
    0.0   B [           ] /dev
@   0.0   B [           ]  initrd.img.old
@   0.0   B [           ]  initrd.img
6

I forgot that ncdu must he used with root permissions. Here is the output of sudo ncdu /:

--- / -------------------------------------------------------------
   77.9 GiB [##########] /var
   58.1 GiB [#######   ] /home
   11.1 GiB [#         ] /usr
    2.7 GiB [          ] /opt
    1.7 GiB [          ] /root
  871.4 MiB [          ] /go
   86.4 MiB [          ] /boot
   77.6 MiB [          ] /run
   19.0 MiB [          ] /etc
    7.7 MiB [          ] /tmp
  468.0 KiB [          ] /.wp-cli
   44.0 KiB [          ] /mnt
   40.0 KiB [          ] /log
e  16.0 KiB [          ] /lost+found
    8.0 KiB [          ] /media
e   4.0 KiB [          ] /srv
.   0.0   B [          ] /proc
    0.0   B [          ] /sys
    0.0   B [          ] /dev
@   0.0   B [          ]  initrd.img.old
@   0.0   B [          ]  initrd.img
@   0.0   B [          ]  vmlinuz.old
@   0.0   B [          ]  vmlinuz
@   0.0   B [          ]  lib64
@   0.0   B [          ]  sbin
@   0.0   B [          ]  lib
 Total disk usage: 152.5 GiB  Apparent size: 128.1 TiB

The value of Apparent size: 128.1 TiB is very strange.

7

I am not sure which files or folders can be deleted without harm.

8

Did you try sudo yunohost tools basic-space-cleanup as suggested by @jarod5001 ?

9

Yes, I did that several times. There are stille 0 bytes free on /dev/sda3.

/var/log/syslog is 15.8GB.

10

Now I deleted one direcory of 1.7GB. df -h now shows this:

Filesystem                                        Size  Used Avail Use% Mounted on
/dev/sda3                                         153G  151G     0 100% /

Shouldn’t there be 2G free space?

11

2 minutes later, /var/log/syslog has risen from 15.8G to 16.3G. Any disk space I make free is filled up with syslog.

12

I looked into /var/log/syslog. There are plenty of messages like

Jul 09 09:33:39 MYDOMAIN.TLD rsyslogd[935914]: file '/var/log/syslog'[8] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2302.0 try https://www.rsyslog.com/e/2027 ]

So as soon as there is some free space, syslog is flooded with ‘disk full’ messages.

Is there a minimum of available free disk space?

13

sudo systemctl stop rsyslog

14

Thanks for your reply. Can I somehow reduce the size (or even delete) the syslog file?

15

First, I would try again sudo yunohost tools basic-space-cleanup

16

Don’t delete /var/log/syslog : you can clear its content with sudo truncate -s 0 /var/log/syslog. AFAIK archives like /var/log/syslog**.1** can be deleted but, before, I would check what is “spamming” /var/log/syslog and filing disk…

17

What parameters are set in the file /etc/logrotate.d/rsyslog?

18

A 16GB log file must be very difficult to read and display; to focus on a specific day and hour, you can do this : grep '^2025-07-09T09:00' /var/log/syslog
edit: or grep '^Jul 09 09:00' /var/log/syslog

19

Or sudo journalctl -f

20

We need to find 1) what is filling the log so quickly, and 2) what is allowing the log to reach such a size (16G ???) and grow indefinitely.
@matalb Could you check and return the output of systemctl status logrotate.timer ?