Lemmy magick temp file issue fills up hard drive

What app is this about, and its version: Lemmy 0.19.11~ynh3
What YunoHost version are you running: 12.1.28
What type of hardware are you using: VPS bought online

Describe your issue

Hello, I installed Lemmy recently on Oct 13, and everything went as expected. I had Lemmy up and running. By Oct 14 end of day, I realized that the VPS 80 GB hard drive space was around 75% capacity and I decided to figure it out next day. Over night, the hard drive filled to maximum capacity, and Lemmy’s Postgres service shutdown.

I looked for solutions and found the below links. I simply found the temporary files mentioned and removed them all - recovering 60GB of space. I restarted Postgres, and Lemmy started normally.

I expect temp files and I hope that they will clear normally. Within 1.5 hours, the temp files returned at about 9GB. I checked the logs for lemmy-pict-rs and saw a number of warnings. I’ll include the Yunopaste logs here.

Share relevant logs or error messages

By way of an update, after manually deleting the contents of /home/yunohost.app/lemmy/magick, Lemmy continues to accumulate these temp files. Without intervention, after 24 hours, the folder has filled again to 60GB.

I’ll manually remove the temp files again.

I did restart the VPS hoping that whatever process would resume, but I don’t think that changed anything.

From the links you provided, it seems that the source of issues is pictrs being killed due to out of memory. Do you see memory issues on your VPS?
I can’t really interpret your logs but there is a trace showing an issue of some sort… Which could explain the /tmp not being cleared…

1 Like

Could be related

ImageMagick policy should be adapted to your server

1 Like

I’m not sure if I do have memory issues on the VPS.

free -h simply confirms that I’ve got 4 GB RAM and 4 GB swap. I looked over the RAM history and I don’t see anything maxing out. I do see a lot of activity that varies between 40% to 76% though.

Following on @jarod5001 I found the policy.xml file at

/var/www/lemmy/pict-rs/magic.appimageextract/usr/etc/ImageMagick-7

The original git-hub solution to set height and width to “8KP” appears to already be default. I went over the other suggested templates suggested by ImageMagick, and simply amended height and width to match the “websafe” policy of “4KP” instead. There are a lot of options to choose from, and I’m still a layperson here so I wasn’t sure what policy options are appropriate for a Lemmy instance.

Just an update. After looking over the policy settings yesterday, I found a policy evaluator tool that was kindly provided with the link below.

I looked over the policy and I realize that the default is that all options are commented out. I’ve adopted most of the starter, restrictive policy suggestions from doyensec for now. I’ll see if this works.

Here’s another look at the log coming from lemmy-pict-rs.

EDIT: Just by way of update, since properly configuring ImageMagick’s policy file by using doyensec’s recommendations as a guide, the Magick temp files are clearing out and no longer pile up. I consider this issue resolved.

As a follow up question though, one last option that I did not implement from doyensec is the temporary path option. Should I enable this one, and can I simply direct it to /home/yunohost.app/lemmy/magick ?

  <policy domain="resource" name="temporary-path" value="/mnt/magick-conversions-with-restrictive-permissions"/> <!-- the location should only be accessible to the low-privileged user running ImageMagick -->