Getting my server to respond to me again
So, in the middle of today my server then seemed to be completely down, and many attempts at regaining access via SSH didn’t even establish a connection, I felt a little bit like in a lottery.
One wild goose chase and a lot of sweat later, I am glad to have found something that approximates a solution.
For anyone running into the same issue I did, after many frustrating hours where my VPS was barely responsive and seemed to constantly even shut me out, here is how I got it back on track:
After rebooting the server the hard way via my provider, the command systemctl --failed showed me that there were two things apparently failing:
UNIT LOAD ACTIVE SUB DESCRIPTION
● systemd-udev-trigger.service loaded failed failed Coldplug All udev Devices
● user@50465.service loaded failed failed User Manager for UID 50465
which was probably also the reason why before, I couldn’t even enter a reboot command or switch to the root user or anything of the like.
With systemctl status I could inquire some more, but was not learning anything useful. The first magic bullet command:
sudo systemctl reset-failed
After this command, my Yunohost-server started responding to me again without massive and weird delays. Yunohost was functional (even if Nextcloud still wasn’t).
Restoring Nextcloud to functionality
- I decided to remove the Nextcould app again. Removing worked without problems.
- I restored my original backup. This time, restoring the backup worked without problems, but: I was back seeing a familiar error: The
Internal Server Erroras described above and in many other threads. - Using the yunohost-webadmin-interface, I experimented putting Nextcloud in maintenance mode. I opened Nextcloud in the user interface, and in fact, instead of an ugly Internal Server Error message I was greeted by a beautiful cloud on blue background informing me of maintenance mode. So, Nextcloud seemed to be somehow intact?
- I started familiarising myself with the occ method of passing commands to Nextcloud via SSH. Seemed scary at first, but simply typing the overview command helped me in reading through and understanding what might help:
sudo -u nextcloud php8.2 --define apc.enable_cli=1 /var/www/nextcloud/occ - Discovered by chance the second magic bullet command:
cd /var/www/nextcloud/
sudo -u nextcloud php8.2 --define apc.enable_cli=1 occ maintenance:repair
That’s all it took! Nextcloud was functional again! Well, more or less, because I started running into the issues in this thread.
Ongoing problem with user@50465.service
Because of that, I also tweaked the php.ini configuration files a bit and restarted php8.2-fpm.service a couple of times. The first few times, I kept getting the same issue immediately after the service restart:
systemctl --failed would yield the following:
UNIT LOAD ACTIVE SUB DESCRIPTION
● user@50465.service loaded failed failed User Manager for UID 50465
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
1 loaded units listed.
systemctl reset-failed would fix it again, and then it was gone.
This repeated for a couple of times. Now, the last 3 service restarts, it disappeared. I don’t know why precisely, but the error is hopefully gone for good.
So far, from me. My setup is still shaky but I hope it will hold the next couple of days. I have no idea though when and how it would be safe for me to try updating my Nextcloud app again? If any of the issues re-emerge, I will reply here again. In the meantime I hope no one else runs into the same issues I did, and if you do, I hope that this thread will be a good help to you!
Best regards and a joyous week.