Mattermost restore failure

Vanitarium · December 12, 2022, 6:24pm

My YunoHost server

Machine ASUSTeK COMPUTER INC./H110T, BIOS 4212 07/24/2019
CPU Number of processors: 4
Intel(R) Core™ i3-6100T CPU @ 3.20GHz
0%
Intel(R) Core™ i3-6100T CPU @ 3.20GHz
0%
Intel(R) Core™ i3-6100T CPU @ 3.20GHz
1%
Intel(R) Core™ i3-6100T CPU @ 3.20GHz
0%
PCI Number of devices: 14
USB Number of devices: 4
I2C Number of devices: 8

YunoHost version: 11.0.10.2 (stable).
I have access to my server : Through SSH | through the webadmin |
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : no
If yes, please explain:

Description of my issue

There was a powercut.
The IP changed
The dynhost reset the IP at OVH
Mattermost service stopped
It did not restart after power returned
I uninstalled mattermost
I restored mattermost from my last backup from this morning
Mattermost restore fails at systemd service restart

Here is the log
https://paste.yunohost.org/raw/yakosinome
IMO this is a permission error. I am thinking of reinstalling and restoring the whole server. Which as I know is bad practice: but how can I restore permissions without the user “Mattermost” in place? He is not visible or known. Mattermost forums are not helpful because the folder and files are different in yunohost compared to standard Debian server!!!???
Please help because this is an important tool for my family!!
Yunohost is a great software system and I am very grateful for your work: that is why I have set up a regular donation!!
Thanks to anybody who can help with this…

Aleks · December 12, 2022, 8:20pm

Hmmmm I’m not sure exactly why the service seems to now require an argument to the start command … I guess you can try to patch it manually though it’s a bit boring to do:

download the backup archive on your machine somehow
open the archive and inside, edit the file that should be located somewhere like: apps/mattermost/backup/conf/systemd.service (NB: I dont know the exact patch)
- find the line looking like ExecStart=/var/www/mattermost/bin/mattermost
- add server at the end such that it looks like ExecStart=/var/www/mattermost/bin/mattermost server
upload the modified archive to your server (to be put inside /home/yunohost.backup/archives/)
retry the restore using that archive

Vanitarium · December 12, 2022, 8:31pm

Thank you. I am doing it now and will report back. Just for info I have tha backup archive on a hard disk on my iMac so I can easily manipulate it as you suggest and then restore it back with a different name and restore from that one keeping the original as it is. Thanks a lot for that helping hand. Will report back with details.

Vanitarium · December 13, 2022, 1:47pm

The path to that file is
/apps/mattermost/settings/conf/systemd.service
I could not work on the archive with macOS because the archiving apps for the mac are not very user friendly and do not allow mods of the archive without extracting it and re-compressing it.

I had to install a linux distro on my laptop, download the archive file to it, open it with archiver and modify the systemd.service file as you suggested. This took around 38 minutes due to the size of the archive:18GB. Then I upoaded it back on to the yunohost server at /home/yunohost.backup/archives and that was another 38 minutes. It was slow going and got very late last night!!
So, now I am ready to try it from the webadmin page
It did not work and yunopaste is not accepting any paste of the log…
Depressing life of an amateur admin!!!

Vanitarium · December 13, 2022, 2:00pm

Now it is working and here is the log:|
https://paste.yunohost.org/raw/ayitexefif

Well it looks as if it will have to be a complete reinstall of the server then and restore from backup.I hate systemd!!!
I shall wait for the kind yunohost helping developers @Aleks and then see what the next action will be!!
second and final attempt failed too
https://paste.yunohost.org/raw/deliyevuhu

Thanks for helping here @Aleks !!!
I am reinstalling now on a headless server using the script from

Vanitarium · December 14, 2022, 12:46pm

That script failed…
I did a frwesh install today and restored the whole server from the backup but it failed with mattermost again.
This is the log:
https://paste.yunohost.org/raw/uripojaluz
I am at the end of my knowledge and patience.
@Aleks any suggestions??
H E L P P L E A S E

Vanitarium · December 14, 2022, 3:33pm

And now the same happens for wikijs as per this log

https://paste.yunohost.org/raw/xuviqepiyi

Vanitarium · December 14, 2022, 4:18pm

I think I found the error … but I don’t know how to fix it:
nginx is listening on port 443
mattermost is also listening on 443
An error is returned in the log.
How can I make nginx listen on another port?
To test it I installed mattermost app and then made it listen on port 443 and it failed to start.

sudo ss -ltnp | grep :443
LISTEN 0      511          0.0.0.0:443        0.0.0.0:*    users:(("nginx",pid=10604,fd=16),("nginx",pid=10603,fd=16),("nginx",pid=10602,fd=16),("nginx",pid=10601,fd=16),("nginx",pid=10600,fd=16))
LISTEN 0      511             [::]:443           [::]:*    users:(("nginx",pid=10604,fd=17),("nginx",pid=10603,fd=17),("nginx",pid=10602,fd=17),("nginx",pid=10601,fd=17),("nginx",pid=10600,fd=17))

Aleks · December 17, 2022, 1:19pm

You definitely don’t want to change nginx port, and I doubt that mattermost listens on port 443 … what makes you think this …

If “an error is returned in the log” then can you share that error … I don’t know how many time we must say this but we can’t simply magicly help people without exact details on errors, ie BOTH the error message, and the context in which it appears

Apparently this is because the special user ‘svc_wikijs_ldap’ was not properly deleted during the app uninstall somehow … you can try removing it manually …

This log is only the log of the automatic app removal/cleanup after the failed restore … can you share the log of the actual restore attempt (Edit: hmokay i’m seeing it in the previous post)

Aleks · December 17, 2022, 1:23pm

Yes reinstalling the server is pointless because there’s nothing wrong with your server per se, this is an issue with the app (or rather one of its conf file - though again not sure how we ended up there)

Anyway,

I think the issue lies in the fact that the appropriate file to edit was in fact not /apps/mattermost/settings/conf/systemd.service (my bad) but something like /apps/mattermost/backup/etc/systemd/system/???/mattermost.service

Vanitarium · December 17, 2022, 1:37pm

I think this is the log you want to see

https://paste.yunohost.org/raw/runokiluci

Thank you for your time on this issue!

Vanitarium · December 17, 2022, 1:41pm

You definitely don’t want to change nginx port, and I doubt that mattermost listens on port 443 … what makes you think this …

Mattermost has this setting which I did activate in the installed app before:

Aleks · December 17, 2022, 1:42pm

Then forget about it, this is unrelated to Yunohost, this is for like a standalone setup where mattermost is handling everything itself …

Edit: or rather this is for when you’re not in a reverse proxy context. Yunohost handles the proxy stuff, just forget about it

Vanitarium · December 17, 2022, 1:43pm

Well, by my bad, I did modify that file and it didnt restore either. I don’t remember which log that was so I cannot retrieve it anymore.

system · January 1, 2023, 1:43pm

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.