[solved] Synapse Update failed, restore not possible

[chmeyer]

Hello there,

I ran into trouble while updating synapse 1.71.0~ynh1.

My YunoHost server

Hardware: VM bought online
YunoHost version: 11.0.10.2
I have access to my server : Through SSH | through the webadmin
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : no

Description of my issue

I clicked to update my YNH and after a while I noticed synapse is not running.
It even is not listed in the installed apps any more:

$ sudo yunohost app list | grep -i syna
$

$ systemctl status matrix-synapse.service
● matrix-synapse.service
     Loaded: not-found (Reason: Unit matrix-synapse.service not found.)
     Active: failed (Result: exit-code) since Mon 2023-01-02 10:19:15 CET; 6h ago
        CPU: 0

I found some tips about modifying the archive, but this does not seem to work for me:

$ sudo tar --delete --file synapse-pre-upgrade2.tar apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/attrs-20.3.0.dist-info/
tar: apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/attrs-20.3.0.dist-info: Not found in archive
tar: Exiting with failure status due to previous errors

My logfile https://paste.yunohost.org/raw/depisiqede says something about python lib problems.

I’m now at the point of deinstalling it, but I doubt if that is the right way to restore my server. Latest update completely broke the server and the ability to restore... · Issue #352 · YunoHost-Apps/synapse_ynh · GitHub

Can anyone please give me a hint on how to keep my settings and restoring synapse?

Thanks a lot
Christian

Update:
I don’t think that it is relevant, but to be honest I must tell that I previously updated the configfile /etc/matrix-synapse/homeserver.yaml by hand, because of a password problem:

The synapse user password is no longer the yunohost user password

Ah, okay. I fixed my issue. Hope that helps you, too: I only was focussed on /etc/yunohost/apps/synapse/conf/homeserver.yaml. After changing the bind_dn in /etc/matrix-synapse/homeserver.yaml, too (and a synapse restart) my password is recognized, again. It now looks like this: bind_dn: "uid=synapse,ou=users,dc=yunohost,dc=org" The only thing I still don’t understand is, why Yunohost doesn’t update that file automatically when updating the Synapse-App to 1.67.0~ynh1. … Christian

[chmeyer]

So not lucky, but I found a way to get it back to work:

First I tried to understand and fix the error messages:

setuptools-rust 1.5.2 requires setuptools>=62.4, but you have setuptools 60.8.2 which is incompatible.
AttributeError: module 'lib' has no attribute 'OpenSSL_add_all_algorithms'

I found an elderly problem with similar structure:

AttributeError: module 'jinja2' has no attribute 'Markup'

and then tried:

$ sudo pip install pip --upgrade
$ sudo pip install pyopenssl --upgrade
$ sudo pip install setuptools --upgrade
$ sudo pip install setuptools-rust --upgrade

but that only resulted in

WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

So, I’m not familiar with those virtual environments and it didn’t change anything.

The solution was to adopt the path to the archive to the correct file:

$ sudo tar --delete --file /home/yunohost.backup/archives/synapse-pre-upgrade1.tar apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/pyOpenSSL-22.0.0.dist-info/
$ sudo tar --delete --file /home/yunohost.backup/archives/synapse-pre-upgrade1.tar apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/OpenSSL/

So I lost my crypto keys and my verification (and all the encrypted messages), but at least the contacts seems to be there.
Christian

[Roborator]

Hello,

I had exactly the same Problem.
thanks a lot for your solution, that helps me a lot.

I have a question to you. Did you find a way to update pip, pyopenssl, setuptools and rust?
Did you find a way to update the synapse server?

1000 Thanks a again for your solution!!!

Greetz Robo

[chmeyer]

Hi,

no, I had no luck with that.
Obviously there is a “virtual environment” for each Yunohost-App.
Updating systems pip (and other programs) does not affect this virtual environment.

Since I got my Synapse back (and updating it after this worked, too), I lost interest in figuring out how to update this virtual environment.
I guess, YNH will update it, when everything is ready to do so.

Glad to hear that it helped for you, too.

When I understand you correctly, then you’re still looking for a way to update Synapse.
Just try to update it the usual YNH-way with the web-GUI. For me this worked after the procedure described above, even with the old environment.

Christian

[Clemi]

Hello,

i had the same Problem. To bad this exists.
Thanks for sharing your solution!

Edit: If you got your security-key backuped, you dont even lose any messages or verifications.

Clemi

[Roborator]

Hello Christian,

yes I still look for a way to update my Synapse. I had tried it the over the web-GUI but it does not work.
Which procedure do you mean?

[MayeulC]

@Roborator

Which procedure do you mean?

That procedure:

$ cd /home/yunohost.backup/archives/
$ ls -l synapse-pre-upgrade* # This is to double check the name of the backup, adjust the file name accordingly later, and take the latest
$ cp synapse-pre-upgrade1.tar synapse-pre-upgrade1.tar.bak # Make a backup of the existing backup before modifying, just in case
$ sudo tar --delete --file synapse-pre-upgrade1.tar apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/pyOpenSSL-22.0.0.dist-info/ apps/synapse/backup/opt/yunohost/matrix-synapse/lib/python3.9/site-packages/OpenSSL/
$ yunohost backup restore synapse-pre-upgrade1 --apps synapse # start restoring the backup, it took around two hours for my big DB on a ssd

(slightlyt adjusted from the one quoted above: I merged the two tar commands in one, and tried to make the process clearer)

@chmeyer I think you should be able to update the synapse environment by doing:

sudo -u matrix-synapse /opt/yunohost/matrix-synapse/bin/python -m pip  list

Or by activating the environment from the dedicated user’s home directory:

sudo -u matrix-synapse bash # be careful about multi-instance, also not completely sure it works when executing as that user
source ~/bin/activate
pip list

(of course, replace list by your upgrade commands)

[Roborator]

Hello,

thank you for your effort!

I have tried again to update the synapse Server and it works now.

Greetz
Robo

[Roborator]

Hello again,

my synapse runs now again, but I see now a few little errors on the element now. May be you can help me again to update the pip, pyopenssl, setuptools and setuptools-rust. I think when I done this it should be run without errors, right?

@MayeulC

sudo -u matrix-synapse /opt/yunohost/matrix-synapse/bin/python -m pip lis

this list the packages ok but which are the next steps to update the pip and so on?

Have a nice day.

Greetz
Robin

[MayeulC]

Hello Robin,

Could you be a bit more specific with the issues you are encountering? I am not sure manually upgrading packages would solve that.