"Invalid version" after installing custom software using pip

nschiwy · December 23, 2023, 10:01pm

My YunoHost server

Hardware: VPS bought online
YunoHost version: 11.2.8.2
I have access to my server : Through SSH & through the webadmin
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : Only “tweaking” was that I added duplicity through a PIP package to do backups - this should be entirely outside of Yunohost

Description of my issue

A few days ago I started trying to figure out a good backup solution for my yunohost server. I already use duplicity on one of my other (standalone nextcloud) server and felt comfortable with it doing what I needed. My current VM is pretty small and I don’t have any excess space to create local archives. I wanted to be able to just kick the backups directly to another location.

Ever since I apparently made changes to the python install, I started getting error emails from my server saying

/etc/cron.daily/yunohost-fetch-apps-catalog:
Failed to read info for mastodon : Invalid version: ‘4.2.1~ynh1’

Now I’m seeing that in the “Applications” page in the yunohost admin portal it says that there are no applications installed. That said, Mastodon (my only installed application) has been running fine and normally the entire time. A big part of this was wanting decent backups before I embarked on an upgrade to the application.

After some poking around I found that Mastodon is still listed as an attached application in the domains page and was able to get this full error:

Error: "500"
Action: "GET" /yunohost/api/apps/mastodon?full&locale=en
Error message:
Unexpected server error
Traceback

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/moulinette/interfaces/api.py", line 453, in process
    ret = self.actionsmap.process(arguments, timeout=30, route=_route)
  File "/usr/lib/python3/dist-packages/moulinette/actionsmap.py", line 580, in process
    return func(**arguments)
  File "/usr/lib/python3/dist-packages/yunohost/app.py", line 159, in app_info
    ret["upgradable"] = _app_upgradable({**ret, "from_catalog": from_catalog})
  File "/usr/lib/python3/dist-packages/yunohost/app.py", line 258, in _app_upgradable
    installed_version = version.parse(app_infos.get("version", "0~ynh0"))
  File "/usr/local/lib/python3.9/dist-packages/packaging/version.py", line 54, in parse
    return Version(version)
  File "/usr/local/lib/python3.9/dist-packages/packaging/version.py", line 200, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '4.2.1~ynh1'

I am also open to doing the upgrades manually and moving to the main branch of Mastodon but I am getting git errors when I try to go that direction and I figured it might make the most sense to start here in case I accidentally caused more issues. Any pointers would be greatly appreciated. I’m also happy to post logs if that would be helpful but I wasn’t sure which yunopaste to actually paste.

Thanks in advance!

wbk · December 24, 2023, 10:31am

Hi nschiwy,

Welcome to the forums!

Ironic that installing a backup solution seems related to breaking the thing you want to back up

I did not know about duplicity, it looks nice. I also don’t recognize the error you receive. “moulinette” is part of Yunohost core. It is, as you guessed, Python based.

Duplicity’s installation says that files are put in 'the default prefix /usr:

The default prefix is /usr, so files are put in /usr/bin,
/usr/share/man/, etc. An alternate prefix can be specified
using the --prefix= option.

I think this has to do with Pyothons venv`s, which I still have to figure out.

I don’t know if this is also the case when installing via pip though. I did not want to take the risk to try it out for now

What I did learn, is that pip has a check command:

$ pip check
pygobject 3.38.0 requires pycairo, which is not installed.
$

is what it does on my Yunohost and on that of one of my daughters, neither of which has problems, so missing pycairo does not seem detrimental to Yunohost general functionality.

Does it give you any other results?

After that I used pip list | more to compare my installed package versions with those required by Duplicity. I am not sure whether all in the Duplicity requirements.txt are installed by default. I`ll list the (mis)match between those in the requirements and my installed packages, maybe you can do the same on your end.

'basic requirements` :
- none of the three are on my system
‘custom versions’ :
- chardet and urllib3 are installed, of a compatible version
- cryptography == 3.4.8 is quite strict; I got 3.3.2. This might break things
backend libraries
- most are not installed, with the exception of psutil (I got 5.8.0) and requests (I got 2.25.1)
testing libraries
- none of these are on my system
documentation
- none of these are on my system

Additionally, my Moulinette is version 11.2; pip show Moulinette has an empty ‘Requires’-line. I think that may be correct for PIP, but apt show moulinette lists some Python packages that don`t intersect Duplicity.

Having checked these, I realize you might have a quite different list of Python packages, as you already have Mastodon installed and not the apps I have installed.

Mastodon has 4.2.3 available in the catalog. Does yunohost app show mastodon give any result? And yunohost app upgrade mastodon?

nschiwy · December 24, 2023, 6:53pm

Hey wbk! I really appreciate your in-depth reply and dive into this. And yeah I know I was trying to be good lol

My result when I use

$ pip check
No broken requirements found.

Also it seems like my cryptography version is the same. This is my full list:

Package             Version
------------------- --------------
appdirs             1.4.4
attrs               20.3.0
b2sdk               1.29.0
beautifulsoup4      4.9.3
blinker             1.4
bottle              0.12.19
cached-property     1.5.2
certifi             2020.6.20
chardet             4.0.0
cloud-init          20.4.1
configobj           5.0.6
cryptography        3.3.2
dbus-python         1.2.16
defusedxml          0.6.0
distro-info         1.0
dns-lexicon         3.11.2
dnspython           2.0.0
duplicity           2.1.4
fail2ban            0.11.2
fasteners           0.19
freezegun           0.3.15
future              0.18.2
gevent              20.9.0
gevent-websocket    0.10.1
greenlet            0.4.17
html5lib            1.1
httplib2            0.18.1
idna                2.10
ifaddr              0.1.7
importlib-metadata  1.6.0
isodate             0.6.0
Jinja2              2.11.3
jsonpatch           1.25
jsonpointer         2.0
jsonschema          3.2.0
logfury             1.0.1
lxml                4.6.3
MarkupSafe          1.1.1
miniupnpc           2.2.1
more-itertools      4.2.0
Moulinette          11.2
oauthlib            3.1.0
packaging           23.2
pip                 20.3.4
prompt-toolkit      3.0.14
psutil              5.8.0
publicsuffix2       2.20191221
pyasn1              0.4.8
pyasn1-modules      0.2.1
pycurl              7.43.0.6
Pygments            2.7.1
pyinotify           0.9.6
PyJWT               1.7.1
pyOpenSSL           20.0.1
pyparsing           2.4.7
pyrsistent          0.15.5
PySimpleSOAP        1.16.2
python-apt          2.2.1
python-dateutil     2.8.1
python-debian       0.1.39
python-debianbts    3.1.0
python-dotenv       1.0.0
python-ldap         3.2.0
pytz                2021.1
PyYAML              5.3.1
reportbug           7.10.3+deb11u1
requests            2.25.1
requests-file       1.5.1
requests-toolbelt   0.9.1
setuptools          69.0.2
setuptools-scm      8.0.4
six                 1.16.0
soupsieve           2.2.1
systemd-python      234
tldextract          2.2.1
toml                0.10.1
tomli               2.0.1
tqdm                4.66.1
typing-extensions   4.9.0
unattended-upgrades 0.1
urllib3             1.26.5
wcwidth             0.1.9
webencodings        0.5.1
wheel               0.34.2
zeep                4.0.0
zeroconf            0.36.6
zipp                1.0.0
zope.event          4.4
zope.interface      5.2.0

Some additional data points:

$ pip show duplicity
Name: duplicity
Version: 2.1.4
Summary: Encrypted backup using rsync algorithm
Home-page: http://duplicity.us
Author: Ben Escoto <ben@emrose.org>
Author-email: ben@emrose.org
License: None
Location: /usr/local/lib/python3.9/dist-packages
Requires: fasteners, setuptools-scm

$ pip show moulinette
Name: Moulinette
Version: 11.2
Summary: Prototype interfaces quickly and easily
Home-page: https://yunohost.org
Author: Yunohost Team
Author-email: yunohost@yunohost.org
License: AGPL
Location: /usr/lib/python3/dist-packages
Requires: 
Required-by:

okay so this seems right…

sudo yunohost app info mastodon
description: Libre and federated social network
domain_path: tusky.town/
name: Mastodon
version: 4.2.1~ynh1

And then pretty much the same as the original error. I hope this is useful. I wish I was able to glean more from this myself

$ sudo yunohost app upgrade mastodon
Info: Now upgrading mastodon...
Traceback (most recent call last):
  File "/usr/bin/yunohost", line 77, in <module>
    yunohost.cli(
  File "/usr/lib/python3/dist-packages/yunohost/__init__.py", line 41, in cli
    ret = moulinette.cli(
  File "/usr/lib/python3/dist-packages/moulinette/__init__.py", line 110, in cli
    Cli(
  File "/usr/lib/python3/dist-packages/moulinette/interfaces/cli.py", line 503, in run
    ret = self.actionsmap.process(args, timeout=timeout)
  File "/usr/lib/python3/dist-packages/moulinette/actionsmap.py", line 580, in process
    return func(**arguments)
  File "/usr/lib/python3/dist-packages/yunohost/app.py", line 617, in app_upgrade
    app_dict = app_info(app_instance_name, full=True)
  File "/usr/lib/python3/dist-packages/yunohost/app.py", line 159, in app_info
    ret["upgradable"] = _app_upgradable({**ret, "from_catalog": from_catalog})
  File "/usr/lib/python3/dist-packages/yunohost/app.py", line 258, in _app_upgradable
    installed_version = version.parse(app_infos.get("version", "0~ynh0"))
  File "/usr/local/lib/python3.9/dist-packages/packaging/version.py", line 54, in parse
    return Version(version)
  File "/usr/local/lib/python3.9/dist-packages/packaging/version.py", line 200, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '4.2.1~ynh1'

wbk · December 24, 2023, 7:52pm

No, it does not give me a hint either.

Would you mind trying to install any other app, to see whether the error is in Mastodon or in Moulinette?

Aleks · December 25, 2023, 1:02am

This indicates that this lib is loaded from /usr/local/bin, which points to the fact that you installed some dependency in the global scope of the system using pip (instead of inside a venv), which in turns broke YunoHost because it’s not designed for that version of the lib …

nschiwy · December 25, 2023, 4:41pm

Thanks Aleks. I guess I’m gonna have to figure out how to undo all of this. Worst case, I do have a yunohost backup from a few weeks ago…

Aleks · December 25, 2023, 7:23pm

YunoHost backup are not images of the entire system and wont undo manual changes

However maybe try sudo pip3 uninstall packaging

nschiwy · December 25, 2023, 8:07pm

Oh I meant like if I had to do a fresh install but that actually fixed it, thank you so much!

Ancillary and fantastic irony of this whole thing is that I never even needed pip3 to get the duplicity backup to work.

Happy Holidays!

system · January 24, 2024, 8:07pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.