Python package: Upgrate/Restore failes on CI

jedie · January 4, 2022, 6:59pm

I have no idea what’s wrong here: YunoRunner for CI with my project GitHub - YunoHost-Apps/pyinventory_ynh: YunoHost app package for: https://github.com/jedie/PyInventory

grafik

It’s currently my PR: Update to PyInventory v0.13.0 by jedie · Pull Request #78 · YunoHost-Apps/pyinventory_ynh · GitHub

My first idea to fix this was this change:

grafik

But both variants results in the same error:

This kind of error happens on debian based systems, if python3-venv is not installed.
But python3-venv is in my pkg_dependencies list and will be (re-)installed in restore script.

So that error should not happen, isn’t it?

Any idea?

jedie · January 6, 2022, 6:18pm

Hm! It seems that this only failed on @tituspijean CI, but not on the official one, see: YunoRunner for CI

jedie · January 7, 2022, 6:06pm

Now it’s the same on the official CI, e.g.: YunoRunner for CI

grafik

Any idea?!? Think it’s a problem in CI, isn’t it?

I now created CI: Backup/Restore fails because of missing "python3-venv" package · Issue #1960 · YunoHost/issues · GitHub

tituspijean · January 8, 2022, 7:34pm

Since I have been pinged, I ran some tests. And I can very officially say that I have no idea what’s going on.

Side note jedie, you are posting your issue all over the place. I completely sympathize, but it’s making it difficult to see the full scope of your investigations. (e.g. you found out that there is no issue on the Bullseye CI).

ewilly · January 8, 2022, 8:47pm

Read that on debian it could be related to locales.
Did you try to add export LC_ALL=C before upgrading your venv ?

jedie · January 9, 2022, 10:14am

No. But why should this been done on a YunoHost package level? This is something that should be setup by YunoHost “core”, isn’t it?

The package works in the past and i never change something around the locales. So i think it’s not the problem.

It seems also that this is flaky. But debug flakyness is hard, because CI server are so slow See also my new Thread about this here: YunoHost package maintaining: Continuous integration? Trouble with CI

Yes, sorry about this. That’s the trouble with forum.yunohost.org vs. GitHub. I didn’t quite understand the YunoHost philosophy behind it

Back to topic: Yes, on Bullseye it works. But it maybe just because of flakiness?!?

emo · January 9, 2022, 11:50am

as a no coder, i am not on github. for that the forum is need and better understandable for me

ewilly · January 9, 2022, 2:07pm

To investiguate firstly at a package level before an escalation at a core level. Just an idea to help, your choice to test or not.

jedie · January 17, 2022, 4:03pm

Somebody a other idea?!?

locales is not the problem: Test venv upgrade with LC_ALL=C by ewilly · Pull Request #89 · YunoHost-Apps/pyinventory_ynh · GitHub

tituspijean · January 17, 2022, 5:09pm

I had a thought, not tested. Could it be that if fails to upgrade because… there’s nothing to upgrade?

–upgrade Upgrade the environment directory to use this version
of Python, assuming Python has been upgraded in-place.

Can you try making a version of the app that specifically uses e.g. Python 3.6, and try to upgrade it to Python 3.7? (But I would agree that if that’s the case, it is quite a cryptic error message)

jedie · January 17, 2022, 5:52pm

Think that’s not the problem. The problem is that python3-venv package is missing…

Interesting, now i have the same error on my test VirtualBox YunoHost: https://paste.yunohost.org/raw/yoqoponafi

EDIT: Think pyinventory-ynh-deps_0.13.0~ynh1_all.deb doesn’t contain python3-venv … How can i generate the same .deb file or get this temporary .deb file?!?

tituspijean · January 17, 2022, 6:07pm

That’s what I don’t understand: if you have listed the dependency in _common.sh and used the ynh_install_app_dependencies, it should be in there.

jedie · January 17, 2022, 6:11pm

Maybe a debian package problem? And bullseye fix it?

With Bugfix CI run by jedie · Pull Request #90 · YunoHost-Apps/pyinventory_ynh · GitHub i merge the “setup venv” code and split it up with without-pip and ensurepip, e.g.:

python3 -m venv --without-pip "${final_path}/venv"
...
ynh_exec_as $app $final_path/venv/bin/python3 -m ensurepip

In the end, it shouldn’t make a difference.
Because of slow CI, i will see the result maybe tomorrow here:

bullseye: YunoRunner for CI

stable seems not to be scheduled, yet: YunoRunner for CI

jedie · January 18, 2022, 8:47am

That’s the solution that worked: YunoRunner for CI

jedie · August 25, 2024, 4:45pm

Seems that i have a better solution, now: I replace --without-pip with --clear and add --upgrade-deps, e.g.:

    ynh_exec_as $app $py_app_version -m venv --clear --upgrade-deps "$data_dir/venv"

But this is now with Python 3.11 (manually installed)
See also: master <- testing by jedie · Pull Request #68 · YunoHost-Apps/django-for-runners_ynh · GitHub