Use newer Python than 3.9 ?!?

Beside my different Python based projects, i used Home Assistant

After the update today, i see a warning that Home Asistant >=2023.2 will need Python 3.10 and open Python 3.9 no longer supported for Home Asistant >=2023.2 · Issue #159 · YunoHost-Apps/homeassistant_ynh · GitHub for this.

The Question here is, how to handle this?

I see these options:

Has this topic ever been discussed anywhere?!?

I’m not the biggest fan of docker. But that is one of the the main reason to use it, isn’t it? :wink:
I think using Docker could be done without changing anything in the main system.
It would be interesting to try it out. Has anyone try this in the past? I jsut search for “docker” all projects Search · org:YunoHost-Apps docker · GitHub and found only GitHub - YunoHost-Apps/gollum_ynh: A simple, Git-powered wiki with a sweet API and local frontend, packaged with YunoHost and Docker. and it seems that docker usage was removed in Non-docker version by ketsapiwiq · Pull Request #1 · YunoHost-Apps/gollum_ynh · GitHub

I would also add the option “stick to whatever version is supported until Debian Bookworm”

The whole thing about “installing custom versions of python locally” is a pandora box. As far as I know you basically have to compile python locally, which significantly lengthens up the app’s install time on low-end device such as ARM boards

Docker is another pandora’s box. Sure we can have one app using Docker, but then we’ll have another, and another, and another, and we’ll ultimately end up with a fragmented app ecosystem with one portion using Docker, and the other not, for no specific reason. It doesn’t “change anything on the main system”, but devil is always in the detail :

  • Docker needs iptables rules and it is not clear how this interfere with Yunohost’s firewall.
  • Not to mention I’ve seen people in the past having apt/dpkg being broken because for some reason docker-ce upgrade would crash (because using some apps that were packaged using docker, as you found out - but there was other such as diaporadocker_ynh etc)
  • People say there is supposedly “no resource overhead in using Docker vs. classic install”, but soon enough you will need PHP-FPM process or a mysql/postgresql DB. Except many apps need those too, and having a commong PHP-FPM master process, or a common mysql/postgresql stack reduces the overhead … yet many advantages of Docker reside in having “everything docker”, including the DB, cf the DB volumes which store the actual data, and so on …
  • everything else we don’t know about - as I like to say : install stuff is usually stupidly easy. The real issue is the long-term maintenance and every stupid edge case you don’t know about yet and will only learn about along the way of “stabilization”.

With the said I’m not really “anti-Docker” or whatever … I use docker everyday for other projects and it’s awesome. YunoHost suffers from quite a lot of stability/upgrading issues because of trying to remain somewhat “low tech” with it’s “container-less” approach, such as what we see in the major debian upgrades and lack of guarantees that restoring an app from a backup will work. I would be glad if some project would explore building a Yunohost-like distro using the full power of the docker paradigm. But for it to be successful, it would also have to be really mindful of creating something simple that regular human being can use, something tech-savvy folks forget too quickly about (and even in the YunoHost we tend to loose our focus regularly…). And I expect there would be important challenges into keeping things lightweight enough to run on ARM boards (despite the claim that there’s no overhead using Docker, cf previous note) and hackable for people that know only a little bit of CLI.

Anyway, the other thing at stake in the low-tech vs. high-tech of adminsys is this frenesy to always seek having top-notch-up-to-date softwares and technology, sometimes defended by the argument of “security” (which you could also argue the other way around : you could expect that old, well-known software are safer than brand new stuff), and ultimately this goes a long way into complexifying everything. That goes also with the fact that upstream devs don’t really care about distribution (as in “making it effectively possible and easy for people to use your software”) beyond Docker (which alone is not an acceptable answer for distributing software to non-tech-savvy people). There would be a lot to say about how Debian packaging remained too complex and was not suited for webapp development and deployment and how this triggered the explosion of language-specific package managers with debatable behavior and policies … but I digress :wink:. My point being : Debian Stable is a pretty good baseline, and people can live with a 6-month/1-year old software for some time.

6 Likes

If you look at the code you will see that the app is able to build the needed python from source (see https://github.com/YunoHost-Apps/homeassistant_ynh/blob/master/scripts/_common.sh). It was already the case at the end of life of debian buster to propose a not too old version of home-assistant.
For the time no need to worry about it its just a caution for the version that would be shipped after February.

1 Like

Oh. Interesting! Then there is a solution for the problem. This part of code is usefull for all Python based projects.
Have you take a look to pyenv?

Not deeply… It’s perhaps a better way… I don’t have enough knowledge about the subject to understand what that imply for the whole system and how to manage it.

My projects still support Python 3.9… But only because of this old one in YunoHost. So i take another look on this topic…

Interesting thing is, that many projects used Python, see: GitHub Search: “org:YunoHost-Apps python path:/^scripts//”

Found two ones that using pyenv, but the bootstrap of this is much more code: GitHub search: “org:YunoHost-Apps github.com/pyenv”

The homeassistant solution is much smaller and also used by YunoHost-Apps/tandoor_ynh

The bad thing about this is IMHO, that you must specify the exactly version of Python, e.g: py_required_version=3.11.4

Would be great if only the main version must be specify e.g.: 3.10 or 3.11 and the last bugfix version will be installed.

And what’s about to build a solution in YunoHost core, so it’s usable for all projects?

No. Using Python 3.9 is not a huge deal, I don’t see what top-notch killer feature would be included in 3.10 or 3.11 that would make it worth it.

We use whatever is shipped in stock Debian.

If you get stucked because of limitations to stock Debian, then maybe contribute by joining the campaign to fix apps so they work on Bookworm.

Working on workarounds instead of helping with the actual issue/fix (the Bookworm transition) is only a waste of time, because then we have epic technical issues to debug because one way or another, these kind of magic tweaks always end up making random stuff explose (cf for example people installing stuff in global scope with pip … or anyway installing custom version of Python usually ends up in compiling locally, which takes an epic amount of time, which is not acceptable considering what YunoHost is trying to achieve) and then we (the support team) have to debug the mess

Yeah … please for the love of god stick to regular python venv, python -m venv venv. Anything else seems to be madness…

1 Like

Note: pyenv is not related to python -m venv :wink:
pyenv are scripts to install different Python Interpreter versions on any systems :wink:

Yes and no, i think :wink:
All of my related projects still work on Python 3.9… Only because of YunoHost :wink:

Other projects (homeassistant, tandoor, etc.) need a newer Python.

Migrating more quickly to any new Debian stable version helps in principle, of course.
However, Debian is not super fast, so even fast YunoHost updates won’t solve the problem completely, i think :wink:

That’s one of the reasons why some people use Docker or Fatpack etc. But this is a other topic.

The bottom line: We can stay with the current situation: Every package maintainer can stay with the default Python or use his own solution to install a newer one.
Or: We add a helper in the core to install a newer Python.

Or: people can invest energy into the Bullseye->Bookworm transition to help move forward with the general situation for everybody, instead of investing energy into their own specific app needs using their own custom trick and creating cyber-frankensteins

9 Likes

I start now to use https://github.com/YunoHost-Apps/homeassistant_ynh/blob/master/scripts/_common.sh too:

aaf605479a56f4109a3f7fee72505a75a7c578c6-350x186

2 Likes

How do you make sure you won’t break user systems when you install this app ?

I see no problem here.