🚧 YunoHost 12.0 beta (Bookworm)

[YunoHost]

Following the alpha-testing opened a few months ago, we are happy to announce that we are moving to the beta-testing stage for Bookworm !

We consider that it should now be okay to upgrade to or install a fresh Yunohost 12.0+ running on Bookworm for a production server if you are a tech-savvy person not afraid to debug stuff if needed. However, you should still remain careful, especially when running the Bullseye->Bookworm migration depending on the complexity of your setup (but doing so helps spotting issues !). Additionally, some apps are still known to not be Bookworm-ready yet.

Please keep in mind that this is a beta-testing and small issues or edge-cases are still expected, so be careful.

Major changes

• The user portal and SSO system have been split into three distinct pieces:
  • SSOwat only handling only the SSO/ACL logic (nginx lua middleware)
  • A new “portal API” (yunohost-portal-api) service delivering authentication cookies and allowing users to retrieve/update infos
  • A new portal front end (yunohost-portal)
• The install script has been reworked with a simpler flow and UI (instead of the old ncurses/whiptail)
• To base system is now more lightweight
  • MySQL/mariadb and PHP are not installed by default anymore (apps will explicitly depend on them hence everything gets installed only when needed)
  • Rspamd (antispam system) and Metronome (XMPP server) are not part of the core anymore. Instead, they are to become separate applications : Rspamd_ynh and Metronome_ynh

Regarding the new user portal

• The portal and authentication mechanism are now separated in terms of “main domain trees”. For example, foo.tld and sub.foo.tld share the same portal, but bar.tld and app.bar.tld share a second, different portal
• Users can only access a portal if they have access to at least one app of that “main domain tree”, or if they are admin
• The new portal include app logos, descriptions, and an optional ‘search engine’ bar.
  • FIXME: sooner or later, people will want to be able to change logos and descriptions
• The new portal’s appearance can be customized from the webadmin in the corresponding “main domain”'s setting (such as changing the portal name, logo, default theme, custom message, …). Each user can also pick the theme in their own settings page.
  • Note that if you themed the old portal, this theming won’t be magically migrated to the new portal
• New option to have the list of public apps as “default app”
• The old “yunohost tile overlay” doesn’t exist anymore
• It is technically possible to write a completely different and independent user portal (as long as it properly interacts with the portal API)
  • FIXME: generate proper documentation for the new portal API, maybe using swagger

Misc/technical changelog

Show/hide

• webadmin: rework cookie/session expiration mechanism. Cookies are now still valid after restarting the API (preventing clumsy disconnect during self-upgrades) and the cookie validity is automatically extended every time an API request is performed.
• mail: DKIM email signing is now done using opendkim instead of rspamd
• various compatibility tweakings for Bookworm
• regenconf: update nginx and dovecot ciphers according to Mozilla recommendation
• regenconf: update fail2ban config
• configpanels: refactor to use pydantic for more typing and consistency, add proper autogenerated doc
• apps: Yarn third-party repo is now available by default in apt config just like Sury, no need for an extra apt resource thingy
• legacy cleanups:
  • apps: drop support for unused --dedicated_service in php helpers
  • apps: drop support for legacy LUA-style regexes in permission urls
  • apps: do not auto-patch old PHP versions anymore because Sury is enabled by default and allows the install of arbitrary PHP versions
  • apps: drop support of the legacy unprotected_uris, redirected_url and similar settings
  • cli: drop support for legacy firstname/lastname args in yunohost user create
  • cli: drop legacy yunohost domain dns-conf and yunohost domain cert-status, cert-install, cert-renew
• perf: minimize regen-conf calls to yunohost settings get, and other misc lazy-loading optimizations
• quality: simplify the logging mess
• quality: rework ci tests workflow

What to test ?

Generally-speaking, you are encouraged to test everything you’d do on your YunoHost server, such as installing and using your favorite apps, browsing the webadmin, checking the diagnosis, installing certificates, checking emails, …

More specifically we are looking for feedback regarding:

• The new user portal, both in terms of “does it work as you’d expect” but also in terms of “how do you like the new UI/UX”
  • especially if you have several “domain trees”
• The SSO layer: are you correctly logged in your favorite apps when logged on YunoHost’s portal? What about external desktop/mobile clients?
• Validating that email DKIM signing is still working properly (for example using https://mail-tester.com)
• The new Rspamd app and Metronome app if these are things you’d use

Pre-installed images

Testing Bookworm ISOs are available in https://build.yunohost.org/ for arm64 and i386

(FIXME: still need to work on a fresh RPi image)

Cloud-init image

We have build a cloud-init yunohost image. So if you have a proxmox (or know a vps provider who support cloud-init), we will be happy if you try to create a virtual machine with it.

The cloud-init image is here: https://image-builds.yunohost.org/

Installing a fresh YunoHost on top of a fresh Debian 12/Bookworm

1. Obtain a brand new server (such as a VPS online, a local virtual machine, or a development LXC)
2. Preinstall your server with Debian Bookworm
3. Then install Yunohost 12.x with :

$ curl https://install.yunohost.org/bookworm | bash -s -- -d testing
# You will get a disclaimer intended to make sure what you're doing. 
# Read it and follow the instructions.

Upgrading an existing YunoHost 11.x/Bullseye server to 12/Bookworm

Latest versions of YunoHost 11.2 already ships a “hidden” migration that allows to upgrade to YunoHost 12/Bookworm.

Before going through this process, we reiterate that ideally, you should have a way to entirely rollback your server before proceeding with the upgrade. That way, if you spot issues, we’ll be able to provide a fix then validate that the fix works by re-running the upgrade from the same starting point.

1. Switch to the “testing” : curl https://install.yunohost.org/switchtoTesting | bash (this is needed because later, Yunohost 12 is only shipped through the testing channel)

2. Enable the “hidden” migration: sudo mv /usr/lib/python3/dist-packages/yunohost/migrations/0027_migrate_to_bookworm.py{.disabled,}

3. In the webadmin, under Tools > Migrations, you should now see an available migration to upgrade to Bookworm. Read the disclaimer and start the migration.

4. … be patient, this will take a while. But try to stay attentive to what’s going on. Share the detailed log if anything that goes wrong.

5. Ideally after the upgrade, test that everything (e.g. apps installed) still works as expected.

[Lapineige]

@YunoHost

Rspamd (antispam system) […] not part of the core anymore. Instead, they are to become separate applications : Rspamd_ynh

Would you recommend to install it anyway in any case ?
Is there any significant drawback of this antispam tool ?

[Aleks]

I’d say it takes a significant amount of RAM for what it does … and it’s not installable on some specific setup / hardware because libhyperscan (the underlying piece of software rspamd leverages) requires a specific processor capability (i don’t remember which one)

And of course it feels like many people don’t really care about receiving emails on their YunoHost accounts, or their email address or domain is not known by spammers, so it might be unecessary to have rspamd in that case

[Tho]

I’m now on Bookworm yup yup

Thanks to all for the great work!

Some little issues while upgrading:

• I had an issue with old wireguard config which was resolved by apt remove wireguard-dkms (thanks @tituspijean).
• I then could make the migration, but the process didn’t update after 0.0% Installing lua-bitop in the webadmin. Looking by cli, the process seemed to have finished but with some errors and the yunohost-api always failed. Cfr. migration logs: https://paste.yunohost.org/raw/wiqovecelu
• I rebooted to update the kernel (don’t know if it was useful)
• I manually did a apt full-upgrade because python and yunohost were not yet updated.
• It worked, and delete_xmpp and postgresql migration succeeded
• I had to manually run python-venv migration (“Migration 0030_rebuild_python_venv_in_bookworm has to be run manually. Please go to Tools → Migrations on the webadmin page, or run yunohost tools migrations run.”) which did not work for some apps. I’ve force upgraded them and all is good now! https://paste.yunohost.org/raw/ekiyovired

[rodinux]

Well, I am afraid with borg not avalaible for my home Yunohost… Perhaps I will try on another instance soon before…

[deljones]

Currently testing 12.0.1+202407262100 (unstable). So far all is good, with Nextcloud, Snappymail, and BookStack all working.

A small detail. I have it on a brand new VPS with NO port 25 open (yes I know SmappyMail is installed, but it’s not being used. It installed correctly that’s the point) I have to request the port to be opened, which the provider does, but less than 24 hours since the server has been up I am being threatened with suspension because of the dreaded Sendscore .com. Again.

I have asked for evidence of the SPAM accusation. We shall see.

Other than that I can still use the server and I am exploring as much as I can

A very job from the team as usual

Dj

[Kayou]

Don’t worry to much about this SendersScore alert, it will soon be removed: fix: Remove SenderScore from the dnsbl_list.yml file by milouse · Pull Request #1918 · YunoHost/yunohost · GitHub

[tufek]

yolo RPI4 migration here,
Just an issue with gcc-8-base package which prevented the migration similarly to this post

Migration to 11 won't start : "libc6-dev : Breaks: libgcc-8-dev"

I’m affected too: Les paquets suivants contiennent des dépendances non satisfaites : Info : + libc6-dev : Casse: libgcc-8-dev (< 8.4.0-2~) mais 8.3.0-6+rpi1 devra être installé Do you think the system is safe now that the migration aborted after marking some yunohost packages on hold and ?

Just removed it and proceeds to smooth migration with nextcloud and wireguard apps.

Great job everybody!

[tufek]

On my digitalocean vps there is some issues though. Seems to miss strptime python module but the admin panel doesn’t show any hint for the full pastebin log command.

Here are the error message I got:

Erreur: "500" Internal Server Error

Action: "PUT" /yunohost/api/migrations?accept_disclaimer

Message d’erreur :

Erreur serveur inattendue

Retraçage

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/yunohost/tools.py", line 786, in tools_migrations_run
    migration.run()
  File "/usr/lib/python3/dist-packages/yunohost/migrations/0027_migrate_to_bookworm.py", line 195, in run
    regen_conf(names=["nsswitch"], force=True)
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 483, in func_wrapper
    result = func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/yunohost/regenconf.py", line 424, in regen_conf
    operation_logger.success()
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 744, in success
    self.close()
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 782, in close
    desc = _get_description_from_name(self.name)
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 870, in _get_description_from_name
    datetime.strptime(" ".join(parts[:2]), "%Y%m%d %H%M%S")
ModuleNotFoundError: No module named '_strptime'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/moulinette/interfaces/api.py", line 450, in process
    ret = self.actionsmap.process(arguments, timeout=30, route=_route)
  File "/usr/lib/python3/dist-packages/moulinette/actionsmap.py", line 574, in process
    return func(**arguments)
  File "/usr/lib/python3/dist-packages/yunohost/tools.py", line 794, in tools_migrations_run
    operation_logger.error(msg)
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 750, in error
    return self.close(error)
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 782, in close
    desc = _get_description_from_name(self.name)
  File "/usr/lib/python3/dist-packages/yunohost/log.py", line 870, in _get_description_from_name
    datetime.strptime(" ".join(parts[:2]), "%Y%m%d %H%M%S")
ModuleNotFoundError: No module named '_strptime'

edit: tried another time: same error message
edit2: full log here: https://paste.yunohost.org/raw/iguvotuqeb

[Aleks]

Hmyeah i have a patch for this (the _strptime thing) coming in the next release … but that’s won’t fix it for you now, you probably need to systemctl restart yunohost-api

[tufek]

That did the trick!
Migration ok and Nextcloud/Wireguard are both functional.

[arkadi]

i just did the migration on my server running nextcloud, transmission, two apps with docker/redirect app (immich/jellyfin).

it finished successfully but gives a 500 internal server error. everything works though.

edited to add log:
https://paste.yunohost.org/raw/uwijifiyaj

[Aleks]

Hmmokay so what exactly “gives a 500 internal error”, do you mean the webadmin ?

If you mean the webadmin, let’s look at the output of journalctl -u yunohost-admin -n 100 --no-pager --no-hostname | yunopaste

[arkadi]

@Aleks

journalctl -u yunohost-admin -n 100 --no-pager --no-hostname | yunopaste

yes in the webadmin.

https://paste.yunohost.org/raw/yujuyeqibi
no error

[Aleks]

Hmf so let’s look instead at yunohost tools migrations list --pending

[arkadi]

strange. nothing there either.

yunohost tools migrations list --pending
migrations: 

[tufek]

Had the same error message in both of my migrations. I thought it was due to API restart or something like that. The program warns it a few seconds before by the way.

[fritz2cat]

@Kayou

Don’t worry to much about this SendersScore alert, it will soon be removed

Looking at the fix fix: Remove SenderScore from the dnsbl_list.yml file by milouse · Pull Request #1918 · YunoHost/yunohost · GitHub :
I see another evidence, the following remaining lines contain:

• name: AntiCaptcha.NET IPv6
  dns_server: dnsbl6.anticaptcha.net
  website: http://anticaptcha.net/

The domain anticaptcha.net is for sale. Any reference to that domain should also be removed.

[pihomeserver]

Just to say that all is good after the migration on my VPS. Only a small issue during distro upgrade and the grub-pc package. I had to run the apt command manually to answer to continue despite an error
I restarted the migration from the command line and now all is good. I only have VS Code as an installed app and 12 redirect apps for my containerized apps

Great job guys

Stupid question maybe : can i now remove the testing part from the apt source for yunohost ?