YunoHost 3.7 spooky testing / Call for feedback

ludovic · November 10, 2019, 7:35pm

Hi,

I upgraded to 3.7.0.1 without any issue for me
The groups and permissions stuff is good and very useful !

Best,
Ludovic

Aleks · December 2, 2019, 11:54pm

So there was a few iterations and we should be getting close to being able to release this …

For people that already are in testing : please not that after upgrading to the 3.7.0.4, if you want everything to run smoothly, you should :

run yunohost user permission list
for every permission with visitors enabled but not all_users, you should add the all_users group to this permission like this :

yunohost user permission update foo.main --add all_users

(replacing foo.main with the name of the permission). Note that you can also do the same thing from the webadmin interface (In Users > Manage groups and permissions)

Jaireimy · December 3, 2019, 10:35am

Hello,

Do you know if the stable version will be released in days / weeks / months ?

Thank you

Gofannon · December 3, 2019, 12:48pm

Mise à jour effectuée sans encombre.

Je n’utilise pas les groupes et peu de permissions différentes sur mon serveur. L’interface est claire en tout cas!

Aleks · December 3, 2019, 2:54pm

Possibly in a few days / weeks, yes… The more feedback we get, the faster we can validate the release with as few issues as possible

ludovic · December 4, 2019, 7:59pm

Hi,

Upgrade went smoothly Thanks !
I added the visitors on all_users groups in both command line and webadmin

Ludovic

ashledombos · December 6, 2019, 9:26pm

Hi, it seems it’s my first post here, so hello everybody

Here is the output I have from launching the command:

root@garnet:/home/admin# curl https://install.yunohost.org/switchToTesting | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   459  100   459    0     0   5010      0 --:--:-- --:--:-- --:--:--  5043
----
Patching sources.list to enable testing repository...
----
----
Running 'apt-get update'
----
Ign:1 http://ftp.debian.org/debian stretch InRelease
Hit:2 http://security.debian.org stretch/updates InRelease 
Hit:3 http://ftp.debian.org/debian stretch-updates InRelease                                                   
Hit:4 http://ftp.debian.org/debian stretch Release                                                             
Hit:5 http://forge.yunohost.org/debian stretch InRelease                                 
Reading package lists... Done                      
----
Running 'apt-get dist-upgrade'
----
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  etckeeper iproute
Use 'sudo apt autoremove' to remove them.
The following packages will be upgraded:
  moulinette ssowat yunohost yunohost-admin
4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.2 MB of archives.
After this operation, 672 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.
root@garnet:/home/admin# curl https://install.yunohost.org/switchToTesting | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   459  100   459    0     0   4379      0 --:--:-- --:--:-- --:--:--  4413
----
Patching sources.list to enable testing repository...
----
----
Running 'apt-get update'
----
Ign:1 http://ftp.debian.org/debian stretch InRelease
Hit:2 http://ftp.debian.org/debian stretch-updates InRelease                                                                
Hit:3 http://security.debian.org stretch/updates InRelease                                                                  
Hit:4 http://forge.yunohost.org/debian stretch InRelease                                       
Hit:5 http://ftp.debian.org/debian stretch Release                       
Reading package lists... Done                      
----
Running 'apt-get dist-upgrade'
----
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  etckeeper iproute
Use 'sudo apt autoremove' to remove them.
The following packages will be upgraded:
  moulinette ssowat yunohost yunohost-admin
4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 10.2 MB of archives.
After this operation, 672 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.

See, at then end it automatically aborts, so I had to apt update/upgrade by hand. After this migration seemed ok, but I have this message in admin backend:

Unable to retrieve the feed: Security | Yunohost Documentation. An extension may prevent your browser from executing this request (or the website may be down).

ashledombos · December 6, 2019, 9:32pm

BTW, will this issue be fixed with new release?

(also this one seems to be the same)

JimboJoe · December 6, 2019, 9:41pm

Weird error happening updating discourse…
https://paste.yunohost.org/ivogibihij.coffeescript

EDIT: nevermind, I removed/re-added all_users and visitors permissions and it worked (now I understand the release notes of the latest beta )

Maniack_Crudelis · December 12, 2019, 5:16pm

Hi guys

is there a documentation or anything similar about the new permission system ?
Also, how the migration is going to be for the apps that don’t use that permission system yet ?

EDIT:
Looks like the answer is into that PR 'visitors' mechanics (replace the old public/private mechanic) + integrate the permission system with SSOwat by alexAubin · Pull Request #797 · YunoHost/yunohost · GitHub
I wasn’t there by the time of that PR, and I’m not so relieved actually…
Found the doc as well, https://github.com/YunoHost/doc/blob/group-and-permissions/groups_and_permissions.md

What’s worry me especially is that the doc says :

Applications scripts should absolutely NOT mess up with any already-existing app accesses

Public/private access is not always just about skipped_uris "/", there’s many specific access, like an admin panel or other stuff.
We can’t expect that users are going to play with all the specific permissions for an app when willing to turn off or on the public access for an app. It as to be simple.
So actions scripts dedicated to that purpose have to be still available for the users willing to turn on or off a public access without complication.

EDIT 2:
Before releasing that testing, a clear announce has to be done to all packagers to explain how to modify their apps with examples and clear explanations.
I’m really worried that we’re going to break a lot of apps if there’s no proper warning about that new feature. I wasn’t aware of it myself.

tituspijean · December 12, 2019, 6:12pm

Do we have a bot on Github that can automagically open an issue in the apps’ repos if scripts contain anything related to accesses? (Now that I wrote it, I think it will involve all apps actually)

Maniack_Crudelis · December 12, 2019, 6:16pm

Exactly @tituspijean, if not almost every apps.
If I understand well, for apps with a simple skipped_uris "/" it shouldn’t be a problem. But for all apps with API access, admin panel or public download links, it’s going to be a problem.

Aleks · December 12, 2019, 7:12pm

Pretty much everything is backward compatible, so basically all install and other scripts will keep working as they do right now (modulo depreciationg warning), and if they don’t then that’s a bug that should be fixed.

As far as I remember, the only not-backward compatible thing is the use of protected uris, in the sense that even when being logged-in you still won’t have access to the protected uris (so that’s kinda conservative). The affected apps are apparently : lstu, lufi, dotclear, lektor, bozon, and lutim.

Admin panels are expected to be handled by a specific permission. Typically you would define such a permission with something like :

ynh_permission_create --permission "admin" --url "/admin" --allowed "$admin_user"

which controls accesses to https://domain.tld/app/admin. Then the instance’s admin can add/remove accesses to this permission from the webadmin interface, just like it does for the main permission which controls access to /app. (Also, if I remember correctly, a user has to be authorized for /app/ to be able to access /app/admin.)

Not sure what you’re thinking about, but right now it’s “just” about : going into Users > Manage permissions and groups, then adding / removing the permission to the ‘Visitors’ group and that’s it. Then visitors will be able to access (or not) to /app (but not to /app/admin if it’s controlled through a diferent permission)

There have been test batches using the unstable CI to validate that there was no regression (at least from the CI point of view) though I didn’t re-ran a batch of test following the release of the testing and the next iterations of fixes

Maniack_Crudelis · December 12, 2019, 7:28pm

What I’m thinking about is that kind of thing:

github.com

YunoHost-Apps/jirafeau_ynh/blob/testing/scripts/install#L150-L156


      
          #!/bin/bash
          
          #=================================================
          # GENERIC START
          #=================================================
          # IMPORT GENERIC HELPERS
          #=================================================
          
          source _common.sh
          source /usr/share/yunohost/helpers
          
          #=================================================
          # RETRIEVE ARGUMENTS FROM THE MANIFEST
          #=================================================
          
          enable_crypt="false"
          
          fpm_footprint="low"
          fpm_free_footprint=0
          fpm_usage="low"

This file has been truncated. show original

That’s not obvious (my bad by the way) but here the idea is to have the links always publicly available, but not the main page of jirafeau nor its admin panel.

It would translate as 3 different permissions. But an user willing to turn the app public or private shouldn’t have to play with those 3 different permissions and know which one should be private or not.
An user that want to turn the app public or private should be able to simply ask for it, without having to know how it should work. And that exactly the purpose of the actions public_private.

Jirafeau is only one example of those permissions.
We have also that kind of situation for API for mobile apps.

Those situations are what I’m thinking about when I say that apps are going to break. The CI can’t test those things.

Again, I’m not saying that all that permission system is a bad thing, but packagers should have a proper warning and a clear explanation before that testing is released.

Kayou · December 17, 2019, 4:55am

I quickly read the install script of all our apps tagged as “working”.

Among them, here a list of the ones which are doing non trivial things with the old permission system (not only a simple protect “/”):

I may miss some of them.
I added those that are doing something like yunohost app addaccess $app -u $admin because I was not sure if it can be a problem with the new permission system.

Kayou · March 15, 2020, 5:09pm

A new testing version has been released, which includes some important fixes:

Detailed changelog

[fix] Match undefined function (SSOwat#151)
[fix] Cohabitation between the old and the new permission system (SSOwat#156)
[fix] Slapd may crash if we try to update the LDAP with no change (Moulinette#231)
[fix] Permission url (YunoHost#871)
[fix] DNS resolver (YunoHost#859)
[fix] Legacy permission management (YunoHost#868, YunoHost#855)
[enh] More informations in hooks permission (YunoHost#877)
[i18n] Improved translations for French, Basque, Hindi, Turkish, Bengali (Bangladesh), Arabic, Hungarian, Polish, Catalan, Dutch, Portuguese, Italian, German, Swedish, Russian, Esperanto, Occitan, Nepali

Thanks so much to all the contributors ! (Aleks, amirale qt, Bram, ButterflyOfFire, elie gavoty, Filip Bengtsson, Josué, Kay0u, ljf, Maniack, Quenti, xaloc33)

ppr · March 15, 2020, 5:37pm

Bonsoir,

Dans le cadre du passage de Stretch à Buster, faudra-t’il passer par un mise à niveau de YunoHost 3.6.x à 3.7.x préalablement à la migration ?

ppr

Kayou · March 18, 2020, 1:10pm

Bonjour,

Rien n’est encore fixé il me semble, mais de toute façon il vaut mieux (si ce n’est pas obligatoire?) être à la version la plus à jour avant de faire la migration.

Nous prévoyons encore une 3.8 avant le passage à la 4.0 (Buster) afin de préparer le terrain.

ppr · March 18, 2020, 1:14pm

Bonjour,

Merci pour ces informations.
La version stable actuelle est 3.6.x.
Est-ce que la 3.7.x testing va bientôt passer en stable ?
On verra pour la 3.8.x après avant la migration vers la 4.0.x

ppr