YunoHost 3.8 testing / Call for feedback

Announcements
, ,
81

any recover plan here?

82

Yes I have VERY ugly workaround

you edit the line 139 and 370 of /usr/lib/metronome/net/server_event.lua and remove the ; at the end of function(). Like

		if self.type == "client" then
			local callback = function()
				self:_lock(false, false, false);

		self._usingssl = true;
		self.startsslcallback = function()  -- we have to start the handshake outside of a read/write event
			self.startsslcallback = nil;

Then restart the upgrade with sudo apt-get dist-upgrade

83

@HugoPoi- great worked out - thanks a lot

1 Like
84

3.13.6.1 has been built and should fix the issue 


85

I upgraded my Lime 2 (dedicated for testing) to 3.8. Overall this release is fantastic. I love the Diagnostic page ! :slight_smile:

Here is a few thing that need to be fixed/enhanced (IMHO) :

  1. Like @francoisa, the script aborted before the end. The fix is quite simple, just add the -y parameter to the apt-get dist-upgrade in the script.

  2. After the dist-upgrade, I noticed these packages are no longer necessary :

    The following packages were automatically installed and are no longer required:
    glances javascript-common libjs-angularjs libjs-lodash python-gnupg python3-psutil

    Are this packages meant to be removed ?

    If it’s the case, good ! By manually autoremoving the glances service, it saves me 18MiB of memory. \o/ Yeah, I have a Lime 2 so every MiB matter. :stuck_out_tongue:

    But could the removing of these packages be automated ? Or could an autoremove button be added to the webadmin ? Because after the upgrade or a restart, the glances service was still running although it is no longer required according to apt-get (the systemd unit is still enabled).

  3. When I went to Applications in the webadmin, I noticed this warning :

    Argument --installed ain't needed anymore when using 'yunohost app list'. It directly returns the list of installed apps..

    It would be nice to replace all yunohost app list --installed commands by yunohost app list in Yunohost’s code. It’s just a warning message though.

  4. In the Diagnosis page :

  • In the Email section :

    • The rDNS test says the current value and the expected one are different. But in fact, the only difference is the dot (.) at the end for the current record (which represents the DNS root).

      image
      image1088×214 42 KB

      I’m pretty sure the dot must be present at the end of the FQDN (to show that the value is absolute and not relative). For example, a dig command shows the final dot.

    • My server’s IPv4 address was blacklisted by SPFBL as shown on this screenshot :

      image
      image1088×90 14.8 KB

      No problem, I removed it from their list. But even after rebooting the server and rerunning the diagnosis, Yunohost still shows me the issue (without the The blacklist reason is [...] line) :
      image
      image1088×66 10.6 KB

      Since SPFBL doesn’t blacklist my IPv4 anymore, I don’t get why Yunohost consider it is still a problem. Caching issue ?
      Edit : Well, it’s gone now. I did several reboot and forced the diagnosis without immediate results. After +/- 1 hour, the error went away (it says that my IPv4 isn’t blacklisted). I don’t get it


  • In the System resources section :

    image
    image1118×432 55.3 KB

    • If you have a look at the picture below, the numbers for the /home partition are really weird. Yunohost says that 2GB are taken (most probably a bit less since the available space has been rounded off) but that there is 100% space left. :smiley:
    • Why in some cases does Yunohost round off to the nearest whole number and in other cases does it round off to one decimal place ? I would round off all numbers to one decimal place (exempt for the percentages).
    • My /var/log partition is 48MiB big (like most of the Lime 2 since it uses zram). There is 87% of the space left, but still Yunohost says that it is an issue. In term of raw space, 39MiB of space left is not big, but that shouldn’t matter. I think that Yunohost should say it’s an issue only when the percentage of left space has reached a specific threshold.

  • In the DNS records section :
    Yunohost says the DNS config is missing this CNAME record :

    image
    image1088×150 20.7 KB

    This is wrong because this record is set and by querying multiple public DNS resolvers, I can see the change has been propagated. I suspect this test fails because dnsmasq is configured as the authoritative server of the main domain name (thus, it won’t forward the request to the upstream DNS resolver) but has no idea of this CNAME record.
    Edit : like the IPv4 blacklist issue, this problem got fixed too. When I think of it, the first time I opened the diagnosis page, the CNAME record wasn’t set. I created the CNAME record after. So, the same way as the blacklist issue, it probably took some time to disappear.

86

Ugh yeah I’ll try to have a look that should be easy to fix 
 I think I originally did not include it because I wanted people to confirm explicitly


Yes, not sure about all of them, but at least it’s expected for glances and python-gnupg. Others might be remains of other stuff that happened on your server

Yeah indeed 
 I don’t know about automating this 
 the reason Debian does not automate this is that for some reason you may want to keep one of these installed and Debian cannot just assume that it’s okay to autoremove them 
 On the other hand, maybe we could decide to assume that yunohost’s users are likely less technical and trigger the autoremove automatically idk 


Indeed 
 I think that’s in fact a bug on the domain info page 
 Fixed it here

Should be fixed by this commit (not yet released)

Indeed, that’s just DNS cache / propagation here I believe 
 we could add a message in the details saying that changes may take ~one hour to propagate.

Indeed many weird numbers / inconsistencies 
 turns out the numbers we get from psutil aren’t as easy as total = used + free and free_percent = free / total 
 Linux has a 5ish % margin 
 Discussed with ljf and should be fixed here

Yup, probably DNS cache / propagation here, should probably add a tip about it as well 


Anyway, thanks for the extensive tests and feedback ! As you can see, allowed to fixed at least 3 or 4 bugs :stuck_out_tongue_winking_eye: :+1:

2 Likes
87

When 3.8 is ready to go out ?
I have tested the release on Vagrant x86 and Lime2, not encountered anything broken yet, I haven’t tested a lot the UI.
If I can give some help to make it land ?

88

@HugoPoi : indeed it’s pretty much done now. I just have a small bunch of misc ux / messaging improvements that I’ll try to finish today.

Then it’s mostly about us being confident that it’s stable enough and that there isn’t any huge issue. The most useful tests are also the most complicated one as in “migrate an existing instance” - preferably as old as you can as it increases the likelihood to spot bugs :stuck_out_tongue_winking_eye: . Also if you didn’t test the webadmin, I think it would make a lot of sense to do so if you have relevant feedback about the diagnosis system and what it reports.

1 Like
89

Here we are with another iteration, 3.8.4 which is a release candidate !

Changes (with respect to 3.8.1.2)

  • [fix] Restoration of custom hooks / missing restore hooks (YunoHost#927)
  • [enh] Real CSP headers for the webadmin (YunoHost#961)
  • [enh] Simplify / optimize reading version of yunohost packages
 (YunoHost#968)
  • [fix] Try to autorestart ldap when it’s down (Moulinette#247, YunoHost#975)
  • [enh] service.py cleanup + add tests for services (YunoHost#979, YunoHost#986)
  • [fix] Enforce permissions for stuff in /etc/yunohost/ (YunoHost#963)
  • [mod] Remove security diagnosis category for now, Move meltdown check to base system (YunoHost/a799740a)
  • [mod] Change warning/errors about swap as info instead 
 add a tip about the fact that having swap on SD or SSD is dangerous (YunoHost/23147161)
  • [enh] Improve auto diagnosis cron UX, add a --human-readable option to diagnosis_show() (YunoHost/aecbb14a)
  • [enh] Rely on new diagnosis for letsencrypt elligibility (YunoHost#985)
  • [i18n] Translations updated for Catalan, Dutch, Esperanto, French, German, Nepali, Polish, Spanish

Thanks to all contributors :heart: ! (autra, Bram, clecle226, É. Gaspar, I. HernĂĄndez, Kay0u, M. Döring, xaloc33, Zeik0s)

9 Likes
91

Hi,
I can’t upgrade to 3.8.4.
When running apt upgrade it says

0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded

but apt list --upgradable returns

yunohost/stable 3.8.4 all [upgradable from: 3.8.2.2].

The log from webadmin upgrade is available here.
It says it is a success but the version remains 3.8.2.2.

92

@guilg : can you tell what kind of hardware you’re running ? For now we have an issue with arm64 and builds are unavailable (but ultimately they should be)

93

The command don’t upgrade the system, just prepare the webdadmin to upgrade after.

curl https://install.yunohost.org/switchToTesting | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   459  100   459    0     0   2811      0 --:--:-- --:--:-- --:--:--  2833
----
Patching sources.list to enable testing repository...
----
----
Running 'apt-get update'
----
Hit:1 http://forge.yunohost.org/debian stretch InRelease
Get:2 http://security.debian.org stretch/updates InRelease [94.3 kB]                                    
Ign:3 http://ftp.debian.org/debian stretch InRelease                                                               
Get:4 http://ftp.debian.org/debian stretch-updates InRelease [91.0 kB]                                             
Get:5 http://forge.yunohost.org/debian stretch/testing amd64 Packages [3,465 B]                 
Hit:6 http://ftp.debian.org/debian stretch Release                                  
Hit:7 https://packages.sury.org/php stretch InRelease
Fetched 189 kB in 1s (112 kB/s)
Reading package lists... Done
----
Running 'apt-get dist-upgrade'
----
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  moulinette ssowat yunohost yunohost-admin
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

Go to the webadmin and upgrade moulinette, yunohost


94

Okay this is it, I’m using an ARM VPS. I’ll wait then :slight_smile:
Thanks!

95

From a fresh install 3.7.1.3 (stable) with one user and zerobin app, on PC local host (with /etc/hosts), the upgrade command show only :

# curl https://install.yunohost.org/switchToTesting | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   459  100   459    0     0   1909      0 --:--:-- --:--:-- --:--:--  1912
----
Patching sources.list to enable testing repository...
----
----
Running 'apt-get update'
----
Hit:1 http://security.debian.org stretch/updates InRelease
Hit:2 http://forge.yunohost.org/debian stretch InRelease
Ign:3 http://ftp.debian.org/debian stretch InRelease
Hit:4 http://ftp.debian.org/debian stretch-updates InRelease
Hit:5 http://ftp.debian.org/debian stretch Release 
Get:6 http://forge.yunohost.org/debian stretch/testing amd64 Packages [3,465 B]
Fetched 3,465 B in 0s (3,576 B/s)                        
Reading package lists... Done
----
Running 'apt-get dist-upgrade'
----
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  etckeeper iproute python-gnupg
Use 'sudo apt autoremove' to remove them.
The following packages will be upgraded:
  moulinette ssowat yunohost-admin
3 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 9,408 kB of archives.
After this operation, 2,048 B disk space will be freed.
Do you want to continue? [Y/n] Abort.

And the webadmin :

  • moulinette (de 3.7.1.1 Ă  3.8.1)
  • ssowat (de 3.7.1.1 Ă  3.8.0)
  • yunohost (de 3.7.1.3 Ă  3.8.4)
  • yunohost-admin (de 3.7.1.1 Ă  3.8.3.1)

The first upgrade just upgrade moulinette ssowat yunohost-admin

https://paste.yunohost.org/raw/hebikeyiya

The apps list is not very nice


After the upgrade of yunohost, the webadmin page show an error on the top :
http://mydomain/versions?locale=fr’
caused an error:
Not found: ‘/versions’

And the upgrade failed, because in the webadmin tool, the yunohost package can already be upgrade from

https://paste.yunohost.org/raw/nubesarifu

apt shows the package upgradable :
apt list --upgradable
Listing
 Done
yunohost/stable 3.8.4 all [upgradable from: 3.7.1.3]
N: There is 1 additional version. Please use the ‘-a’ switch to see it

It would be great to have a message or notification when the upgrade is finish, because waiting 10 minutes is not really friendly.

The second upgrade failed too.
The weadmin says :

Error: 404 Not Found

Sorry, the requested URL ‘http://mydomain.tld/versions?locale=fr’
caused an error:
Not found: ‘/versions’

And the paste :
https://paste.yunohost.org/raw/meyiwukosu

96

The problem is that yunohost 3.8.4 needs metronome >= 3.14.0

apt list --upgradable
Listing... Done
yunohost/stable 3.8.4 all [upgradable from: 3.7.1.3]
N: There is 1 additional version. Please use the '-a' switch to see it
root@yunohost:/home/admin# apt-get install yunohost
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 yunohost : Depends: metronome (>= 3.14.0)
E: Unable to correct problems, you have held broken packages.
root@yunohost:/home/admin# dpkg -l|grep metronome
ii  metronome                             3.12.0+yunohost-1                  amd64        Lightweight XMPP server with advanced microblogging features
root@yunohost:/home/admin# apt-get install metronome
Reading package lists... Done
Building dependency tree       
Reading state information... Done
metronome is already the newest version (3.12.0+yunohost-1).
metronome set to manually installed.
The following packages were automatically installed and are no longer required:
  etckeeper iproute python-gnupg
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
97

Upgrade for

  • moulinette (de 3.7.1.1 Ă  3.8.1)
  • ssowat (de 3.7.1.1 Ă  3.8.0)
  • yunohost-admin (de 3.7.1.1 Ă  3.8.3.1)

ran smoothly. But yunohost was not upgraded. Now some webadmin URLs are not reachable:

  • the apps catalog
# Error: 404 Not Found

Sorry, the requested URL 'http://mydomain.nohost.me/appscatalog?full&with_categories&locale=de'
caused an error:

Not found: '/appscatalog'
  • The diagnosis
# Error: 404 Not Found

Sorry, the requested URL 'http://mydomain.nohost.me/diagnosis/run?except-if-never-ran-yet'
caused an error:

Not found: '/diagnosis/run'

In applications, the whole list of application (catalog) is displayed.
And when I click on applications that are not installed I get :

Die App 20euros konnte nicht in der Liste installierter Apps gefunden werden:
* borgserver
* cesium
* duniter
* hotspot
* ihatemoney
* mautrix_whatsapp
* netdata
* synapse
* vpnclient

Lastly, the duniter webui seems to be broken, but the app is still experimental.

Then I tried to upgraded yunohost again but still it appears in the system update page, and same issues persists.

98

@francoisa @gauthier67

Ugh yes indeed there was an issue with the availability of metronome builds for technical reasons 
 I launched a build and should be available in ~10 min, please try to reupgrade then

99

Yeah ! It’s better. All packages are upgrade. YunoHost 3.8.4 (testing).
No error in webadmin login.
I test every link and they work. Tested with no-cache on my Firefox 76

  • Previous Users are presents. I can create a new user and log in.
  • Domain is ok.
  • Apps are presents and usefull.
  • The app list show categories. I can install an app : lstu
  • Upgrade : all is ok
  • Services : all are present, But !!! There is an issue : I can stop a service like fail2ban or lstu.
    When i click on a line, I’ve got an error.
Action

GET /services//log
 {"locale":"fr","name":"fail2ban","number":"50"}

Trace

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/api.py", line 489, in process
    ret = self.actionsmap.process(arguments, timeout=30, route=_route)
  File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 587, in process
    return func(**arguments)
  File "/usr/lib/moulinette/yunohost/service.py", line 421, in service_log
    log_list.remove(name)
AttributeError: 'str' object has no attribute 'remove'

  • Tools :

    • logs : all presents. I can use Yunopaste
    • Migrations : no issues
    • Firewall : I can block and open a port
    • admin password : I can change it.
    • Stop and reboot. : I can stop and reboot.

  • Diagnostics show me errors that are ok, because my YNH is only local. No NAT to my PC.
    https://paste.yunohost.org/raw/qamimocupo
    Logs are very nice !
    My resolv.conf is custom because My yuno is in a proxmox container.

  • Backups are ok : I can see my previous backups and I can backup.
    But if I want to restore a backup, users are restored, but apps can’t :
    Yunohost says app already exist.

    2020-05-10 20:11:58,207: INFO - Restauration de l’application lstu 

    2020-05-10 20:11:58,210: ERROR - Une application est dĂ©jĂ  installĂ©e avec l’identifiant ‘lstu’

    2020-05-10 20:11:58,217: INFO - Restauration de l’application zerobin 

    2020-05-10 20:11:58,217: ERROR - Une application est dĂ©jĂ  installĂ©e avec l’identifiant ‘zerobin’

100

Bad timing then. Indeed now it worked. I just upgraded
https://paste.yunohost.org/raw/gacikiloji
https://paste.yunohost.org/raw/efipafimow
https://paste.yunohost.org/raw/azaziloqer
https://paste.yunohost.org/raw/uqapefalab

I love the new app list! Very much easy to find a new app. And the app status and poping explanations are just enough to understand. Maybe the explanations could pop up faster.

Some issue left:

  • I cannot access the page of any specific service.
GET /services//log
{"name":"ynh-hotspot","number":"50"}

Traceback

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/moulinette/interfaces/api.py", line 489, in process
    ret = self.actionsmap.process(arguments, timeout=30, route=_route)
  File "/usr/lib/python2.7/dist-packages/moulinette/actionsmap.py", line 587, in process
    return func(**arguments)
  File "/usr/lib/moulinette/yunohost/service.py", line 421, in service_log
    log_list.remove(name)
ValueError: list.remove(x): x not in list
  • Diagnosis section web
    Domain mondomain.nohost.me appears unreachable through HTTP from outside the local network.
    Although have a VPN (ARN). I guess HTTPS is forced so it should not be a problem?
  • Diagnosis section service
    Service ynh-vpnclient-checker is dead :(
    Probably this should not be considered a “problem” rather a “warning” or “info” as dead=good. Or this should be ignored by default?
  • System resources SWAP.
    The system has only 500 MiB swap. You should consider having at least 512 MiB to avoid situations where the system runs out of memory.
    Probably set the threshold to <=499 or <500 to avoid having an info message in many cases.

Awesome work ! As usual :wink: