502 bad gateway after trying to install app (Yunohost 3.3.2)

gorgothzilla · December 5, 2018, 3:37pm

Hi folks,

I recently made the switch to Yunohost 3 (after migratin from Debian Jessie to Stretch), and everything was going smooth for a week or so. Then I tried to install Piwigo today through the web interface. The installation failed for obscure reasons, didn’t save the logs , all I rememberd was this line : “Packages need to be removed but remove is disabled.” . Right after that almost all my apps were throwing a 502 bad gateway errors. Here is my apps list:

➜  ~ _ yunohost app map
tutu.email/rainloop/: Rainloop → not working
tototatatiti.net/cloud/: Nextcloud → not working
tototatatiti.net/couchpotato/: CouchPotato → working
tototatatiti.net/dokuwiki: Dokuwiki → not working
tototatatiti.net/home: home → working (html only, no php)
tototatatiti.net/htmltool/: HTML Tools → not working
tototatatiti.net/jirafeau/: Jirafeau → not working
tototatatiti.net/rss/: FreshRSS → working
tototatatiti.net/strut/: Strut → working
tototatatiti.net/torrent/: Transmission → working
tototatatiti.net/webmail/: Rainloop → not working
tototatatiti.net/zerobin/: Zerobin → not working

So I looked up on the forum for similar errors and almost everyone was speaking about php7-fpm not starting, but on my server it started well. But I think it’s not serving fast-cgi service for applications as when I check its status, it seems to be only working for freshrss and www.

➜  ~ sudo systemctl status php7.0-fpm.service
● php7.0-fpm.service - The PHP 7.0 FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php7.0-fpm.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2018-12-05 15:46:52 CET; 35min ago
     Docs: man:php-fpm7.0(8)
 Main PID: 4953 (php-fpm7.0)
   Status: "Processes active: 0, idle: 5, Requests: 25, slow: 0, Traffic: 0req/sec"
    Tasks: 6 (limit: 4915)
   CGroup: /system.slice/php7.0-fpm.service
           ├─4953 php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf)
           ├─4954 php-fpm: pool freshrss
           ├─4955 php-fpm: pool freshrss
           ├─4956 php-fpm: pool freshrss
           ├─4957 php-fpm: pool www
           └─4958 php-fpm: pool www

Dec 05 15:46:52 tototatatiti.net systemd[1]: Starting The PHP 7.0 FastCGI Process Manager…
Dec 05 15:46:52 tototatatiti.net systemd[1]: Started The PHP 7.0 FastCGI Process Manager.

So I thought maybe trying to install another app would regenerate some conf files somewhere. So I tried installing Firefly III, with no success either. Here are the logs output https://paste.yunohost.org/raw/likasegate

Here is my server diagnostic from the web interface https://paste.yunohost.org/raw/zeyahojohi

If you need further information, please let me know.

Oh, and I speak french but I thought this message, and possible solution, would be usefull to more people in english, but if you prefer to speak french, c’est vous qui voyez .

Aleks · December 5, 2018, 3:51pm

Hmmmm can you check that the migrations were actually ran ? There’s a migration thing that’s supposed to migrate all your php stuff.

You should be able to check that with :

yunohost tools migrations list --pending

and then you can run them with

yunohost tools migrations migrate

gorgothzilla · December 5, 2018, 4:03pm

You may be right, I was checking nginx error logs and found this

tototatatiti# tail -n5 /var/log/nginx/tototatatiti.net-error.log
2018/12/05 16:48:44 [crit] 11185#11185: *735 connect() to unix:/var/run/php5-fpm-nextcloud.sock failed (2: No such file or directory)
while connecting to upstream, client: 81.64.79.73, server: tototatatiti.net, request: "GET /cloud/status.php HTTP/1.1", upstream: "fas
tcgi://unix:/var/run/php5-fpm-nextcloud.sock:", host: "tototatatiti.net"
2018/12/05 16:49:03 [crit] 11185#11185: *722 connect() to unix:/var/run/php5-fpm-nextcloud.sock failed (2: No such file or directory)
while connecting to upstream, client: 81.64.79.73, server: tototatatiti.net, request: "GET /cloud/ HTTP/1.1", upstream: "fastcgi://uni
x:/var/run/php5-fpm-nextcloud.sock:", host: "tototatatiti.net", referrer: "https://tototatatiti.net/yunohost/admin/"
2018/12/05 16:49:16 [crit] 11185#11185: *746 connect() to unix:/var/run/php5-fpm-nextcloud.sock failed (2: No such file or directory)
while connecting to upstream, client: 81.64.79.73, server: tototatatiti.net, request: "GET /cloud/status.php HTTP/1.1", upstream: "fas
tcgi://unix:/var/run/php5-fpm-nextcloud.sock:", host: "tototatatiti.net"
2018/12/05 16:49:48 [crit] 11185#11185: *748 connect() to unix:/var/run/php5-fpm-nextcloud.sock failed (2: No such file or directory)
while connecting to upstream, client: 81.64.79.73, server: tototatatiti.net, request: "GET /cloud/status.php HTTP/1.1", upstream: "fas
tcgi://unix:/var/run/php5-fpm-nextcloud.sock:", host: "tototatatiti.net"
2018/12/05 16:50:20 [crit] 11185#11185: *752 connect() to unix:/var/run/php5-fpm-nextcloud.sock failed (2: No such file or directory)
while connecting to upstream, client: 81.64.79.73, server: tototatatiti.net, request: "GET /cloud/status.php HTTP/1.1", upstream: "fas
tcgi://unix:/var/run/php5-fpm-nextcloud.sock:", host: "tototatatiti.net"

Running the first command :

➜  ~ _ yunohost tools migrations list --pending
migrations:
  0:
description: Reconfigure the PHP pools to use PHP 7 instead of 5
disclaimer: None
id: 0004_php5_to_php7_pools
mode: auto
name: php5_to_php7_pools
number: 4
  1:
description: Migrate databases from postgresql 9.4 to 9.6
disclaimer: None
id: 0005_postgresql_9p4_to_9p6
mode: auto
name: postgresql_9p4_to_9p6
number: 5
  2:
description: Synchronize admin and root passwords
disclaimer: Yunohost now expects admin and root passwords to be synchronized. By running this migration, your root password is goi
ng to be replaced by the admin password.
id: 0006_sync_admin_and_root_passwords
mode: manual
name: sync_admin_and_root_passwords
number: 6

then, running the second command

~ _ yunohost tools migrations migrate --accept-disclaimer
Warning: Running migration 4 php5_to_php7_pools...
Synchronizing state of php7.0-fpm.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable php7.0-fpm
Failed to stop php5-fpm.service: Unit php5-fpm.service not loaded.
Failed to disable unit: No such file or directory
rm: cannot remove '/etc/logrotate.d/php5-fpm': No such file or directory
Warning: Running migration 5 postgresql_9p4_to_9p6...
Warning: Postgresql was not installed on your system. Nothing to do!
Warning: Running migration 6 sync_admin_and_root_passwords...
Info: Your root password have been replaced by your admin password.

Chekcing php7.0-fpm.service now gives processes for all my apps that need it. It’s working. Thanks @Aleks, I’m editing the topic to [Solved].

Aleks · December 5, 2018, 4:07pm

Protip : on the forum there’s a feature such that if you click on the ... symbol on a post, you can pin it as “Solves the issue” instead of manually flagging the topic as solved

Anyway, what you encounter is a more general issue about the way migration works and it’s actually a bug we were discussing yesterday in the contributor meeting. We’ll have a look ~quickly to fix this as it kinda “break” the migration to stretch.

gorgothzilla · December 5, 2018, 4:12pm

Good to know, thanks for your help again, and yes, if it’s a bug, it would be interesting to implement a check feature in the migration tool so that pending migration stuff can be processed as well.