Sury repository is pushing possibly unwanted upgrades

SohKa · October 30, 2019, 10:08am

My YunoHost server

Hardware: VPS bought online
YunoHost version: 3.6.5.2
I have access to my server : Through SSH + through the webadmin
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : yes
If yes, please explain: I replaced the nameservers in /etc/resolv.dnsmasq.conf with Hetzner’s ones. I also added Hetzner’s mirrors in /etc/apt/sources.list.d.

Description of my issue

Hi !

I was checking my server’s upgrades this morning and I saw that the Sury repository is pushing the following upgrades :

$ apt list --upgradable                                                                                                                                    
Listing... Done                                                                                                                                                                               
libgd3/unknown 2.2.5-5.2+0~20190808.4+debian9~1.gbp6d9343 amd64 [upgradable from: 2.2.4-2+deb9u5]
libidn2-0/unknown 2.2.0-2+0~20191009.3+debian9~1.gbpf85c2e amd64 [upgradable from: 0.16-1+deb9u1]
libpcre3/unknown 2:8.43-1+0~20190710.6+debian9~1.gbpbfc49f amd64 [upgradable from: 2:8.39-3]
libssl-dev/unknown 1.1.1d-1+0~20191009.15+debian9~1.gbpd6badf amd64 [upgradable from: 1.1.0l-1~deb9u1]
libssl1.1/unknown 1.1.1d-1+0~20191009.15+debian9~1.gbpd6badf amd64 [upgradable from: 1.1.0l-1~deb9u1]
libtidy5/unknown 1:5.2.0-2+0~20180714172546.1+stretch~1.gbpc7c60a amd64 [upgradable from: 1:5.2.0-2]
libxml2/unknown 2.9.9+dfsg-1+0~20190914.4+debian9~1.gbp3b6674 amd64 [upgradable from: 2.9.4+dfsg1-2.2+deb9u2]
libzip4/unknown 1.5.1-4+0~20190318173229.9+stretch~1.gbp333132 amd64 [upgradable from: 1.1.2-1.1+b1]
openssl/unknown 1.1.1d-1+0~20191009.15+debian9~1.gbpd6badf amd64 [upgradable from: 1.1.0l-1~deb9u1]
php-apcu/unknown 5.1.17+4.0.11-2+0~20190828.11+debian9~1.gbp013b27 amd64 [upgradable from: 5.1.8+4.0.11-1]
php-common/unknown 2:70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:49]
php-curl/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-fpm/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-igbinary/unknown 3.0.1+2.0.8-2+0~20190814.12+debian9~1.gbpaafd11 amd64 [upgradable from: 2.0.1-1]
php-imagick/unknown 3.4.4-1+0~20190814.12+debian9~1.gbpc5da26 amd64 [upgradable from: 3.4.3~rc2-2]
php-intl/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-ldap/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-mbstring/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-mysql/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-pear/unknown 1:1.10.9+submodules+notgz-1+0~20191010.12+debian9~1.gbp296d25 all [upgradable from: 1:1.10.1+submodules+notgz-9+deb9u1]
php-redis/unknown 5.0.2+4.3.0-2+0~20190814.15+debian9~1.gbp070358 amd64 [upgradable from: 3.1.1-1]
php-tidy/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-xml/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php-zip/unknown 2:7.3+70+0~20190814.17+debian9~1.gbp1e7da2 all [upgradable from: 1:7.0+49]
php7.0-cli/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-common/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-curl/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-fpm/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-gd/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-intl/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-json/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-ldap/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-mbstring/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-mcrypt/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-mysql/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-opcache/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-readline/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-tidy/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-xml/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
php7.0-zip/unknown 7.0.33-12+0~20191026.23+debian9~1.gbp940de0 amd64 [upgradable from: 7.0.33-0+deb9u6]
python3-httplib2/unknown 0.11.3-1+0~20190212170628.3+stretch~1.gbp2efb8a all [upgradable from: 0.9.2+dfsg-1]
yunohost/stable 3.6.5.3 all [upgradable from: 3.6.5.2]

I checked with the apt-cache policy and every upgrade (appart from the yunohost package) are provided by the Sury repository. It is quite worrying because, among the other upgrades, it would upgrade php-fm from the 7.0 version to the 7.3. I’m afraid these upgrades will break my server.

Is that something expected with the last yunohost package release ? Is someone else experiencing the same issue (I don’t know if it really is an issue, in fact) ?

Kind regards,
sohka

Aleks · October 30, 2019, 11:05am

Hmmm so first : do you think you have a good reason to have sury enabled ? This is only supposed to be there because of an application which requires PHP 7.1, 2 or 3 … (though there are quite a lot of them)

SohKa · October 30, 2019, 11:13am

I really don’t know… The sury.list file has apparently been created (or modified…) yesterday at 08h07. I remember that I upgraded Nextcloud to the version 15.0.12~ynh1 around that time.

Anyway, here is my applications list :

 "applications": {
    "redirect": "Redirect 1",
    "redirect__4": "Redirect 4",
    "redirect__7": "Redirect 7",
    "bitwarden": "Bitwarden",
    "wallabag2": "Wallabag",
    "unattended_upgrades": "Unattended-upgrades",
    "dokuwiki": "Dokuwiki",
    "redirect__2": "Redirect 2",
    "redirect__9": "Redirect 9",
    "redirect__8": "Redirect 8",
    "funkwhale": "Funkwhale",
    "ttrss": "Tiny Tiny RSS",
    "redirect__5": "Redirect 5",
    "redirect__6": "Redirect 6",
    "rainloop": "Rainloop",
    "borg": "Borg",
    "shellinabox": "Shell In A Box",
    "opensondage": "OpenSondage",
    "zerobin": "Zerobin",
    "pleroma": "Pleroma",
    "nextcloud": "Nextcloud"
}

Aleks · October 30, 2019, 11:19am

Hmokay so it’s not clear if you really need the sury repo …

Do you remember if before running the Nextcloud upgrade you upgraded the system packages, which I’m guessing contained PHP upgrades ?

If so the combination of all this might have triggered the addition of the sury repo because of this (if that means anything to you))

Anyway, if that sounds plausible, you may just remove the sury.list file

SohKa · October 30, 2019, 11:35am

Indeed, I was checking the last commits in the ynh repo and the logs and came to the same conclusion.

So yeah, before the Nexcloud upgrade, I upgraded the PHP packages (as you can see here). It makes sense, so I’ll remove the sury.list file.

SohKa · October 30, 2019, 11:42am

For the record, here is the confirmation that you’re assumption was right (from Nextcloud’s upgrade logs) :

2019-10-29 08:07:22,099: DEBUG - + local dep_app=nextcloud
2019-10-29 08:07:22,099: DEBUG - + echo php-gd, php-json, php-intl, php-mcrypt, php-curl, php-apcu, php-redis, php-ldap, php-imagick, php-zip, php-mbstring, php-xml, imagemagick, acl, tar, smbclient, at
2019-10-29 08:07:22,099: DEBUG - + grep -q php
2019-10-29 08:07:22,099: DEBUG - + grep -q -v 7.0.33-0+deb9u5
2019-10-29 08:07:22,099: DEBUG - + dpkg --list
2019-10-29 08:07:22,204: DEBUG - + grep php7.0
2019-10-29 08:07:22,205: WARNING - --2019-10-29 08:07:22--  https://packages.sury.org/php/apt.gpg
2019-10-29 08:07:22,206: INFO - [##########++........] > Upgrading dependencies...
2019-10-29 08:07:22,206: DEBUG - + grep -nrq sury /etc/apt/sources.list /etc/apt/sources.list~ /etc/apt/sources.list.d
2019-10-29 08:07:22,206: WARNING - Resolving packages.sury.org (packages.sury.org)... 104.31.95.169, 104.31.94.169, 2606:4700:30::681f:5ea9, ...
2019-10-29 08:07:22,207: DEBUG - ++ lsb_release -sc
2019-10-29 08:07:22,207: DEBUG - + echo 'deb https://packages.sury.org/php/ stretch main'
2019-10-29 08:07:22,208: DEBUG - + wget -O /etc/apt/trusted.gpg.d/sury.gpg https://packages.sury.org/php/apt.gpg

Edit: Oh, I’m so rude… I forgot to thank you for all your amazing work ! So, thanks !

sytnfh · October 30, 2019, 1:52pm

Hi all,

I am sorry, but I am a bit lost:

Hardware: old pc at home
YNH version: 3.6.5.3
Access: full
Special context / tweaking: none
Apps: ttrss, piwigo, nextcloud, roundcube.

I did a routine system update/upgrade, then an app upgrade (the only app upgraded was Nextcloud), then other routine, morning update(s?) / upgrade(s?);
Nextcloud complained that php7.0-zip was not installed;
I manually "apt install"ed php7.0-zip (which came with dependencies), but the source of the package (blah sury.org blah: WTH?!?) caught my eye;
I read some posts about the dependency hell of pixelfed, which is not installed on my instance, but I have not understood everything;
58 packages (yes, fifty-eight) from this Sury repo are now installed on my instance, most of them related to php, but also openssl, libargon, libpcre, libssl, libsodium…

I am a bit worried about this repo mixture in my instance. Is there any way to come back to a pure Stretch/ynh blend? What should I do now, apart from suppressing sury.list?

Thx a lot for everything!

Aleks · October 30, 2019, 2:16pm

Well now that’s pretty much too late, but that might not be such a big deal, as long as php-fpm runs correctly and stuff …

sytnfh · October 30, 2019, 2:36pm

Ouch… I had not realized that yunohost tools update and yunohost tools upgrade could be that dangerous…

Could you explain me why, considering the apps installed on my instance, the sources.lists had to be modified, which is something I really did not expect?

What was the wrong action? The manual php7.0-zip installation?

Has an apt install --reinstall on all the alien packages any chance to repair something?

EDIT: apt install --reinstall openssl is « […] impossible, il ne peut pas être téléchargé »…

Aleks · October 30, 2019, 2:56pm

There was nothing wrong done on your side, it’s just caused by this ugly trick that wasnt pushed/fixed fast enough, combined with the fact that you upgraded the php stuff before we release the fix, and the fact that you ran some app operations in that window, … If you want the full story about this, it’s pretty technical and would make a good cyber-horror story for halloween : ynh_install_app_dependencies fails silently to install all dependencies ... ? / Sury repo fucks up php7.0 deps · Issue #1407 · YunoHost/issues · GitHub.

TL;DR : system administration is a nightmare because it’s fragile as hell in its current paradigm.

That’s assuming there’s actually something broken … But right now you are just afraid that these packages broke something, which is not necessarily the case …

You could try to downgrade the packages if you really have to, but that’s a nightmare as well.

Just don’t fix stuff unless you really have indication that it’s actually broken …

sytnfh · October 30, 2019, 3:09pm

~~Two~~Three last questions ;o):

should I keep the Sury repo in my sources.list or mute it? How reliable are the binaries from this repo?
would some apt pinning be useful?
would such a “contamination” be worth a full reinstallation?

Many thanks!

Aleks · October 30, 2019, 3:28pm

Yes you should, otherwise you won’t receive any upgrade from Debian because the Sury’s package (now installed on your server) appear more “up to date” than those in Debian’s repo (because of the version number)

Also if an app needs new php dependecy not yet installed, it won’t be able to install them if you don’t have the sury repo (which a consequence of the previous paragraph, though not obvious, but if you dig into the technical details, that’s what happens, c.f. again the github issue if you want all the details…)

I don’t think so

Please no, again, don’t fix something that ain’t broken.

This not “contamination”, this is a “regular” situation in which you would end up if you installed any app that requires php 7.1, 7.2, or 7.3. It may sound dirty because you don’t have any such app installed, but if that ain’t causing any issue then that’s fine too…

I wish as much as you that this kind of stuff could be done without adding an external third-party repo (though managed by the same folk that manages the Debian php dependency, if i remember correctly), but this is what we have now. If you are interested in improving this, then we need humanpower to do some R&D to investigate and implement virtualenv-like solutions for PHP stuff, or (lol) YunoHost could try to migrate to a development cycle where we ship Debian testing by default or whatever. Or we should convince users and app developers to stop with the “brand new shinny stuff effect”. Or convince the Debian maintainers to stop with the “stable-but-old-as-fuck” strategy.

sytnfh · October 30, 2019, 3:48pm

But if keep the repo, won’t the Sury packages always appear newer than Debian’s?

Thank you very much again. Oh, and did I tell you that having my own server at home, owing to Ynh, is something for which I’m very grateful, everyday?

PS. Not sure I would be able to help for the phpenv solution, but I know that there are other ways to help: I will consider them very carefully.

Aleks · October 30, 2019, 4:00pm

It will be the case even if you remove the Sury repo. Because now that the packages are installed, they are of version 7.0.33.10, compared to Debian’s packages which have version 7.0.33.0 (note the 10 vs. 0). So except if Debian suddently release a php package with version 7.0.33.X with X > 10, the package from Sury will still appear as more recent than Debian’s package … (even though some package may be actually newer, but what really matters is the version number …)

The only way would be to ask apt to downgrade the package, but this what we are really talking about would be to downgrade dozens of them, that’s not easy (though doable if you really wanted to …)

sytnfh · October 30, 2019, 4:42pm

OK, you convinced me to keep things as they are.

But keeping the Sury’s packages means that my instance will not benefit for the security updates offered by Debian/oldstable. This could be questionable, isn’t it?

And when Ynh will switch to Buster (;o), finger crossing might not be enough?

Aleks · October 31, 2019, 12:29am

Yes and no, since the Sury repo also receive upgrades … since it’s a custom repo managed by a third party, - depending on that third party - upgrades in case of security stuff like what happened with the recent CVE can be slower or faster than Debian’s … My guess is that in the case of Sury it happens faster, but that’s really based on just a feeling.

Since the use of Sury is now so common, we’ll make sure to test what happens in such case

rockyIII · November 1, 2019, 10:58am

Hi @Aleks
For an update of a Yunohost App that needs PHP 7.1 I´m waiting and hoping since quid some time that Yunohost supports PHP 7.1 … when do you think this will be the case? or is there a workaround up to than? THANKS

SohKa · November 1, 2019, 11:05am

Hi @rockyIII
It will be the case when Yunohost will support Debian Buster. So in a few months.

system · November 16, 2019, 11:10am

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.