More Migration to V11 Problems - APT cache but maybe more?

[cyberpunklibrarian]

My YunoHost server

Hardware: 2012 Mac mini running Yunohost in VirtualBox 6.1.36 at home. Provisioned with a single processor and 8GB of RAM.
YunoHost version: 4.4.2.5
I have access to my server : Through SSH, webadmin, via keyboard and screen
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : No

Description of my issue

I know this is going around, with migration issues and it being connected to the APT cache. I’ve tried everything over here in this thread:

Migration to YunoHost 11 fails / APT cache update problem?

My YunoHost server Hardware: Kimsufi KS4 YunoHost version: 4.4.2.9 (stable) Description of my issue Error when trying to migrate to Debian Bullseye et YunoHost 11.x. https://paste.yunohost.org/raw/qaziqamane I’ve tried what is explained here with no success: Migration to 11 won't start : "libc6-dev : Breaks: libgcc-8-dev" - #43 by Aleks Thanks for your help.

But so far, I keep hitting the wall. Something I haven’t come across while searching the forum are the key verification problems I’m seeing. I’ve been running into those for a while now, but it hasn’t been a problem until recently. The log file share is below but, one note: I ran the Yunohost upgrade before beginning the migration. When things didn’t work out, I reverted to a back up. So I’m currently running 4.4.2.5 even though the log files say I’m running 4.4.2.10.

ended_at: 2022-08-22 11:22:28.209114
error: "Migration 0021_migrate_to_bullseye did not complete, aborting. Error: Unable\
  \ to update the cache of APT (Debian's package manager). Here is a dump of the sources.list\
  \ lines, which might help identify problematic lines: \nsources.list:deb http://ftp.debian.org/debian\
  \ bullseye main contrib\nsources.list:deb-src http://ftp.debian.org/debian bullseye\
  \ main contrib\nsources.list:deb http://forge.yunohost.org/debian/ bullseye stable\n\
  sources.list:deb http://security.debian.org/debian-security bullseye-security main\
  \ contrib non-free\nsources.list:deb-src http://security.debian.org/debian-security\
  \ bullseye-security main contrib non-free\nsources.list:deb http://ftp.debian.org/debian\
  \ bullseye-updates main contrib non-free\nsources.list:deb-src http://ftp.debian.org/debian\
  \ bullseye-updates main contrib non-free\nsources.list.d/extra_php_version.list:deb\
  \ https://packages.sury.org/php/ bullseye main"
interface: cli
operation: tools_migrations_migrate_forward
parent: null
started_at: 2022-08-22 11:22:19.414257
success: false
yunohost_version: 4.4.2.10

============

2022-08-22 06:22:19,424: INFO - Running migration 0021_migrate_to_bullseye...
2022-08-22 06:22:19,441: INFO - Starting migration to Bullseye
2022-08-22 06:22:20,935: INFO - Patching the sources.lists...
2022-08-22 06:22:24,068: INFO - Fetching available upgrades for system packages...
2022-08-22 06:22:24,376: DEBUG - Hit:1 http://security.debian.org/debian-security bullseye-security InRelease
2022-08-22 06:22:24,376: DEBUG - Hit:2 http://ftp.debian.org/debian bullseye InRelease
2022-08-22 06:22:24,425: DEBUG - Hit:3 http://ftp.debian.org/debian bullseye-updates InRelease
2022-08-22 06:22:24,775: DEBUG - Hit:4 http://forge.yunohost.org/debian bullseye InRelease
2022-08-22 06:22:25,036: DEBUG - Get:5 https://packages.sury.org/php bullseye InRelease [6841 B]
2022-08-22 06:22:25,446: DEBUG - Err:5 https://packages.sury.org/php bullseye InRelease
2022-08-22 06:22:25,447: DEBUG -   The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B188E2B695BD4743
2022-08-22 06:22:27,123: DEBUG - Reading package lists...
2022-08-22 06:22:27,184: WARNING - W: GPG error: https://packages.sury.org/php bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B188E2B695BD4743
2022-08-22 06:22:27,189: WARNING - E: The repository 'https://packages.sury.org/php bullseye InRelease' is not signed.
2022-08-22 06:22:28,191: ERROR - Migration 0021_migrate_to_bullseye did not complete, aborting. Error: Unable to update the cache of APT (Debian's package manager). Here is a dump of the sources.list lines, which might help identify problematic lines: 
sources.list:deb http://ftp.debian.org/debian bullseye main contrib
sources.list:deb-src http://ftp.debian.org/debian bullseye main contrib
sources.list:deb http://forge.yunohost.org/debian/ bullseye stable
sources.list:deb http://security.debian.org/debian-security bullseye-security main contrib non-free
sources.list:deb-src http://security.debian.org/debian-security bullseye-security main contrib non-free
sources.list:deb http://ftp.debian.org/debian bullseye-updates main contrib non-free
sources.list:deb-src http://ftp.debian.org/debian bullseye-updates main contrib non-free
sources.list.d/extra_php_version.list:deb https://packages.sury.org/php/ bullseye main
Traceback (most recent call last):
  File "/usr/lib/moulinette/yunohost/tools.py", line 944, in tools_migrations_run
    migration.run()
  File "/usr/lib/moulinette/yunohost/data_migrations/0021_migrate_to_bullseye.py", line 121, in run
    tools_update(target="system")
  File "/usr/lib/moulinette/yunohost/tools.py", line 385, in tools_update
    raise YunohostError(
yunohost.utils.error.YunohostError: Unable to update the cache of APT (Debian's package manager). Here is a dump of the sources.list lines, which might help identify problematic lines: 
sources.list:deb http://ftp.debian.org/debian bullseye main contrib
sources.list:deb-src http://ftp.debian.org/debian bullseye main contrib
sources.list:deb http://forge.yunohost.org/debian/ bullseye stable
sources.list:deb http://security.debian.org/debian-security bullseye-security main contrib non-free
sources.list:deb-src http://security.debian.org/debian-security bullseye-security main contrib non-free
sources.list:deb http://ftp.debian.org/debian bullseye-updates main contrib non-free
sources.list:deb-src http://ftp.debian.org/debian bullseye-updates main contrib non-free
sources.list.d/extra_php_version.list:deb https://packages.sury.org/php/ bullseye main

https://paste.yunohost.org/raw/ziriwokase

[cyberpunklibrarian]

Okay, had another go at it today, and got the migration to work! Here’s what got it done for me, in case anyone else is having issues:

First things first, that damn sury.org key was the biggest hold up for the process. Over on this thread, @Aleks dropped the knowledge that helped me fix that.

Migration to YunoHost 11 fails / APT cache update problem?

This ain’t your issue The issue is : 2022-08-21 15:18:34,538: WARNING - E: The repository 'https://last-public-ovh-rtm.snap.mirrors.ovh.net/debian bullseye Release' does not have a Release file. 2022-08-21 15:18:34,540: WARNING - E: The repository 'https://last-public-ovh-metrics.snap.mirrors.ovh.net/debian bullseye Release' does not have a Release file. 2022-08-21 15:18:34,543: WARNING - W: GPG error: https://packages.sury.org/php bullseye InRelease: The following signatures were invalid: EX…

You can solve the sury php issue with:

wget --timeout 900 --quiet "https://packages.sury.org/php/apt.gpg" --output-document=- | gpg --dearmor | sudo tee -a "/etc/apt/trusted.gpg.d/extra_php_version.gpg" >/dev/null

After that, I ran the migration. I prefer to do all of this from the command line because I notice that, sometimes, the web UI can be a bit janky and timeout. The CLI is more stable. As expected, the terminal filled with a bunch of text as all the new packages, apps, and goodies came down and installed. But… I ran into another issue. This time, GRUB decided to play the fool and I’m pretty sure I know why. When I first gave Yunohost a try, I installed it on a VirtualBox VM with a fixed storage space. After I decided that I like Yunohost (a lot) I basically cloned it over to a VM with a dynamic storage space. That was dandy, except by doing so, I brought over the old VirtualBox drive name.

Now, that was never a problem… until I needed to update GRUB. And then GRUB wouldn’t install because the drive it wants to install to is gone.

SadPanda

But after searching the Forum for a bit, I found that @tomas ran into a very similar issue. I followed those instructions, making sure to replace his hard drive with the name of my VM HARDDRIVE.

Update Errors, also on fix commands!

Sure, First I tried to see my old and new HDD sudo debconf-show grub-pc | grep -wi install_devices | awk -F: '{print $2} but I only got the old one /dev/disk/by-id/ata-OCZ_VERTEX_PLUS_19JE1KI41W61AY8PEP3Y Then I checked the new HDD by listing $ cd /dev/disk/by-id/ /dev/disk/by-id$ ls ata-ST9250315AS_6VC0JAYW Then I changed it to the new one echo grub-pc grub-pc/install_devices multiselect /dev/disk/by-id/ata-ST9250315AS_6VC0JAYW | sudo debconf-set-selections $ sudo DEBIAN_FRONTEND=…

Disco!

After that I:

sudo apt update

And the updates to GRUB that were being held back were held back no longer. Ran:

sudo apt upgrade

After the upgrades were completed I retried:

sudo yunohost tools migrations run --accept-disclaimer

That ran through the process of downloading and installing the missing bits that didn’t come down after the first migration attempt. GRUB installation didn’t complain and the migration completed!

Big thanks to @Aleks and @tomas for the assistance!

[Aleks]

Thanks for the super detailed feedback !

I’ll add a patch for the sury key at the beginning of the migration because that seems to happen way too often …

[cyberpunklibrarian]

Nah worries!

While searching for solutions I found that I was having bits and pieces of issues that others were having, but nothing was coalescing into the exact problem with the migration. The sury key issue is all over other forums, but none of them had your solution. The GRUB problem… that one was my fault!