Borg 1.4.1~ynh4 released, what you should check after the upgrade

:united_kingdom: English version

(:france: French version below)

Version 1.4.1~ynh4 has just been released.

It fixes a serious bug in 1.4.1~ynh3: when the passphrase contains ampersands (&) leading it not being correctly written in /var/www/__APP__/.env, and therefore making either backups creation fail or repository initiated with the wrong passphrase [1].

In other word, there are three cases:

  1. If your passphrase does not contain any ampersand (&): you are probably good :white_check_mark:. You may still upgrade to 1.4.1~ynh4 just to keep track with the release versions.
  2. If your passphrase contains at least one ampersand, but your initial borg installation has been done before 1.4.1~ynh3, you are probably experiencing issues with backuping[2], you should upgrade yunohost to version 12.1.28 and upgrade borg to version 1.4.1~ynh4 and make the checks below
  3. If your passphrase contains at least one ampersand, AND your initial borg installation has been using version 1.4.1~ynh3, then you probably need to fix the passphrase of your repository, see below

Foreword

Let’s remind that this work is driven by the work of volunteers. We do care about providing a reliable service. That being said, it is important to remind you have agreed on the terms of service and especially to this part:

YunoHost as software is released under the AGPLv3 license and is therefore provided without warranty of any kind and is not liable for any damages resulting from its use, nor from the use of the applications it allows to install.

So does this post, and its author is not liable for any possible damages.

That’s the reason why you are invited to fully read from the start to the end and understand the below steps before running the commands.

The upgrade

Follow these two steps:

  1. Upgrade your system, this will upgrade Yunohost to version 12.1.28
  2. Upgrade all your borg apps (borg and not borgserver which is another package)

The post-upgrade checks and fix

After the upgrade, open a shell session, log in as root, and retrieve all the borg instance IDs using the following command:

yunohost app list | grep -Po 'id: \Kborg(\b|__\d+)'

:backhand_index_pointing_down: The following steps have to be run for each borg instance: :backhand_index_pointing_down:

Then let’s check whether the passphrase is now correctly set in the system:

app_id=borg # change the value borg__2, borg__3, ... to check 
            # the other borg instances as returned by the command above
/var/www/$app_id/wrapper/borg list

:raised_hand: If the command succeeds in listing your archives, STOP HERE, and repeat the commands above for the next borg instance :raised_hand:

But if the command fails with this error:

passphrase supplied in BORG_PASSPHRASE, by BORG_PASSCOMMAND or via BORG_PASSPHRASE_FD is incorrect.

:warning: Then first and foremost, be sure that you’re passphrase actually contains an ampersand:

grep BORG_PASSPHRASE /var/www/$app_id/.env | grep --color '&' && echo "The passphrase contains an ampersand, run the fix" || echo "Do NOT run the fix, that's another problem"

If the command above prints your passphrase correctly, then your borg repository has probably been initiated with a wrong one, having & replaced with __PASSPHRASE__ (see [1:1] for more info).

Let’s print the command that will be run to change the passphrase with the command below:

export app_id
(set -a; source /var/www/$app_id/.env; set +a; echo BORG_NEW_PASSPHRASE="${BORG_PASSPHRASE}" BORG_PASSPHRASE="${BORG_PASSPHRASE//&/__PASSPHRASE__}"  /var/www/$app_id/venv/bin/borg key change-passphrase)

Inspect the result and when you’re ready, remove the call to echo in the command above, which will make the change effective.

If you do NOT see any output, then the command should have been run successfully.

Check now that borg is able to list your backups successfully:

/var/www/$app_id/wrapper/borg list

Don’t forget to repeat the check and (if applicable) the fix for the other instances.

And you’re done! :tada:

:france: French version

Une nouvelle version 1.4.1~ynh4 du paquet est maintenant disponible.

Elle corrige un bug important prĂ©sent dans la version 1.4.1~ynh3 : lorsque la phrase de passe (passphrase ou mot de passe si vous prĂ©fĂ©rez) contient des esperluette (&), elle n’est pas correctement restituĂ©e dans le fichier /var/www/__APP__/.env, ce qui entraĂźne l’échec des sauvegardes ou l’initialisation d’un dĂ©pĂŽt borg avec la mauvaise phrase de passe [1:2].

En d’autres termes, il y a trois cas :

  1. Si votre phrase de passe ne contient aucune esperluette (&) : vous n’avez probablement rien à faire :white_check_mark:. Vous pouvez toujours mettre à niveau vers 1.4.1~ynh4 juste pour suivre les versions de sortie.
  2. Si votre phrase de passe contient au moins une esperluette, mais que votre installation initiale de Borg a été effectuée avant la 1.4.1~ynh3, vous rencontrez probablement des problÚmes de sauvegarde[2:1]. Vous devez mettre à niveau Yunohost vers la version 12.1.28 et mettre à niveau Borg vers la version 1.4.1~ynh4 et effectuer les vérifications ci-dessous.
  3. Si votre phrase de passe contient au moins une esperluette, ET que votre installation initiale de Borg a Ă©tĂ© rĂ©alisĂ©e Ă  l’aide de la version 1.4.1~ynh3, vous devrez probablement corriger la phrase de passe de votre dĂ©pĂŽt, voir ci-dessous.

# Préambule

Nous tenons Ă  rappeler que ce travail est menĂ© par des bĂ©nĂ©voles. Nous nous soucions de fournir un service fiable. Cela Ă©tant dit, il est important de rappeler que vous avez acceptĂ© les conditions d’utilisation et en particulier ce paragraphe :

YunoHost en tant que logiciel est publiĂ© sous licence AGPLv3 et est donc fourni sans garantie d’aucune sorte et n’est pas responsable des dommages rĂ©sultant de son utilisation, ni de l’utilisation des applications qu’il permet d’installer.

De la mĂȘme maniĂšre, dans ce message, l’auteur n’est pas responsable des Ă©ventuels dommages.

C’est pourquoi vous ĂȘtes invité·es Ă  lire attentivement ce post du dĂ©but Ă  la fin et Ă  comprendre les Ă©tapes ci-dessous avant d’exĂ©cuter les commandes.

# La mise Ă  niveau

Suivez ces deux étapes :

  1. Mettez Ă  niveau votre systĂšme, ce qui mettra Ă  niveau Yunohost vers la version 12.1.28
  2. Mettez à niveau toutes vos applications Borg (Borg et non Borgserver, qui est un paquet différents)

Les vérifications et la correction post-mise à niveau

AprĂšs la mise Ă  niveau, ouvrez une session shell, connectez-vous en tant que root et rĂ©cupĂ©rez tous les ID d’instance Borg Ă  l’aide de la commande suivante :

yunohost app list | grep -Po 'id: \Kborg(\b|__\d+)'

:backhand_index_pointing_down: Les Ă©tapes ci-dessous doivent ĂȘtre exĂ©cutĂ©es pour chaque instance Borg : :backhand_index_pointing_down:

Vérifions ensuite si la phrase de passe est maintenant correctement définie dans le systÚme :

app_id=borg # change the value borg__2, borg__3, ... to check
            # the other borg instances as returned by the command above
/var/www/$app_id/wrapper/borg list

:raised_hand: Si la commande parvient Ă  lister vos sauvegardes, ARRÊTEZ-VOUS ICI, et rĂ©pĂ©tez les commandes ci-dessus pour la prochaine instance Borg :raised_hand:

Mais si la commande échoue avec cette erreur :

passphrase supplied in BORG_PASSPHRASE, by BORG_PASSCOMMAND or via BORG_PASSPHRASE_FD is incorrect.

:warning: Alors, assurez-vous d’abord que votre phrase de passe contient bien une esperluette :

grep BORG_PASSPHRASE /var/www/$app_id/.env | grep --color '&' && echo "The passphrase contains an ampersand, run the fix" || echo "Do NOT run the fix, that's another problem"

Si la commande affiche votre phrase de passe correctement, alors votre dĂ©pĂŽt Borg a probablement Ă©tĂ© initialisĂ© avec une mauvaise, ayant & remplacĂ© par __PASSPHRASE__ (voir [1:3] pour plus d’informations).

Affichons maintenant la commande qui sera exécutée pour modifier la phrase de passe :

export app_id
(set -a; source /var/www/$app_id/.env; set +a; echo BORG_NEW_PASSPHRASE="${BORG_PASSPHRASE}" BORG_PASSPHRASE="${BORG_PASSPHRASE//&/__PASSPHRASE__}"  /var/www/$app_id/venv/bin/borg key change-passphrase)

Inspectez le rĂ©sultat et, lorsque vous ĂȘtes prĂȘt·e, supprimez l’appel Ă  echo dans la commande que vous venez d’exĂ©cuter, ce qui rendra le changement effectif.

Si Ă  l’exĂ©cution de la commande aucun retour ne vous est affichĂ©, alors la commande a dĂ» s’exĂ©cuter avec succĂšs.

Vérifiez maintenant que Borg est capable de lister vos sauvegardes correctement :

/var/www/$app_id/wrapper/borg list

N’oubliez pas de rĂ©pĂ©ter le contrĂŽle et (si nĂ©cessaire) la correction pour les autres instances.

Et voilĂ  ! :tada:


  1. See Passphrase not recognized when it contains ampersands (`&`) · Issue #231 · YunoHost-Apps/borg_ynh · GitHub for a more detailed explanation ↩ ↩ ↩ ↩

  2. You should probably this message in /var/log/borg/borg.log: passphrase supplied in BORG_PASSPHRASE, by BORG_PASSCOMMAND or via BORG_PASSPHRASE_FD is incorrect. ↩ ↩

5 Likes

Post-mortem of the bug

This post is now about why this bug has happened. If you’re a just a user of this pacakge, you may skip this post, unless if of course you’re curious to know more.

The version 1.4.1~ynh3 of the package has changed the way the passphrase is retrieved.
Prior to this version, the passphrase used to be retrieved by directly getting it from the settings:

But now starting with ~ynh3, we retrieve it by sourcing an .env config file:

Where the __PASSPHRASE__ value is (or should be) replaced by the actual password as read from the settings.

This .env file is convenient for running the borg command without having to specify the long list of env variables, see this feature request that has been implemented for version ynh3.

BUT that’s at this point that the unexpected regression appears! Because Yunohost had this bug already known and reported:

For usual apps, that’s not a very serious bug, as passwords are simply a way to access an app. It’s easy to circumvent the issue by just setting another password and that’s it :person_shrugging:.

But when dealing with apps like borg, in which the passphrase is used to encrypt and decrypt the archives, it is actually a big deal, especially when the bad passphrase is used for initiating the borg repo.

Under the hood, the templates are hydrated by using the sed command, and because the ampersand (&) is used by this command to print the matched substitution. For example:

echo "Yunohosters" | sed 's/.*/Hello &/g'

Will print Hello Yunohosters.

The fix has just consisted in escaping every occurrences of &:

That’s the reason why in order to fix this issue, you have to upgrade both Yunohost and less importantly borg (which just regenerates the .env file to have the correct passphrase when Yunohost is upgraded).

6 Likes

Thanks for your work!

When I try this, I got an error :

# app_id=borg 
# /var/www/$app_id/wrapper/borg list
Got unexpected RPC data format from server:

(Details here if you have an idea to solve the problem.)