Borg backup failed cannot execute hooks.d/backup_method/05-borg_app

Cellophile · December 15, 2021, 2:05pm

My YunoHost server

Hardware: Old laptop or computer
YunoHost version: 4.3.4.2
I have access to my server : Through SSH | through the webadmin | direct access via keyboard / screen | …
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : no

Description of my issue

Hello,
I’ve installed borg two days ago. First backup has been done successfully. This morning, I’ve a fail message, just for some app backup, not for all.



The backup miserably failed to backup system configurations.
The backup miserably failed to backup system data.
The backup miserably failed to backup borg application.
The backup miserably failed to backup borgserver application.
The backup miserably failed to backup roundcube application.
The backup miserably failed to backup roundcube__2 application.
The backup miserably failed to backup roundcube__3 application.



Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 323.9KiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130148-backup_create'
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 772.3MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130204-backup_create'
Collecte des fichiers devant être sauvegardés pour borg...
Loading installation settings...
Declaring files to be backed up...
Backup script completed for borg. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 104.2KiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130221-backup_create'
Collecte des fichiers devant être sauvegardés pour borgserver...
Loading installation settings...
Declaring files to be backed up...
Borg backup repo in /home/backupVPS/ won't be backup to avoid backup of backup loop issue.
Backup script completed for borgserver. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 59.4KiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130233-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 92.1MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130243-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube__2...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube__2. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 99.6MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130256-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube__3...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube__3. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 100.1MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130310-backup_create'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Source path '/root/.ssh/id_borg_ed25519' does not exist
Source path '/root/.ssh/id_borg_ed25519.pub' does not exist
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'

If I do `systemctl status borg.service`, I have

borg.service - Run backup borg
Loaded: loaded (/etc/systemd/system/borg.service; static; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2021-12-15 14:43:21 CET; 4min 52s ago
Process: 32435 ExecStart=/usr/bin/sudo /usr/local/bin/backup-with-borg borg (code=exited, status=1/FAILURE)
Main PID: 32435 (code=exited, status=1/FAILURE)

déc. 15 14:43:20 domain.tld.fr sudo[4244]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/yunohost app
déc. 15 14:43:20 domain.tld.fr sudo[4244]: pam_unix(sudo:session): session opened for user root by (uid=0)
déc. 15 14:43:21 domain.tld.fr sudo[4244]: pam_unix(sudo:session): session closed for user root
déc. 15 14:43:21 domain.tld.fr sudo[4246]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/yunohost app
déc. 15 14:43:21 domain.tld.fr sudo[4246]: pam_unix(sudo:session): session opened for user root by (uid=0)
déc. 15 14:43:21 domain.tld.fr sudo[4246]: pam_unix(sudo:session): session closed for user root
déc. 15 14:43:21 domain.tld.fr sudo[32435]: pam_unix(sudo:session): session closed for user root
déc. 15 14:43:21 domain.tld.fr systemd[1]: borg.service: Main process exited, code=exited, status=1/FAILURE
déc. 15 14:43:21 domain.tld.fr systemd[1]: borg.service: Failed with result ‘exit-code’.
déc. 15 14:43:21 domain.tld.fr systemd[1]: Failed to start Run backup borg.
lines 1-16/16 (END)

You can see logs here.

I’ve checked my passphrase with yunohost app setting borg passphrase, and it’s ok.
I’ve tried to start again borg : service borg start
But this time everything is failed…

I don’t know what to do…
for info, hook’s content :

#!/bin/bash

set -eo pipefail
app="${0#"./05-"}"
app="${app%"_app"}"

BORG_PASSPHRASE="$(yunohost app setting $app passphrase)"
repo="$(yunohost app setting $app repository)"   #$4
if ssh-keygen -F "" >/dev/null ; then
    BORG_RSH="ssh -i /root/.ssh/id_${app}_ed25519 -oStrictHostKeyChecking=yes "
else
    BORG_RSH="ssh -i /root/.ssh/id_${app}_ed25519 -oStrictHostKeyChecking=no "
fi

do_need_mount() {
    true
}

LOGFILE=/var/log/backup_borg.err
log_with_timestamp() {
  sed -e "s/^/[$(date +"%Y-%m-%d_%H:%M:%S")] /" | tee -a $LOGFILE
}

do_backup() {

I don’t do this. But I had another hook, about nextcloud, like this : :

[Resolved] Change where goes Nextcloud after logout

/etc/yunohost/hooks.d/conf_regen/16-nginx_nextcloud

#!/bin/bash
action=$1
pending_dir=$4
nginx_dir=$pending_dir/../nginx/etc/nginx
nextcloud_dir=/var/www/nextcloud/core/Controller
[[ $action == "pre" ]] || exit 0
[[ -d $nginx_dir ]] || exit 0
[[ -e $nextcloud_dir ]] || exit 0

if [[ -e $nextcloud_dir/LoginController.php.orig ]];then
        mv $nextcloud_dir/LoginController.php $nextcloud_dir/LoginController.php.ynhmod &&
        mv $nextcloud_dir/LoginController.php.orig $nextcloud_dir/LoginController.php
fi

And directory conf_regen is disapeared… I don’t understand

Mon serveur YunoHost

Matériel: Vieil ordinateur
Version de YunoHost: 4.3.4.2
J’ai accès à mon serveur : En SSH | Par la webadmin | En direct avec un clavier/écran | …
Êtes-vous dans un contexte particulier ou avez-vous effectué des modificiations particulières sur votre instance ? : non

Description du problème

Bonjour,
J’ai installer borg il y a deux jours. Le premier backup s’est fait correctement. Ce matin par contre, j’ai reçu un mail d’erreur, avec juste quelques applications qui n’ont pas pu être sauvegardées.



The backup miserably failed to backup system configurations.
The backup miserably failed to backup system data.
The backup miserably failed to backup borg application.
The backup miserably failed to backup borgserver application.
The backup miserably failed to backup roundcube application.
The backup miserably failed to backup roundcube__2 application.
The backup miserably failed to backup roundcube__3 application.



Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 323.9KiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130148-backup_create'
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 772.3MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130204-backup_create'
Collecte des fichiers devant être sauvegardés pour borg...
Loading installation settings...
Declaring files to be backed up...
Backup script completed for borg. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 104.2KiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130221-backup_create'
Collecte des fichiers devant être sauvegardés pour borgserver...
Loading installation settings...
Declaring files to be backed up...
Borg backup repo in /home/backupVPS/ won't be backup to avoid backup of backup loop issue.
Backup script completed for borgserver. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 59.4KiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130233-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 92.1MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130243-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube__2...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube__2. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 99.6MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130256-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube__3...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube__3. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 100.1MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211215-130310-backup_create'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Source path '/root/.ssh/id_borg_ed25519' does not exist
Source path '/root/.ssh/id_borg_ed25519.pub' does not exist
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'

Vous pouvez voir les logs ici.

J’ai vérifié ma passphrase avec yunohost app setting borg passphrase, et elle est juste et entière.
J’ai essayé de relancer borg : service borg start
Mais cette fois tout a échoué…

Si je fais systemctl status borg.service, j’obtiens

 borg.service - Run backup borg
   Loaded: loaded (/etc/systemd/system/borg.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2021-12-15 14:43:21 CET; 4min 52s ago
  Process: 32435 ExecStart=/usr/bin/sudo /usr/local/bin/backup-with-borg borg (code=exited, status=1/FAILURE)
 Main PID: 32435 (code=exited, status=1/FAILURE)

déc. 15 14:43:20 domain.tld.fr sudo[4244]:     root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/yunohost app
déc. 15 14:43:20 domain.tld.fr sudo[4244]: pam_unix(sudo:session): session opened for user root by (uid=0)
déc. 15 14:43:21 domain.tld.fr sudo[4244]: pam_unix(sudo:session): session closed for user root
déc. 15 14:43:21 domain.tld.fr sudo[4246]:     root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/yunohost app
déc. 15 14:43:21 domain.tld.fr sudo[4246]: pam_unix(sudo:session): session opened for user root by (uid=0)
déc. 15 14:43:21 domain.tld.fr sudo[4246]: pam_unix(sudo:session): session closed for user root
déc. 15 14:43:21 domain.tld.fr sudo[32435]: pam_unix(sudo:session): session closed for user root
déc. 15 14:43:21 domain.tld.fr systemd[1]: borg.service: Main process exited, code=exited, status=1/FAILURE
déc. 15 14:43:21 domain.tld.fr systemd[1]: borg.service: Failed with result 'exit-code'.
déc. 15 14:43:21 domain.tld.fr systemd[1]: Failed to start Run backup borg.
lines 1-16/16 (END)

Je ne sais pas trop quoi faire…
Juste pour info, le hook contient :

#!/bin/bash

set -eo pipefail
app="${0#"./05-"}"
app="${app%"_app"}"

BORG_PASSPHRASE="$(yunohost app setting $app passphrase)"
repo="$(yunohost app setting $app repository)"   #$4
if ssh-keygen -F "" >/dev/null ; then
    BORG_RSH="ssh -i /root/.ssh/id_${app}_ed25519 -oStrictHostKeyChecking=yes "
else
    BORG_RSH="ssh -i /root/.ssh/id_${app}_ed25519 -oStrictHostKeyChecking=no "
fi

do_need_mount() {
    true
}

LOGFILE=/var/log/backup_borg.err
log_with_timestamp() {
  sed -e "s/^/[$(date +"%Y-%m-%d_%H:%M:%S")] /" | tee -a $LOGFILE
}

do_backup() {

Ce n’est pas moi qui l’ai mis là. Par contre, j’avais fait un Hook sur la proposition de metyun :

[Resolved] Change where goes Nextcloud after logout

/etc/yunohost/hooks.d/conf_regen/16-nginx_nextcloud

#!/bin/bash
action=$1
pending_dir=$4
nginx_dir=$pending_dir/../nginx/etc/nginx
nextcloud_dir=/var/www/nextcloud/core/Controller
[[ $action == "pre" ]] || exit 0
[[ -d $nginx_dir ]] || exit 0
[[ -e $nextcloud_dir ]] || exit 0

if [[ -e $nextcloud_dir/LoginController.php.orig ]];then
        mv $nextcloud_dir/LoginController.php $nextcloud_dir/LoginController.php.ynhmod &&
        mv $nextcloud_dir/LoginController.php.orig $nextcloud_dir/LoginController.php
fi

Et le dossier conf_regen a disparu… Je ne comprends pas ce qui se passe

ljf · December 15, 2021, 5:20pm

Can you give some context about those 2 directories:

/home/backupVPS
/media/svgd

Cellophile · December 15, 2021, 5:40pm

Yes sure !

/home/backupVPS // directory where are stocked backups of another yunohost server, with borgerserver.

/media/svgd // Directory where I stock backups of this server, mount on a external storage.

ljf · December 17, 2021, 12:01pm

You probably should do a

touch /home/backupVPS/.nobackup

Is there some errors in /var/log/backup_borg.err ?

Could you display the content of

ls /media/svgd

Maybe you should inidicate a subdir in /media/svgd like /media/svgd/backup ?

Cellophile · December 17, 2021, 12:43pm

Hello @ljf, and thanks for your answer !

Yes there is one, with this content :

[2021-12-17_00:00:16] Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:00:16] Traceback (most recent call last):
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
[2021-12-17_00:00:16]     exit_code = archiver.run(args)
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
[2021-12-17_00:00:16]     return set_ec(func(args))
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 177, in wrapper
[2021-12-17_00:00:16]     return method(self, args, repository=repository, **kwargs)
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 595, in do_create
[2021-12-17_00:00:16]     create_inner(archive, cache)
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 560, in create_inner
[2021-12-17_00:00:16]     archive.save(comment=args.comment, timestamp=args.timestamp)
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archive.py", line 530, in save
[2021-12-17_00:00:16]     self.repository.commit()
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 475, in commit
[2021-12-17_00:00:16]     self.compact_segments()
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 756, in compact_segments
[2021-12-17_00:00:16]     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1437, in iter_objects
[2021-12-17_00:00:16]     read_data=read_data)
[2021-12-17_00:00:16]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1533, in _read
[2021-12-17_00:00:16]     segment, offset))
[2021-12-17_00:00:16] borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:00:16]
[2021-12-17_00:00:16] Platform: Linux mydomain.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
[2021-12-17_00:00:16] Linux: debian 10.11
[2021-12-17_00:00:16] Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
[2021-12-17_00:00:16] PID: 31753  CWD: /home/yunohost.backup/tmp/auto_conf
[2021-12-17_00:00:16] sys.argv: ['/opt/borg-env/bin/borg', 'create', '/media/svgd::_auto_conf-2021-12-17_00:00', './']
[2021-12-17_00:00:16] SSH_ORIGINAL_COMMAND: None
[2021-12-17_00:00:16]
[2021-12-17_00:00:37] Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:00:37] Traceback (most recent call last):
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
[2021-12-17_00:00:37]     exit_code = archiver.run(args)
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
[2021-12-17_00:00:37]     return set_ec(func(args))
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 177, in wrapper
[2021-12-17_00:00:37]     return method(self, args, repository=repository, **kwargs)
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 595, in do_create
[2021-12-17_00:00:37]     create_inner(archive, cache)
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 560, in create_inner
[2021-12-17_00:00:37]     archive.save(comment=args.comment, timestamp=args.timestamp)
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archive.py", line 530, in save
[2021-12-17_00:00:37]     self.repository.commit()
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 475, in commit
[2021-12-17_00:00:37]     self.compact_segments()
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 756, in compact_segments
[2021-12-17_00:00:37]     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1437, in iter_objects
[2021-12-17_00:00:37]     read_data=read_data)
[2021-12-17_00:00:37]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1533, in _read
[2021-12-17_00:00:37]     segment, offset))
[2021-12-17_00:00:37] borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:00:37]
[2021-12-17_00:00:37] Platform: Linux mydomain.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
[2021-12-17_00:00:37] Linux: debian 10.11
[2021-12-17_00:00:37] Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
[2021-12-17_00:00:37] PID: 32606  CWD: /home/yunohost.backup/tmp/auto_data
[2021-12-17_00:00:37] sys.argv: ['/opt/borg-env/bin/borg', 'create', '/media/svgd::_auto_data-2021-12-17_00:00', './']
[2021-12-17_00:00:37] SSH_ORIGINAL_COMMAND: None
[2021-12-17_00:00:37]
[2021-12-17_00:01:31] Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:01:31] Traceback (most recent call last):
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
[2021-12-17_00:01:31]     exit_code = archiver.run(args)
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
[2021-12-17_00:01:31]     return set_ec(func(args))
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 177, in wrapper
[2021-12-17_00:01:31]     return method(self, args, repository=repository, **kwargs)
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 595, in do_create
[2021-12-17_00:01:31]     create_inner(archive, cache)
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 560, in create_inner
[2021-12-17_00:01:31]     archive.save(comment=args.comment, timestamp=args.timestamp)
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archive.py", line 530, in save
[2021-12-17_00:01:31]     self.repository.commit()
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 475, in commit
[2021-12-17_00:01:31]     self.compact_segments()
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 756, in compact_segments
[2021-12-17_00:01:31]     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1437, in iter_objects
[2021-12-17_00:01:31]     read_data=read_data)
[2021-12-17_00:01:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1533, in _read
[2021-12-17_00:01:31]     segment, offset))
[2021-12-17_00:01:31] borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:01:31]
[2021-12-17_00:01:31] Platform: Linux mydomain.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
[2021-12-17_00:01:31] Linux: debian 10.11
[2021-12-17_00:01:31] Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
[2021-12-17_00:01:31] PID: 996  CWD: /home/yunohost.backup/tmp/auto_borg
[2021-12-17_00:01:31] sys.argv: ['/opt/borg-env/bin/borg', 'create', '/media/svgd::_auto_borg-2021-12-17_00:01', './']
[2021-12-17_00:01:31] SSH_ORIGINAL_COMMAND: None
[2021-12-17_00:01:31]
[2021-12-17_00:01:41] Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:01:41] Traceback (most recent call last):
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
[2021-12-17_00:01:41]     exit_code = archiver.run(args)
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
[2021-12-17_00:01:41]     return set_ec(func(args))
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 177, in wrapper
[2021-12-17_00:01:41]     return method(self, args, repository=repository, **kwargs)
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 595, in do_create
[2021-12-17_00:01:41]     create_inner(archive, cache)
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 560, in create_inner
[2021-12-17_00:01:41]     archive.save(comment=args.comment, timestamp=args.timestamp)
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archive.py", line 530, in save
[2021-12-17_00:01:41]     self.repository.commit()
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 475, in commit
[2021-12-17_00:01:41]     self.compact_segments()
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 756, in compact_segments
[2021-12-17_00:01:41]     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1437, in iter_objects
[2021-12-17_00:01:41]     read_data=read_data)
[2021-12-17_00:01:41]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1533, in _read
[2021-12-17_00:01:41]     segment, offset))
[2021-12-17_00:01:41] borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:01:41]
[2021-12-17_00:01:41] Platform: Linux mydomain.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
[2021-12-17_00:01:41] Linux: debian 10.11
[2021-12-17_00:01:41] Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
[2021-12-17_00:01:41] PID: 1502  CWD: /home/yunohost.backup/tmp/auto_borgserver
[2021-12-17_00:01:41] sys.argv: ['/opt/borg-env/bin/borg', 'create', '/media/svgd::_auto_borgserver-2021-12-17_00:01', './']
[2021-12-17_00:01:41] SSH_ORIGINAL_COMMAND: None
[2021-12-17_00:01:41]
[2021-12-17_00:01:50] Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:01:50] Traceback (most recent call last):
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
[2021-12-17_00:01:50]     exit_code = archiver.run(args)
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
[2021-12-17_00:01:50]     return set_ec(func(args))
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 177, in wrapper
[2021-12-17_00:01:50]     return method(self, args, repository=repository, **kwargs)
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 595, in do_create
[2021-12-17_00:01:50]     create_inner(archive, cache)
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 560, in create_inner
[2021-12-17_00:01:50]     archive.save(comment=args.comment, timestamp=args.timestamp)
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archive.py", line 530, in save
[2021-12-17_00:01:50]     self.repository.commit()
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 475, in commit
[2021-12-17_00:01:50]     self.compact_segments()
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 756, in compact_segments
[2021-12-17_00:01:50]     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1437, in iter_objects
[2021-12-17_00:01:50]     read_data=read_data)
[2021-12-17_00:01:50]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1533, in _read
[2021-12-17_00:01:50] borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:01:50]
[2021-12-17_00:01:50] Platform: Linux mydomain.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
[2021-12-17_00:01:50] Linux: debian 10.11
[2021-12-17_00:01:50] Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
[2021-12-17_00:01:50] PID: 2015  CWD: /home/yunohost.backup/tmp/auto_roundcube
[2021-12-17_00:01:50] sys.argv: ['/opt/borg-env/bin/borg', 'create', '/media/svgd::_auto_roundcube-2021-12-17_00:01', './']
[2021-12-17_00:01:50] SSH_ORIGINAL_COMMAND: None
[2021-12-17_00:01:50]
[2021-12-17_00:02:10] Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:02:10] Traceback (most recent call last):
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
[2021-12-17_00:02:10]     exit_code = archiver.run(args)
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
[2021-12-17_00:02:10]     return set_ec(func(args))
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 177, in wrapper
[2021-12-17_00:02:10]     return method(self, args, repository=repository, **kwargs)
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 595, in do_create
[2021-12-17_00:02:10]     create_inner(archive, cache)
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 560, in create_inner
[2021-12-17_00:02:10]     archive.save(comment=args.comment, timestamp=args.timestamp)
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archive.py", line 530, in save
[2021-12-17_00:02:10]     self.repository.commit()
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 475, in commit
[2021-12-17_00:02:10]     self.compact_segments()
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 756, in compact_segments
[2021-12-17_00:02:10]     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1437, in iter_objects
[2021-12-17_00:02:10]     read_data=read_data)
[2021-12-17_00:02:10]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1533, in _read
[2021-12-17_00:02:10]     segment, offset))
[2021-12-17_00:02:10] borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 68, of
fset 833]
[2021-12-17_00:02:10]
[2021-12-17_00:02:10] Platform: Linux mydomain.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
[2021-12-17_00:02:10] Linux: debian 10.11
[2021-12-17_00:02:10] Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
[2021-12-17_00:02:10] PID: 2562  CWD: /home/yunohost.backup/tmp/auto_roundcube__2
[2021-12-17_00:02:10] sys.argv: ['/opt/borg-env/bin/borg', 'create', '/media/svgd::_auto_roundcube__2-2021-12-17_00:02',
 './']
[2021-12-17_00:02:10] SSH_ORIGINAL_COMMAND: None
[2021-12-17_00:02:10]
[2021-12-17_00:02:31] Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:02:31] Traceback (most recent call last):
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
[2021-12-17_00:02:31]     exit_code = archiver.run(args)
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
[2021-12-17_00:02:31]     return set_ec(func(args))
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 177, in wrapper
[2021-12-17_00:02:31]     return method(self, args, repository=repository, **kwargs)
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 595, in do_create
[2021-12-17_00:02:31]     create_inner(archive, cache)
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 560, in create_inner
[2021-12-17_00:02:31]     archive.save(comment=args.comment, timestamp=args.timestamp)
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/archive.py", line 530, in save
[2021-12-17_00:02:31]     self.repository.commit()
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 475, in commit
[2021-12-17_00:02:31]     self.compact_segments()
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 756, in compact_segments
[2021-12-17_00:02:31]     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1437, in iter_objects
[2021-12-17_00:02:31]     read_data=read_data)
[2021-12-17_00:02:31]   File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1533, in _read
[2021-12-17_00:02:31]     segment, offset))
[2021-12-17_00:02:31] borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]
[2021-12-17_00:02:31]
[2021-12-17_00:02:31] Platform: Linux mydomain.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
[2021-12-17_00:02:31] Linux: debian 10.11
[2021-12-17_00:02:31] Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
[2021-12-17_00:02:31] PID: 3017  CWD: /home/yunohost.backup/tmp/auto_roundcube__3
[2021-12-17_00:02:31] sys.argv: ['/opt/borg-env/bin/borg', 'create', '/media/svgd::_auto_roundcube__3-2021-12-17_00:02', './']
[2021-12-17_00:02:31] SSH_ORIGINAL_COMMAND: None
[2021-12-17_00:02:31]

What the data integrity error says ?

Data integrity error: Segment entry checksum mismatch [segment 68, offset 833]

I’ve tried command touch /home/backupVPS/.nobackup, but this looks like this does’nt work, any file is created… I’ve tested with nano /home/backupVPS/.nobackup with “test” word in. I don’t see file, but if I redo nano /home/backupVPS/.nobackup, there is always my “test” word… Why doesn’t it appear with a ls -l command ?

And there is the content of /media/svgd:

# ls -l
total 2672
-rwxr-xr-x 1 root root     700 déc.  14 00:00 config
drwxr-xr-x 3 root root   16384 déc.  14 00:00 data
-rwxr-xr-x 1 root root     461 déc.  17 00:02 hints.137
-rwxr-xr-x 1 root root 2621498 déc.  17 00:02 index.137
-rwxr-xr-x 1 root root     190 déc.  17 00:02 integrity.137
-rwxr-xr-x 1 root root      16 déc.  17 00:02 nonce
-rwxr-xr-x 1 root root      73 déc.  14 00:00 README

ljf · December 17, 2021, 9:31pm

Are you able to list your archives ?

borg list /media/svgd

Cellophile · December 19, 2021, 11:48am

Yes, here is the result :

Data integrity error: Invalid segment entry size 1078413148 - too big [segment 148, offset 240656]
Traceback (most recent call last):
  File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4690, in main
    exit_code = archiver.run(args)
  File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 4622, in run
    return set_ec(func(args))
  File "/opt/borg-env/lib/python3.7/site-packages/borg/archiver.py", line 164, in wrapper
    kwargs['manifest'], kwargs['key'] = Manifest.load(repository, compatibility)
  File "/opt/borg-env/lib/python3.7/site-packages/borg/helpers.py", line 378, in load
    cdata = repository.get(cls.MANIFEST_ID)
  File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1122, in get
    self.index = self.open_index(self.get_transaction_id())
  File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 401, in get_transaction_id
    self.check_transaction()
  File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 398, in check_transaction
    self.replay_segments(replay_from, segments_transaction_id)
  File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 858, in replay_segments
    self._update_index(segment, objects)
  File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 891, in _update_index
    size = self.io.read(s, offset, key, read_data=False)
  File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1497, in read
    size, tag, key, data = self._read(fd, self.put_header_fmt, header, segment, offset, (TAG_PUT, ), read_data)
  File "/opt/borg-env/lib/python3.7/site-packages/borg/repository.py", line 1521, in _read
    size, segment, offset))
borg.helpers.IntegrityError: Data integrity error: Invalid segment entry size 1078413148 - too big [segment 148, offset 240656]

Platform: Linux mondomaine.fr 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64
Linux: debian 10.11
Borg: 1.1.16  Python: CPython 3.7.3 msgpack: 0.5.6.+borg1
PID: 791  CWD: /home/admin
sys.argv: ['/opt/borg-env/bin/borg', 'list', '/media/svgd']
SSH_ORIGINAL_COMMAND: None

Cellophile · December 19, 2021, 11:49am

can it be a solution to empty the directory /media/svgd ?

ljf · December 21, 2021, 9:54am

Yes i think you could try to do this.

Cellophile · December 22, 2021, 6:52pm

I have suppressed all things to permit a new backup.
So I recieved a firt mail :

Hello,

Your first backup on /media/svgd is starting.

This is an automated message from your beloved YunoHost server.

And 6 minutes later, another

The backup miserably failed to backup roundcube application.
The backup miserably failed to backup roundcube__2 application.
The backup miserably failed to backup roundcube__3 application.



Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 414.6KiB de données.
Sauvegarde terminée
name: auto_conf
results: 
  apps: 
  system: 
    conf_ldap: Success
    conf_manually_modified_files: Success
    conf_ynh_certs: Success
    conf_ynh_settings: Success
size: 424566
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 794.0MiB de données.
Sauvegarde terminée
name: auto_data
results: 
  apps: 
  system: 
    data_home: Success
    data_mail: Success
    data_multimedia: Success
    data_xmpp: Success
size: 832517904
Collecte des fichiers devant être sauvegardés pour borg...
Loading installation settings...
Declaring files to be backed up...
Backup script completed for borg. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 104.2KiB de données.
Sauvegarde terminée
name: auto_borg
results: 
  apps: 
    borg: Success
  system: 
size: 106745
Collecte des fichiers devant être sauvegardés pour borgserver...
Loading installation settings...
Declaring files to be backed up...
Borg backup repo in /home/backupVPS/ won't be backup to avoid backup of backup loop issue.
Backup script completed for borgserver. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 59.4KiB de données.
Sauvegarde terminée
name: auto_borgserver
results: 
  apps: 
    borgserver: Success
  system: 
size: 60839
Collecte des fichiers devant être sauvegardés pour roundcube...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 92.1MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211221-230453-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube__2...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube__2. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 99.6MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211221-230606-backup_create'
Collecte des fichiers devant être sauvegardés pour roundcube__3...
Loading installation settings...
Declaring files to be backed up...
Backing up the MySQL database...
Backup script completed for roundcube__3. (YunoHost will then actually copy those files to the archive).
Création d'une archive de sauvegarde à partir des fichiers collectés ...
L'archive contiendra environ 100.1MiB de données.
L'opération 'Créer une archive de sauvegarde' a échoué ! Pour obtenir de l'aide, merci de partager le journal de l'opération en utilisant la commande 'yunohost log share 20211221-230617-backup_create'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
Failed to securely erase old repository config file (hardlinks not supported>). Old repokey data, if any, might persist on physical storage.
By default repositories initialized with this version will produce security
errors if written to with an older version (up to and including Borg 1.0.8).
If you want to use these older versions, you can disable the check by running:
borg upgrade --disable-tam /media/svgd
See https://borgbackup.readthedocs.io/en/stable/changes.html#pre-1-0-9-manifest-spoofing-vulnerability for details about the security implications.
IMPORTANT: you will need both KEY AND PASSPHRASE to access this repo!
Use "borg key export" to export the key, optionally in printable format.
Write down the passphrase. Store both at safe place(s).
Failed to format translated string 'backup_method_custom_finished': 'La méthode de sauvegarde personnalisée '{method}' est terminée' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_method_custom_finished': 'Custom backup method '{method}' finished' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Failed to format translated string 'backup_method_custom_finished': 'La méthode de sauvegarde personnalisée '{method}' est terminée' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_method_custom_finished': 'Custom backup method '{method}' finished' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
Source path '/root/.ssh/id_borg_ed25519' does not exist
Source path '/root/.ssh/id_borg_ed25519.pub' does not exist
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Failed to format translated string 'backup_method_custom_finished': 'La méthode de sauvegarde personnalisée '{method}' est terminée' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_method_custom_finished': 'Custom backup method '{method}' finished' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Failed to format translated string 'backup_method_custom_finished': 'La méthode de sauvegarde personnalisée '{method}' est terminée' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_method_custom_finished': 'Custom backup method '{method}' finished' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'
Failed to format translated string 'backup_applying_method_custom': 'Appel de la méthode de sauvegarde personnalisée '{method}' ...' with arguments '()' and '{}, raising error: KeyError('method') (don't panic this is just a warning)
Failed to format translatable string 'backup_applying_method_custom': 'Calling the custom backup method '{method}'...' with arguments '()' and '{}', raising  error: KeyError('method') (don't panic this is just a warning)
A repository already exists at /media/svgd.
Échec de l'exécution du script : /etc/yunohost/hooks.d/backup_method/05-borg_app
Échec de la méthode de sauvegarde personnalisée à l'étape 'backup'

So there is a deeper problem !
If I make manually a backup with the yunohost interface, it’s work. But It’s manually

can the problem come from the fact that the support is a usb key ? it is quasi empty (32GO, for env. 16Mo currently)

If you want to see the logs :
https://paste.yunohost.org/raw/ecafokulec
https://paste.yunohost.org/raw/okifuronaf
https://paste.yunohost.org/raw/oqupivovok

ljf · December 22, 2021, 10:14pm

Your usb key might be not totaly reliable and cause data loss and integrity error ?
I don’t know how to help you more, can you try with an other usb key ?

Cellophile · December 23, 2021, 12:00pm

Thanks @ljf for your answers !
Yes, I will try with another external disk or key, to see.
See you later !