Yunohost backup create KO

#1

Bonjour,

Depuis quelques jours, la backup yunohost ne fonctionne plus. Il semble que nextcloud pose problème lors du message d’erreur suivant rencontré :

Voici ma version de yunohost sur une VM PROXMOX :
image

Avec ce type d’erreur, j’ai tout d’abord vérifié l’espace disque mais c’est OK, vérifié s’il n’y avait pas des fichiers mal supprimé avec lsof et enfin, j’ai ajouté dans la config de nextcloud le paramètre backup_only_core=1 :

image

Ensuite, lors d’un test manuel, en settant la variable cela fonctionne :
image

Il semble que la variable suivante BACKUP_CORE_ONLY soit aussi checker en plus de backup_only_core (d’ailleurs, j’avais déjà ce paramètre auparavant et tout était OK) et en automatique elle est égale à 0 donc plantage car j’ai trop de data donc erreur normale :


Par contre, comme je ne veux pas du backup des datas, comment setté ce paramètre qui est arrivé depuis la dernière MAJ nextcloud en gros ?

En vous remerciant,

martoni

#2

Tu utilises quelle méthode automatique ? Un truc fait main ? Le tuto borg? L’ap borg ? Archivist ?

#3

Hello,

Pour la partie en automatique, j’ai juste en crontab la commande suivante :

yunohost backup create

image

Ceci fonctionnait très bien quotidiennement jusqu’à la MAJ nextcloud d’après mes recherches.

martoni

#4

La mise à jour a peut être remis backup_core_only à 0…

Je te suggère de modifier ton cron ainsi:

yunohost backup create --apps app1 app2 --system
BACKUP_CORE_ONLY=1 yunohost backup create --apps nextcloud
#5

Hello,

Le paramètre est bien dans la config de nextcloud :
image

Ta proposition contourne le souci malgré que cela marcherait.
Cependant, c’est une rustine qui ne me règlepas la root cause, on est d’accord je pense ?

Est-ce qu’il y a des fichiers de config (yunohost) qui ont évolué avec l’intégration de la dernière version de nextcloud par rapport à avant ?

martoni

#6

Après recherche il s’agit d’un refactoring du script de backup qui a fait disparaître cette option.

A priori je pense que ça concerne de nombreux scripts.

Avant:

Après:

#7

I made a fix, you can apply it manually and say if it works:

1 Like
#8

Hello,

This fix was apply manually with success :slight_smile:


The merge could be done :wink:

#9

Hello,

New check Ok with new changes “auto”

#10

How funny :neutral_face:

I use for a long time this helper, design to check the available space before any backup.
This helper isn’t in the core of YunoHost because someone told me many times that it was useless because the backup process already check the available space… That I denied many times…

Never mind…

backup_core_only has been removed from the config and used as an environment variable because we had many issues with the setting into the config, especially with failed backups.
That’s not an error, nor a bug. We have changed the way this setting is handled in order to manage it from the backup helper.
We probably lack some communication about it, but the best way to manage that is to fix your backup script, not to restore the old behavior into the app.

#11

I tell you that because it’s true, the core does the check. How you explain the first error message in the first screenshot of this topic “L’espace disque est insuffisant sur /home/yunhost.backup/archives” ? This message is displayed because we check that before to do any backup. If no, there wouldn’t be free space on the ynh instance, and it’s not the case.

Note: Info message “Backing up…” have been added after and should be corrected by a message like “Adding SOMETHING to backup list”. Indeed at this step nothing is backup…

Summary of the backup process (in the case of tar method):

  1. Collect of all path to backup and generation of database dump
  2. Estimation of backup size
  3. Add each path to the new archive

So in this case, the backup fail at step 2, before to do the true backup.

#12

With this change, is there a way to let end users configuring their backup to ignore big data ? If yes my PR isn’t necessary, if no I think i have made better than the “old behaviour”…

#13

Of course there is, we didn’t just removed a needed option. We have replaced it because it was causing issues.
As explained in the documentation,https://github.com/YunoHost/doc/blob/master/backup.md#apps-specific-configuration, to ignore big data, the backup command should be preceded by BACKUP_CORE_ONLY=1

#14

So you think the first solution i suggested in this topic is the good one:

I see 3 (minor) limits with this way:

  • we loose retrocompat (but it make several month now, i don’t know if it’s really important)
  • users can’t use the webadmin to make backup manually with a backup_core_only preconfiguration
  • no way to create a mono archive with this method

I think we should search a better way with the new work on backup.

#15

We loose retro-compatibility only for users that use custom backup script, not for the upgrade script.
For those users, they can just modify their scripts.

There were no way before to set backup_core_only from the webadmin, nor there is now. This settings wasn’t made to stay set, scripts remove it when it’s set.
We have to keep in mind that this setting was existing for the upgrade script.

If you need to ignore big data, you can set this variable for a “mono” archive if you want. It will be used for all apps having big data.

#16

Hello there,

Just bumped on a similar issue while doing a backup with the “new flag” as said here [Official app] Nextcloud

After following the documentation https://github.com/YunoHost/doc/blob/master/backup.md#apps-specific-configuration, I modified my linux command like thisBACKUP_CORE_ONLY=1 yunohost backup create --description 'Sauvegarde complete sans donnees nextcloud et transmission'

I end up with a 20 gb backup instead of 1gb :open_mouth: Not good! After digging, it appears that transmission is not “compatible” with this new feature/behaviour.

  1. Is there any list of apps that needs to be fixed to handle this setting?
  2. Is there any plan to fix them?
  3. Is there an example/documentation to modify the scripts? (I’m willing to adapt Transmission)
#17

I’m currently upgrading all Official apps to last packaging standards, I’m on Synapse currently, Transmission will be the next one.
However, as I’m going to be unavailable next week, it’s not going to be until 1 or 2 weeks.

We don’t have any list of apps to upgrade. I guess though you can search backup_core_only into YunoHost-Apps to find them, https://github.com/search?q=org%3AYunoHost-Apps+backup_core_only&type=Code

Not sure any other apps than nextcloud have been updated yet. So you can have a look to nextcloud.

1 Like
#18
  1. we loose retrocompat (but it make several month now, i don’t know if it’s really important)

“just modify”, not sure that all people using Yunohost will be aware of this change when it will break their backups.
Especially that all applications are not compatible with the change yet (hello Transmission)

  1. users can’t use the webadmin to make backup manually with a backup_core_only preconfiguration

There should be a way to do it graphically. Or it will come later when there will be a way to download easilly the backups from the web admin?

Could you elaborate on this please ?

There should be two backup system then? One for upgrade and one for the user to decide which application “big data” to use? (in my case, backup nextcloud but not transmission. I might also avoid backuping Nextcloud data as they are already replicated on other devices)

Or maybe, having a mechanism to choose what to backup with checkbox in GUI. (it will be able to “overide” and remove “big folder” to be backuped)

Nextcloud : backup all
transmission : backup settings only (no data)
baikal: backup all
[…]

  1. no way to create a mono archive with this method

Not an issue for someone using the CLI

#19

I was talking about user using a custom script for their backups. We can’t keep retro-compatibility for custom scripts.

By “We have to keep in mind that this setting was existing for the upgrade script.” I meant that this setting has been added for upgrade script when doing a backup before upgrade.
Indeed, some could want to use it as well for their recurrent backups. But that wasn’t the idea when the option has been added.

Now, considering the experimental feature Config-Panel, we can imagine adding a setting for backup of big data for apps that would use that kind of setting.
But this setting should be different of the one used by the upgrade script.

I understand the need of such an option. I think Config-Panel is a good way to do it, both graphically or by CLI.

1 Like
#20

Hello,

Depuis la migration en 3.5.2, j’ai à nouveau la même erreur qui est revenu !!!
J’ai tenté d’appliquer le code dans [fix] backup_core_only settings doesn’t work
En effet, la partie du code ajouté dans ce fix était absente.
Nouveau test, toujours Ko avec le même message que ce billet …

Pourriez-vous checker les modifications apportées à cette version 3.5.2 ?

En vous remerciant,

Cordialement,

martoni