Transfer a YNH instance to a new VPS without enough place for archive

[lops]

français (english below)

Salut à tou.te.s !

Déjà merci beaucoup aux contribut.eurs.rices pour ce super projet ! Ca fait bientôt 4 ans qu’on fait tourner deux serveurs Yunohost pour un usage quotidien par une dizaine de personnes. C’est impressionant de voir toutes les avancées au niveau communautaire, gestion des applications, de l’expérience utilisateur etc.

Voilà un nouveau topic pour chercher quelques conseils/idées pour la migration d’une instance YNH d’un VPS à un autre, merci d’avance.

Contexte

On a besoin de migrer une instance YNH d’un VPS (serveur-actuel) à un autre (serveur-cible). En terme de doc voilà ce que j’ai trouvé :

• YNH doc: 'Sauvegarder son serveur et ses apps’
• discussions sur le forum :
  • Transfert d'une instance Yunohost vers un nouveau serveur
  • Transfert d'hébergmement OVH -> Raspberry
  • Restoring a complete backup on a fresh install

A priori la manière classique de faire est de faire un backup complet avec yunohost backup create puis de scp l’archive vers le serveur-cible afin de la restaurer sur une instance toute fraîche de YNH (à la place de faire la post-install).

serveur-actuel

• last stable YunoHost (3.6.5.2) - Debian 9.11, hébergé sur Gandi VPS
• nom de domaine et enregistrements DSN gérés chez l’hébergeur
• accès via SSH ou WebAdmin
• uniquement des apps officielles et à jour (Nextcloud, Wordpress, Jirafeau, Zerobin, Wallabag, Synapse)

serveur-cible

• Debian 9.11 toute neuve, hébergée sur un autre VPS Gandi

Description de la problématique

J’utilise déjà 80% de l’espace disque disponible sur le serveur-actuel et le serveur-cible a la même taille que le serveur-actuel. Donc j’imagine que je n’aurai pas assez de place pour qu’une archive complète cohabite avec le système Yunohost. A priori la commande yunohost backup create créera l’archive sous /home/yunohost.backup/archive et on ne peut rien y faire.

Quelques questionnements :

• est-il possible de choisir où l’archive doit être sauvée ? Idéallement j’aimerai lancer la commande yunohost backup create via ssh (ssh user@host yunohost backup create) et sauver l’archive directement sur mon PC.
• sinon est-il possible d’utiliser rsync pour synchroniser la racine / de mon serveur-actuel sur le serveur-cible. J’imagine qu’il y aura des soucis avec la config serveur (nom du disque, adresse IP ou quoi que ce soit d’autre…), pensez-vous que ce soit gérable ?
• d’autres idées ?

---

/ English

Hello to everyone !

First I’d like to thanks all the contributors for the amazing project ! It has been almost 4 years that we are running two yunohost servers for a daily usage by 10 active users. It’s impressive to see the constant improvements about community organization, app management, user experience and so on.

Here is a new topic to ask for some advices/ideas about migration a YNH instance from a VPS to another. Thanks in advance.

Context

We need to migrate one YNH instance from one VPS (current-server) to another (target-server). I’ve checked some documentation/discussions about it:

• YNH doc: 'Backing up your server and apps’
• some discussions on this forum:
  • Transfert d'une instance Yunohost vers un nouveau serveur
  • Transfert d'hébergmement OVH -> Raspberry
  • Restoring a complete backup on a fresh install

It seems that the common way would be to make a full backup with yunohost backup create and then scp it to my target-server to restore it on a freshly installed YNH server (instead of running a post-install).

current-server

• last stable YunoHost (3.6.5.2) on up-to-date Debian 9.11, hosted on a Gandi VPS
• domain name and DNS record managed through my host system
• access through SSH or WebAdmin
• only official and up-to-date apps (Nextcloud, Wordpress, Jirafeau, Zerobin, Wallabag, Synapse)

target-server

• freshly install Debian 9.11, hosted on another Gandi VPS

Description of my issue

My problem is that I already use 80% of my disk-space on the current-server and the target-server is the same size as the current-server. So I guess there is not enough place to make a full-backup co-exists with the YNH system. It seems that’s the command yunohost backup create will create the archive under /home/yunohost.backup/archive and that I can’t change that.

I wonder if:

• is there any way to choose where the archive must be saved ? Ideally I would like to run the yunohost backup create command over ssh (ssh user@host yunohost backup create) and save the archive directly on my local computer.
• if not, is it possible to use rsync commands to synchronize the root / of my current-server to the target-server ? I guess there would be some troubles with some server config (disk name, IP adress or anything else…), is it manageable?
• any other idea ?

[anonyme18]

Bonjour,

Est-ce que cette partie de la page YunoHost • index répond à ta question

Restauration d’une archive à la place de la post-installation
Une fonctionnalité particulière est la possibilité de restaurer une archive entière à la place de faire la post-installation. Ceci est utile pour réinstaller un système entièrement à partir d’une sauvegarde existante. Pour faire cela, il vous faudra d’abord téléverser l’archive sur le serveur et la placer dans /home/yunohost.backup/archives .
Ensuite, à la place de yunohost tools postinstall , réalisez la restauration de l’archive téléversée par cette ligne de commande avec le nom de l’archive (sans le .tar.gz ) :

yunohost backup restore <nom_d'archive>

Note: si votre archive n’est pas dans /home/yunohost.backup/archives , vous pouvez spécifier où elle se trouve comme ceci :

yunohost backup restore /path/to/<archivename>

[Aleks]

@lops

My problem is that I already use 80% of my disk-space on the current-server and the target-server is the same size as the current-server . So I guess there is not enough place to make a full-backup co-exists with the YNH system. It seems that’s the command yunohost backup create will create the archive under /home/yunohost.backup/archive and that I can’t change that.

Indeed the lack of disk space is really critical here … Isnt there any way to add additional storage space to your server ? (Having less than 20% available disk space sounds like living dangerously ?)

[ljf]

Method 1: use tar backup method BUT split component by component

I suggest you to try to split your backup/restoration operations in several steps.

STEP1 system configurations
STEP2 each APP one by one
STEP3 data_mail
STEP4 data_home

Note: if you have a lot of data in nextcloud you can backup without configuration and transfert data directly with rsync or scp. Idem for mail or home users dir…

For each step do:

1. Backup
2. Transfert
3. Restore
4. Delete archive (or better move it on your computer)

Like that even if you have 20% of space available you might be able to restore… But you wont if some of one of this component need more than20 % of space…

In theory: this method could be done with Webadmin and filezilla (and rsync if you separate nextcloud data or mail/home user dir).

Method 2: clone the server in “rescue mode” with rsync

I don’t remember the exact way with rsync their is some file to avoid, but it could be a method to migrate.

Method 3: if you have physical access or some options available

Add a usb disk or remote disk (remote storage and mount /home/yunohost.backup/archives on it

Advice

In your case it’s ok because you have the old server and the new one. But for people who reinstall… Please check your backup consistency before
You can check for example each backup contains:

• info.json
• backup.csv

And if it’s an app backup with a database:

• db.sql

Example:

tar -ztvf /home/yunohost.backup/archives/NAME_BACKUP.tar.gz | grep "backup.csv"

[lops]

Thanks a lot for your feedbacks.

@anonyme18: maybe I was not really clear, my main problem is lack of disk space, but yes this doc is useful
@Aleks: yes we live dangerously, but in fact we don’t use email server and set nextcloud quota for each users so we manage to monitor our disk usage for some times and no accidents for now ! Disk space is expansive and if I expand existing disk there is no way to go back.
@ljf: thanks a lot for these ideas !

@ljf

Method 1: use tar backup method BUT split component by component

I feel like it will be a bit tiring … but it’s a way

@ljf

Method 2: clone the server in “rescue mode” with rsync

Mmh I will investigate more, maybe this kind of example could be useful: How to migrate a Debian server (with rsync)

@ljf

Method 3: if you have physical access or some options available

Of course ! I hadn’t thought before … it’s maybe the easiest way: I can buy a new disk just for the operation and delete it after, like a temporary extra space. I think I’ll try this option because I really would like to test the backup create/restore yunohost commands.

I’ll post feedbacks when it will be done. I’ll begin in one week so any other ideas are welcome