Questions before starting with remote storage and falback

[bbo]

/ (english)

Hi,
I want to know if what I imagine is realistic. I want to create a family self host server (about 10 peoples).
I want to host this applications :

• Calendar, Contact => Baikal
• Sync, organize they own picture privately and share them if they want => Librephotos (If you know better tell me if it’s note NextCloud)
• RSS READER => Tiny Tiny RSS
• e-book READER => Calibre-web
  For some of them :
• Password manager => Keeweb
• A little Cloud => Seafile
• I don’t want a mail server or WEBMAIL. I already have one in an other server.
• I may add extra applications like Grammalecte server or Scrumblr in a second time

Here is how i imagine all the infrastructure :

• 1 dedicate sever (principal),
• 1 VPS sever (secondary, maybe undersized, but it will work only if the primary is off).
• The medias (photos, e-book, Cloud) will be on a remote network storage. So i don’t have to think aboit space disk and backup.
• For the DNS and the fail-over IP it’s already done in an other server.
  i put a schema at the end of the topic to be more clear.

I think YUNOHOST aswered all my needs for the user part and the applications part. However, my architecture is atypical, and I would like your opinion on the feasibility.

I have few questions (I found some answered on my own but I leave question/answer here in case of) :

• Do you think the project is achievable ?

• Has anyone done a similar setup and have any feedback?

• I don’t want to use YUNOHOST mail server. Is it possible? (I found answer here, Yunost Forum And Here, Yunohost Documentation).

• I found the fallback application. In it there is two configuration files. One where i can set the application i want to backup. The second one is more the backup configuration.
  Do you know if i can choose what to save? In example, I would like to save, configuration or database from application but not the data (media). The data will be already save in the remote storage

• The remote storage will be mount on the server. Is there a better place to do it ? /home/media, /mnt.media, /mnt.remote,…
  This storage have to deal whith all big files or data like photo, e-book,… I don’t want the configuration file or database to be there. I see there is "/home/yunohost.multimedia/ in YUNOHOST to deal with media, is that right? If i mount my remot in this does it do what i want?

Thanks a lot dor the answered and advice
BBO

(français)

Bonjour,

Je vous sollicite pour savoir si mon projet est faisable. Le but est de monter un serveur pour ma famille (gestion d’environ une dizaine de personnes).
je veux leur proposer ces services :

• gestion calendrier, contact => Baikal
• gestion des photos de chacun de manière privée avec partage s’il le souhaite => Librephotos (si vous avez mieux autre que NextCloud je suis preneur)
• Lecteur RSS => Tiny Tiny RSS
• Lecteur d’e-book => Calibre-web
  et pour certains :
• Gestionnaire de mots de passe => Keeweb
• un petit Cloud => Seafile
• Je ne veux pas de serveur de mail ni de WEBMAIL. c’est déjà géré ailleurs (pour un groupe restreins).
• Je pourrais peu-t-être ajouter quelques applications supplémentaires genre Grammalecte server ou Scrumblr, mais ce sera dans un second temps

Pour les convaincre de partir de Google ou Apple pour ces tâches, voilà le montage prévue :

• 1 serveur dédier (principal),
• 1 seveur VPS (secondaire, certainement sous-dimentionné, mais qui sera là en cas de problème sur le principal le temps d’une réinstallation).
• Les médias (photos, e-book, Cloud) seront stocker sur un espace de stockage réseaux. Ainsi la taille et sa sauvegarde seront décorrélées du reste.
• La partie DNS et failover est géré ailleurs .
  (petit schéma au dessus du post)

je pense que YUNOHOST répond à mes besoins pour la gestion des utilisateurs et des applications dont j’ai besoin. Cependant, mon architecture étant atypique je voudrais votre avis sur la faisabilité.

Voici mes questions (j’ai déjà trouvé à force quelques réponses, mais je laisse les questions et réponses pour d’autre au cas ou) :

Pensez-vous que mon projet soit faisable, viable, ou complètement surréaliste ?

Quelqu’un a-t-il déjà fait un montage similaire et aurait un retour d’expérience?

Est-il possible de ne pas utiliser le serveur de mail de YUNOHOST, mais de faire en sorte que le serveur envoie les mail d’alerte (ou via un autre moyen comme un WS de notification)? (réponse trouvéeici, Yunost Forum et ici, Yunohost Documentation).

Il existe une application pour faire la continuité de service. J’ai compris qu’il y avait deux fichiers pour la configuration. L’un permet de définir les applications à sauvegarder et l’autre plus sur la sauvegarde en elle-même. Peut on choisir ce que l’on veut sauvegarder ? Par exemple, pour mon application de gestion de photos ou d’e-book, je n’ai besoin que de sauver les BDD et la configuration des applications. Les photos et/ou e-book le seront déjà via le stockage distant.

Mon stockage réseaux sera monté sur ma machine. Y a-t-il un emplacement mieux qu’un autre ? /home/Media, /mnt/remote,…? Le but de ce stockage et de gérer tous les gros volumes comme les photos ou e-book. Elles n’auront donc pas besoin de faire partie des sauvegardes. Cependant, j’aimerais éviter que les fichiers de conf et/ou les fichiers de bases de données soient dessus. J’ai l’impression que beaucoup de médias se trouve maintenant dans le répertoire /home/yunohost.multimedia/. Ai-je bon ? Dans ce cas suffit-il que mon remote soit monté justement là ?

Merci beaucoup par avance pour vos réponse conseil.
BBO

[wbk]

Hi bbo,

I have (had) parts of your infra picture, but not all at the same time. It may/may not help you

• failover: I had the fallback server installed and tried the functionality once, by accident, when I took my server offline for migration. There are some strings attached:
  • It’s a long time ago. I recall being surprised at how well it worked, but memory is thin on details
  • It does need a switch of DNS, or a DNS loadbalance/failover mechanism. I did not have that in place and had to manually switch A / AAAA records
  • I had only few applications installed, with little data.
  • The app is, according to the catalog, not maintained. Does not need to be a problem, but is a point of attention.
• Mix of VPS and at-home-storage: my wife’s Peertube-instance runs on Yunohost on a VPS with small storage; the video’s are stored on the Yunohost at home on an SSHFS mountpoint.
  • I contemplated doing the same for Nextcloud-storage (app on VPS, storage at home), but feared fast growing database files (on VPS) and latency problems (between VPS and home) and did not put Nextcloud on the VPS. No Nextcloud in your picture, so that’s an off-topic data point

To which extend is the failover picture ‘interesting as a thought experiment’ versus ‘really needed to offer 99.999% uptime’?
In my (fairly limited) experience, it’s really fun to work on complex setups like that with fantastic possibilities. On the downside, once there is downtime, it’s a complex setup with many things that need to work just right . It takes more time to troubleshoot ( → longer downtime) than having a simple setup with regular(ly verified) backups. What is your view on that?

[bbo]

Hi,

Thank you for the answered.

Mix of VPS and at-home-storage: my wife’s Peertube-instance runs on Yunohost on a VPS with small storage; the videos are stored on the Yunohost at home on an SSHFS mount point.

Can you please just tell me where you put the mount point? Do you have to change something in the app to change the Storage of the videos?

• It does need a switch of DNS, or a DNS loadbalance/failover mechanism. I did not have that in place and had to manually switch A / AAAA records

Yes, and Cloudflare allow me to do that. I have already tested it (it’s a pay feature).

• The app is, according to the catalog, not maintained. Does not need to be a problem, but is a point of attention.

I saw that. Before going in production, I will make some tests. So, I could see your last point about complex setup. However, This is a good point. I will think about it. Maybe if I have an installation script or process to make it simple and quick one server is enough. I’m still thinking about that.

Again thank you .

[wbk]

@bbo

Mix of VPS and at-home-storage: [data] stored on the Yunohost at home on an SSHFS mount point.

Can you please just tell me where you put the mount point? Do you have to change something in the app to change the Storage of the videos?

I didn’t change settings in the app, but added directories of which I want to store data at home to /etc/fstab, each line in this way:

sshfs#yuno_home_ssh_user@home_domain:/home/data_remotes/peertube/video /home/yunohost.app/peertube/storage/video  fuse _netdev,reconnec
t,allow_other       0       0

• sshfs#yuno_home_ssh_user
  • sshfs to signal the mounting mechanism that it’s SSHFS
  • yuno_home_ssh_user is a user that is known on the Yunohost at home, and is allowed to use SSH or SCP. It does not have to be Yunohost of course, any system that supports SSH/SCP would do
• @home_domain : domain or IP address of the storage server
• /home/data_remotes/peertube/video /home/yunohost.app/peertube/storage/video as the location on the storageserver where the data is stored
• /home/yunohost.app/peertube/storage/video the location where it will be mounted on your VPS-Yunohost
• fuse _netdev,reconnect,allow_other : mount options; allow_other to make sure not only the mounting user (root in this case) can use the mounted location, but others as well;

This way you can cherry pick the (sub)directories that you think will grow a lot, and move the data to a storage server. Backups (with the Borg-app) run the same way. Borg thinks it is storing locally, but via SSHFS the ‘local’ directory is actually on the home server.

@bbo

Yes, and Cloudflare allow me to do that. I have already tested it (it’s a pay feature).

Nice (I actually hoped you had a self hosted solution that you could share )

For the failover system you could have a look at running Yunohost in a container (not VM) on Proxmox, and have your two VPS’s as nodes in a Proxmox-cluster.

(small warning)

I misconfigured my Proxmoxes, the nodes can not find each other (when you do that: make sure Proxmox networking is configured the way you want it, and that you read the documentation before creating a cluster). So, I can’t share experience on running a cluster this way