Uploading photos and videos to server and integrating s3 storage

I have two yunohost servers, one at home (Intel atom 2G/500G) and a vps (1 core/2G/20G).

I have added on the vps an s3 bucket on nextcloud using external storage on the administration settings. But I am getting some photo uploads incomplete, and when connection is unstable, it’s worse. The upload folder in appdata (a temporary folder in nextcloud) gets filled up with chunks “eating” the vps storage. So this solution is not the best way to sync photos to server (in my case).
I have also foldersync that syncs the download folder of my phone to my home server and it’s working fine, and files are not corrupt.

What I have recently done :
I have mounted the s3 bucket on /mnt/yunohost using rclone (systemd), and mount --bind a subfolder (named backup) with yunohost.backup. The backup process on the webadmin starts and fails at creating a symlink but if I remove this mount --bind, make a backup and copy using a cp command (or move) the whole folder, everything is preserved. So I can make a local backup, send it to the mounted s3 bucket and when needed, recopy it and restore.
So since s3 doesn’t support symlinks (and rclone emulates symlinks using a file), I think I might mount for every user, its multimedia folder to a subfolder or another bucket.

My questions :
1.How can I sync directly to my multimedia/photos and videos folder?
2. How do you mange your photos on your server? (I have deleted all my data on Google photos after downloading a takeout)

Hi jarod5001,

Starting at the bottom,

Do you mean to imply “I need a lot of space”, or “I used to use Google photos and I look for similar functionality”?

I saw your post right after I [hijacked someones thread about Immich] (Immich worked just fine for me - #9 by tituspijean), with a similar question. I recall the developer started Immich when he was not happy with Google, so it may be an option for you as well. I have to shut down my computer now, I’ll check the thread tomorrow :wink:

1 Like

My downloaded Google photos is about 60G. So yes, it’s a lot more than what I have free on my vps. And no, not really the same functionality since it has become a lot better than any other alternative what makes it hardly replaceable. I need a good photo management platform. Photoprism is excellent, photoview is very good, immich is very promising but it is still in alpha state and the devs recommend to not using it as the only photo storage

The project is under very active development. Expect bugs and changes. Do not use it as the only way to store your photos and videos!

But since I opened this thread, I have rethought about what I really need.
May be syncing from my phone to my multimedia/pictures folder on my home server over ssh (way better than webdav) and also to my pictures folder on my laptop. I won’t send my photos to my vps server, my home server is enough. But I will need my current projects documents accessible, so adding a bucket to my multimedia folder is a reasonable thing to do.
Besides, I think I will use another bucket as backup storage for my photos. I just found out that rclone supports encryption with password, which is enough and easier for me.

What still doesn’t work, is to have the yunohost backup folder mounted on the s3 bucket because of the symlinks issue (I am not sure to understand what the symlink in the backups do and if there is a way to bypass it).
This is a big brainstorming :sweat_smile:. Rearchitecturing my servers is inevitable and needs to be wisely planned.

That is always useful :slight_smile:

Do you use a bunch of apps and scripts for that, or is it easily exposed on your phone? WebDAV does have its benefits, but if I have a choice, I usually opt for SSHFS.

Would that be S3-compatible storage on your Yunohost via eg. Garage? Is it easy to use?

I also wondered and had a look. It is to support having backups at an alternate location without breaking/altering restoring of backups.

In that case, you wouldn’t need to have the backup folder itself mounted on S3, but you could configure an alternate backup directory and have that be a remote mount point. The bulk of the backups would be sent remote, with the json-metadata and the symlink in /home/yunohost.backups.

I am using foldersync, I have been using it for years. It’s very simple and stable but lacks detailed logs. There is also zpush. I have tried it and looks promising, though its Ui is basic.

It’s an s3 bucket mounted using rclone and systemd (so it’s mounted on boot)

:flushed: I don’t believe it, it’s possible to change the backup location and I didn’t figure it out!! Or it’s not available in the webadmin?