Using Minio app as dependency of another app

Contribute Room Apps packaging
1

With other persons, I am packaging Docs for Yunohost (work in progress here).

Docs require to have a S3 storage backend. For that purpose, we take advantage of the MinIO package (and we would like to also support Garage).

There is a tricky part, where we would like the backups to be self-sufficient (we would like to not have the user worry about the backups of MinIO and whether it is synced with Docs), hence to be able to recreate the content of the bucket if it is missing when restoring.

The scripts/backup script seems to only declare files to be backed up, we could use this script to copy the content of the bucket in a temporary directory and declare this target with ynh_backup, but we don’t know when the cleanup would occur.

I don’t have a clear idea of what could be the good practice:

  • maybe we should not rely on a third-party app for backend storage? (and rather install the backend storage like MinIO ourselves, but hmm, feels like it’s a pity to not mutualize)
  • maybe we should not worry much about the cleanup, after all these will probably be just a bunch of text documents and images in most cases for Docs;
  • maybe there is room for making a contribution in the core to tell Yunohost that a directory to be backed up is temporary and to clean it up after the upgrade?

Thanks in advance!

2

Instead of asking for minio in the manifest, why not just asking for the url, credentials and bucket (so it is not dependent on minio only). And add data_dir used as the location where to copy the content of the bucket during backup.
If the bucket is not empty when restoring, move its content to bucket.old in data_dir and report it in the logs. (to avoid data loss).

3

So you rather suggest not to worry about the cleanup? (the second point of my original post)

4

You mean the remove script?
It will be like keeping data_dir when uninstalling an app. May be adding some information to ADMIN.md that the remove script will NOT delete the bucket content and it’s up to the user to keep it, remove it or move it elsewhere.
The choice of using s3 storage may have some implications on app packaging and data maintenance, as managing s3 content in the package will add unnecessary complications, on the other side, users used to s3, will find some flexibility.
It seems the app itself doesn’t give you a choice other than s3.

I already have Minio installed to sync Joplin Notebooks. It’s extremely light. But having another instance of Minio for every app or usage, would be a waste of resources.

But, it’s up to you to choose the right algorithm for your package. Integrate Minio so backup and restore would be a complete backup/restore. Allow the user to choose any s3 storage (local or distant), this way you’ll have two choices : (1) let the user manage the data himself, (2) add an s3 cli client to the package dependencies so it copies the s3 bucket content to data_dir in the backup script and restore the data_dir content to the bucket (in this case, if the bucket is not empty, copy its content somewhere before overwriting).

5

Yes, my point is that as the backup script is only meant to declare what you copy, not to copy per se, you can only copy the content of the S3 bucket in data_dir and declare the path, but you can’t clean it up once the backup is done.

That means that the data is copied at 2 different locations. One way to handle that would be to clean up the location of the copy with a cron job.
Also you loose the versions, but maybe that’s not such a big issue.

1 Like