Using Docker in a YunoHost package

Because of Paperless-ngx is broken (ghostscript is to old in Debian bullseye: Paperless-ngx MissingDependencyError: gs - #21 by jedie ) and it’s hard to get a newer python version: Use newer Python than 3.9 ?!? i experiment with Docker in a YunoHost package, here:

Some notes:

  • Use docker-compose from Debian repository (installed by manifest v2 [resources.apt] section)
  • Add tree containers:
    • The Web App container with the Django project
    • Postgres
    • Redis
  • Use SystemD service file to start the Docker container
  • Use YunoHost system nginx as proxy
  • Store docker volumes (postgres database files) in __DATA_DIR__/home/yunohost.app/$app/
  • Serve static files from system nginx (From: __INSTALL_DIR__ → `/var/www/$app/
2 Likes

that’s really cool. i wonder if many apps would benefit from being container based? you would have a rock solid Debian base install with containers on top.

One of the problems: hub.docker.com has a rate limit. Think the YunoRunner CI should cache the downloaded images.

I don’t know how to setup this.

Another thing is: Maybe Podman ist a better idea? (Edit: found a old thread: Consider packaging apps with podman? ) Or a complete other tool?

EDIT: Currently it looks like:
grafik
master <- testing by jedie · Pull Request #1 · YunoHost-Apps/docker_django_example_ynh · GitHub

EDIT2: Next thing i would test is package “Paperless-ngx”: GitHub - YunoHost-Apps/paperless-ngx_docker_ynh at testing

currently i am just running some apps in docker on top of yunohost. i feel more confident that i won’t mess up my system.

i lose a lot of the benefits of Yunohost though (LDAP, handling of reverse proxy).

would be interested to see how you handle paperless-ngx. i’m running this in docker atm.

The normal paperless-ngx YunoHost package doesn’t have SSO integration.

It’s possible to make this: GitHub - YunoHost-Apps/docker_django_example_ynh: Demo YunoHost Application to demonstrate the integration of a Django project via Docker under YunoHost. has SSO integration via my GitHub - YunoHost-Apps/django_yunohost_integration: Python package with helpers for integrate a Django project as YunoHost package. package.

The only “real” problem i currently have: YunoHost CI will ran into docker hub rate limit…

After reading this issue on github about Paperleass and gs problems I think it would be good to have this running dockerized, it will break again in the future.
Also it is bad that the update was released for the Yunohost app when not all requirements are fulfilled.

Thanks for the effort.

Sorry for leaving this here, I have no clue, so I asked ChatGPT about that problem, here is the reply:
To prevent your Continuous Integration (CI) process from running into Docker Hub’s rate limits, you can consider several strategies:

  1. Authenticate with Docker Hub: Even if you’re pulling public images, authenticating with Docker Hub increases your rate limit. Make sure your CI system is using an authenticated Docker Hub account.

  2. Use a Docker Registry Mirror: Set up a local mirror of Docker Hub or use a cloud provider’s registry mirror. This can reduce the number of requests to Docker Hub, as images are cached locally.

  3. Optimize Docker Image Usage:

    • Reduce the number of image pulls by optimizing your CI pipelines. Cache images where possible.
    • Use smaller, more efficient base images to decrease the frequency and size of downloads.
    • Avoid unnecessary docker pull commands in your CI scripts.
  4. Docker Hub Subscription: Consider subscribing to a paid plan on Docker Hub if your usage is consistently high. Paid plans offer higher rate limits.

  5. Self-Hosted Docker Registry: Host your own Docker registry for your images. This approach gives you full control over your registry without external rate limits.

  6. Rate Limit Handling in CI: Implement logic in your CI pipelines to handle rate limiting, like retries with exponential backoff.

  7. Use Alternative Registries: Leverage other container registries like GitHub Container Registry, GitLab Container Registry, or Amazon Elastic Container Registry, which might offer better or different rate limits.

  8. Monitor and Analyze Usage: Regularly monitor your Docker Hub usage to understand and optimize your pull patterns. Tools and logs can provide insights into how often and what your CI is pulling.

Each of these methods has its own trade-offs in terms of complexity, cost, and maintenance. You may need to use a combination of these strategies based on your specific CI workflows and requirements.

Think the main problem is, that the YunoHost CI will not cover the real functionality:

The tests do not go deep enough: Check if root page returns status 200 doesn’t mean that everything works.

See: YunoHost package maintaining: Continuous integration? Trouble with CI

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.