EDIT : The app is now level 7 and available in the catalog, keeping the original post below for history.
Hello everyone,
As I really prefer SearXNG over Searx, I launched myself into packaging it. You can find my progress here UPDATE : my repository have been forked by the YunoHost Apps organization, you can now find the progress here instead. Please note that as of today, this app is NOT ready to use.
I’m posting here to give news about my progress, to ask for help regarding some troubles I run into while packaging and seek advice from more experienced packagers, and to get feedback from testers (once my package will be suitable for testing). I’ll update this post regularly to give feedback about my progress.
I’m using mainly this doc to guide me for packaging, along with a strong inspiration from the YNH Searx package which is obviously very similar to the one I’m creating.
My progress :
I’m currently working on making the install script work. I’m stuck trying to run and daemonise uwsgi correctly.
I’m working on the setup of the github CI pipeline in order to not have to update the hash of the source manually each time a new commit is pushed on SearXNG.
CI and most of the code is working now, I’m now trying to pass all the tests, the app is nearly ready !
My problems / questions :
I’m not sure how to deal with the fact that SearXNG is a rolling-release. Can I automate the packaging process with CI to always keep up with the latest version or am I supposed to do it a different way (for example, plan only one release per month) ?
Answer here :
For now, I’m getting the source using the ynh_setup_source helper to get a tarball. However some functionalities (such as running the app or updating) would be easier using git commands. Is there a specific helper to manage git repos and / or can I do it by myself in the install scripts without impacting the YNH instance ?
Answer here :
Indeed, Docker is an easy way to do this. I saw some attempts like this one to bring Docker to YNH, without success it seems. I don’t know if YNH maintainers have an opinion on this, but I think using Docker images are not the targeted goal when packaging apps.
I’m not sure how to deal with the fact that SearXNG is a rolling-release. Can I automate the packaging process with CI to always keep up with the latest version or am I supposed to do it a different way (for example, plan only one release per month) ?
We provide a way to automate the PR creation when a new release appears.
So if you create the samed .github and change a bit updater.sh you will get PR created after each SearXNG release.
Could you elaborate on the advantage to get this as a git repos ?
The normal way for yunohost is the tarball way, and with the sha256 hash. We prefer to stick a specific version (cause we need our CI to test this specific version we publish). So that’s mean that just cloning a repo or git pull it, is a bad practice for us, cause you d’ont know if the upstream (searxng) will make some breaking changes.
Some packages use docker images, others download docker images to extract build arm version. But in your case it’s just a simple python app, so you should keep it simple without docker.
Note: in yunohost 11.1 beta you can test the new app v2 mechanism with manifest.toml. There is a conversion script to convert an app from v1 to v2 format.
I’ll take a look at that then. I don’t know if there is some limitations to the CI/CD time on GitHub, but for this simple usage I suppose that’ll do.
There is two :
The app is designed to run as a git repo. This means that Flask, for example, use info stored in the .git folder to run. The app run nevertheless (even if throwing errors), and I haven’t found yet if this is introducing bugs.
There are .sh scripts helpers for installing and updating the app, relying on git. I thought about using them to package the app, but they need that the app have been git cloned in the first place.
None of these two issues is a real problem, and I can work around if needed. I was just wondering if it was possible to work from a git repo, if not the tar.gz will do.
I haven’t found if there is a unified version to install / manage docker in YNH. But you’re right, it would be better to skip Docker and to make it work from scratch. However it could be nice as a backup if I’m not able to package (fastly enough) searxng.
Thanks for the tip, I’ll take a look. I haven’t found info on this, will there be a mandatory switch from v1 to v2, and if yes what will be the time frame ?
I’m happy to announce that the install script is finally working. Feel free to test the install on test environments and to report any issue you may find afterwards.
I will now focus on the CI to automate the update of the source hash, I’m getting bored of having to update the hash manually each time I want to make a test.
I finally found what the git repo is used for : as a rolling release, searxng is using git to create a version based on the date of the commit and the commit hash (e.g. 2023.01.09-afd71a6c). This version is displayed at the bottom of searxng. If the git repo is not present, the displayed version is 1.0.0 by default, which is the case now with my packaging as I’m using the zip source which does not contain the .git folder. However I can edit the version manually by modifying this file.
A few options from there :
I leave it like that
I change the version to something fixed, but including a mention to yunohost or whatever
I change the displayed version using Github Actions each time there is an update
Perhaps an idea… Rather than linking in app.src the master.zip you can link an archive corresponding to a commit for example https://github.com/searxng/searxng/archive/d669da81fbe820df2d97c98560d00dee32979dc9.zip so the hash is stable until you decide to update the SOURCE_URL.
Indeed, I had the same idea just yesterday. I updated the app.src file in the Yunohost-Apps repo available here. As for the version displayed in searxng, I will not bother anymore for now as this is not impacting anything important. I’ll maybe go back to it later. I’ll just finish the CI and move on.
I am happy to announce that the app is nearly ready for use !! It is working on my test instance and most of the tests are passing on my computer. I created the PR here to have it added to catalog. Meanwhile don’t hesitate to test it and give feedback, either here or on the PR.
The app have been integrated into the catalog ! It is noted as broken for now as the CI results haven’t been integrated yet, but it’s a matter of time. For now, the app is level 3, but will be level 7 once I fix the (hopefully last) problem…
Indeed, the test suite (you can find one here) is failing the backup/restore part. I’m not able to figure out why, as it is working without any problem on my Virtualbox test instance, with different YunoHost versions and configurations
That’s why I’m asking for help to troubleshoot this : if some people can test the backup / restore manipulations on their own (test) instances, and share to me if it’s working or not, I’d really help me troubleshoot the problem. Thanks in advance
SearXNG is written in Python so it is interpreted on the fly, without need for compilation. The application is installed on the server directly, as most of the YunoHost applications. Containerization as a mean of packaging YunoHost apps have been discussed on this forum (see the link I shared above), but I don’t know if YunoHost devs intend to switch massively to it.
