Help for new app packaging : 13ft [12ft.io alternative] - bypass paywalls

Hello :slight_smile:

I would like to package 13ft, a selfhosted equivalent of 12ft ladder. It allows to bypass some paywalls by fooling the website, impersonating GoogleBot (guess what : a lot of website gives their whole content to GoogleBot for SEO ranking).

It seems “simple”:

  • install python flask dependency
  • run a python script
  • the app is working.

I started to work on it here : GitHub - lapineige/13ft_ynh: A 12ft.io replacement, based on 13ft.

Feel free to help :slight_smile:

Thanks :smiley:


My current questions:

  • how do I start this python script properly ? Including on server restart.

  • how to stop it properly ? (shut down a SystemD service ?)

    • how users could do it ?
  • how to uninstall properly

    • how can we check that flask isn’t used by another app before removing it ?
  • how do we transfer an app to Yunohost catalog ? I don’t remember :sweat_smile:
    edit : I found how to do it (in the repository settings), now I waiting for github to unfork the repository as I can’t transfert a example_ynh fork, even with a different name.

2 Likes

This is called a daemon or service, so you want to look into systemd service configuration

For example, pepettes (powers donate.yunohost.org and is also a flask app) : https://github.com/YunoHost-Apps/pepettes_ynh/blob/master/conf/systemd.service

you will usually want to use gunicorn which is a small piece of software that creates a bunch of workers for the app such that it’s not a single process handling all the request (imagine only 1 waiter/waitress/cook in a 1000-person restaurant)

(and cf also the ynh_add_systemd_config and similar helper calls in the app install, remove, upgrade, restore))

Not sure why you’d want to stop it ? Maybe you’re referring to having the systemd service listed in the “Services” section of the webadmin. For this you can have a look at the call to yunohost service add in the app install script

Usually Python apps use what’s called virtualenv to properly separate all their dependencies from the rest of the system (similar to nodejs node_modules folder)

This implies the app provides a requirements.txt as part of its source to list its dependencies (and for the love of god, please don’t install stuff with pip globally on a yunohost system, that tends to make everything explode)

correct example in pepettes : https://github.com/YunoHost-Apps/pepettes_ynh/blob/master/scripts/install#L123-L126

2 Likes

I configured a systemd service (and registration into yunohost web admin interface), but I’m unsure how to do it properly. Right now the command is python 3 /var/www/[app path]/init.py and that’s it.

I don’t know if it’s needed in that case, it’s a very simple and lightweight app, and I believe it won’t be used that much.
Is it simple to do ? What’s the drawbacks ?

When I uninstall it ?
And also I think it’s good that users have the control to shut it down.

Already done, thanks :), it’s very simple with the handler and the default example :slight_smile:

Oh yeah, I haven’t deployed a python app for so long that I forgot about that :joy: :sweat_smile:

Great, thanks for the tip. What the syntax ? Just flask ?

Thanks for pointing this, very appreciated, I was a bit lost about how to do it properly the yunohost way :sweat_smile:

Hmmm yeah maybe you’re right, I didnt really have a look into the actual app :stuck_out_tongue:. Just highlighting that’s the usual stuff, but big +1 for not blindly copypasting stuff, each app has its own quirks

Then yeah I guess you mean the service integration in the Services section of the webadmin

During the uninstall, just stop the service, remove it from yunohost with yunohost service remove(?) and remove the systemd conf with the appropriate helper … the example app should show all this

Hmmm not sure what you mean, requirements.txt should be provided by the upstream app … though you could also build one manually … The syntax is for example this: https://github.com/YunoHost/pepettes/blob/main/requirements.txt (you can see Flask in the list along a bunch of other stuff). requirements.txt are usually generated by running something like pip freeze which list all packages installed + their version

Note that if this is just to install Flask, then there’s no need for a requirements.txt … They become relevant when you start having a bunch of dependencies. To install flask inside the venv, you can just run venv/bin/pip install Flask (not sure about the exact syntax and the call to pip inside the venv depends on the exact context)

:joy: I understand :wink:
On the other side, the configuration you shared allows for an easy way to shutdown the app, as well as reloading it (not sure it’s useful).

Great, thanks.

It doesn’t have it. The app is very simple, and html file, a python file, that’s it.
edit: the software author added this file ! So I’m using it.

I guess I’ll have to do that… thanks for sharing this example.

Oh ok great, so I’ll only do this.

By the way : do you know how I can enable again “issues” and the github repository ? I can’t find a way.

go to “Settings” on the repo, first page, tick the “Issues” feature

Hm just saw you only had Write access to the repo, I changed it to “Maintain” which should give you access to repo settings

1 Like

That’s the thing I didn’t know !
I believed that I would keep the right to do that as I can write to it and shared it with Yunohost-Apps organisation.

Thanks !

edit : I only have “Wikis” and “Projects” :confused:

Changed to “Admin”, should be good now

1 Like

It is !

Is it possible that the source files (in that cas only one file) is not formatted as an archive ?

In that case it’s only a python script.
Can I specify something in the app source configuration so it will not try to untar or something like that ?

According to the helpers doc (ynh_setup_source) :

The src file need to contains:

[…]

# (Optional) If it set as false don't extract the source. Default: true
# (Useful to get a debian package or a python wheel.)
SOURCE_EXTRACT=(true|false)

Oh, thanks, I didn’t manage to find it ! :tada:

The app packaging main page is a bit… loose on some specific points like this :sweat_smile:

74sh7m

(The current packaging doc is absolutely horendous and outdated, i started to rework it from scratch but ugh there’s quite a lot of work x_x)

1 Like

I understand that… good luck with that amount of work, it’s very valuable ! :smiley:

Unfortunately I can’t help much as I don’t even know the basics of packaging :confused:

Hi,
I get the application working thanks to the answers on Github, but unfortunately the result isn’t at all convincing: I’ve tried with L’Humanité and Alternatives Économiques, and in both cases I can’t get past the paywall (whereas I can with the Bypass Paywalls Clean extension for Firefox).

It’s clearly not working 100% of the time, it’s only a trick for some special cases…
Does it work with https://12ft.io ?

Yep. In fact I’m facing two different issues:

  • with Alternatives Économiques, it works with 12ft, not at all with 13ft
  • with L’HumanitĂ©, it works with 12ft and 13ft, but the style of the page is “destroyed” with the last one (as if css didn’t load)

I’ll just have to find which websites 13ft is capable to browse :slight_smile:

1 Like

I’m curious to figure out if it’s an 13ft issue or a Yunohost packaging one…
But I don’t know how to check that :thinking: