Simplifying Yunohost installation with a reverse proxy and tunnel managing service

Discuss
1

Hello good people -

Could the installation of Yunohost be simplified dramatically (thereby making self hosting available to many more people) by a relatively low-cost simple reverse proxy service?

This would be a service that forwards requests from a URL to a yunohost intance. As a service, it would be similar to pagekite or ngrok other utilities that enable sharing a localhost / various services from machines behind firewalls.

Advantages:

  1. No need to open ports on router / firewall. Communication between the yunohost machine in the userā€™s home and the reverse proxy could be handled in the application, the user doesnā€™t have to know or care how.
  2. This would make it possible to make the dns / subdomain management / etc. handled automagically between yunohost and the service.
  3. Could still keep the SSL certs / etc. on the local yunohost, since the service is just in charge of forwarding requests.

This seems too easy - and would make self-hosting much more availble to the common person, right? Or am I missing something?

Thanks,
Amos

1 Like
2

Ah actually it looks like this is what Iā€™m looking for. It just seems like the whole ā€œgetting startedā€ process should get reworked around something like this.

https://boringproxy.io/

3

Hi,

I may be missing your point, but I donā€™t see how self-hosting with Yunohost would be easier with a reverse proxy.

  1. What ? Which ports wouldnā€™t you need to open ? Maybe ports that some specific applications need, but this is not that common. The user will still need to open the following ā€œstandardā€ TCP ports (depending of which services he uses) : 22, 25, 80, 443, 587 and 993.
  2. Isnā€™t it already automagically managed ? :slight_smile: You create the domains/subdomains and assign applications to them. How could it be easier for the user ?
  3. This isnā€™t really an advantage.

For the disadvantages, by adding a new service, the resource consumption and the complexity of the system will increase.

4

Hey SohKa - thanks for diving into this suggestion! Iā€™ll answer point by point.

  1. You would need to open 0 ports in the router - in fact you wouldnā€™t need to do anything at all with the router. Forward facing proxy services like boringproxy.io or ngrok allow you to map ports on your local server to the reverse proxy entirely through their application, which maintains a tunneling connection with the external reverse proxy. So the end user would not have to know what a port even is, much less configure one on their router.

  2. DNS / domain management. The user still has to configure their DNS in some way / add ports and subdomains for xmpp / etc. With a service like boringproxy (or ngrok) configured to reduce the burden on the yunohost user, you could dramatically reduce the need to understand or configure domains at all. For example - the default config could be to simply assign each app its own subdomain when it is installed (and do all the various xmpp / mail / etc. dns port stuff as well). This could be handled via requests between yunohost and the proxy service.

The user setup experience becomes: 1) Sign up for reverse proxy service (with customized config / setup for yunohost), 2) download and flash yunohost, 3) enter reverse proxy information / secret, 4) choose and install apps.

I think that would dramatically increase the number of people who can self-host. It would require that the service also handles some dns. but you guys have vastly more knowledge and experience about this. Am I missing something?

2 Likes
5

I guess one big ā€œno-noā€ is the fact that this would pretty much look like a yuno-cloudflare, with one centralized proxy (or man-in-the-middle depending on how you look at it) which :

  • is a single point of failure - after you start reaching 100~1000 instances depending on it, you start being really worried about what happens if the proxy goes down. Also legal authority may take you (the proxy owner) responsible to block illegal content.
  • privacy concerns: the proxy owner has quite a lot of power over all the traffic of people using the service. Even if the traffic is theoretically encrypted you still have access to plenty of metadata, not to mention the possibility or redirect (or not) HTTP to HTTPS - or to craft a Lets Encrypt certificate considering the DNS points to the proxy anyway

Nevertheless the idea is interesting if people would be made aware of those concerns ā€¦

A few other notes :

  • Not sure how it relates to the whole DNS story ā€¦ We already have nohost.me & other auto-configured domain ā€¦ to me this is pretty much independent of how/where youā€™re hosting the server (behind a reverse proxy or not)
  • Thereā€™s already a VPNclient app which pretty much solves the ā€œhow-to-bypass-NATā€ in a similar-but-different way. The 2 main issues right now are: a) itā€™s mainly designed for a specific .cube format created by the FFDN folks, and itā€™s not trivial for people where to find such a VPN provider ; b) additional monthly cost (4ā‚¬/month I think ?)
  • The whole ā€œhow-to-bypass-NATā€ issue is nonexistent if you just host on a VPS. Which, yes, isnā€™t ā€œpure self-hostingā€ because you donā€™t have control over the physical layer, but still represent a huge step forward in the grand scheme of ā€œnot depending on megacorps for your digital lifeā€
4 Likes
6

boringproxy is a combination of a reverse proxy and a tunnel manager

Thatā€™s what I was missing. Itā€™s not just about setting up a local reverse proxy.

@aleks summarized my thoughts in his previous post.

7

Ah - there had to be a catch. The current way to use boringproxy would be by hosting your boringproxy instance on a digital ocean droplet / VPS, so one would have to be subject to their terms for illegal content / etc. (The creator has plans for adding DNS config etc.) But I wonder if setting things up with yunohost such that it was easy to do this reverse proxy / tunnel with any external VPS would be worth exploring. You still have to trust the provider, but thatā€™s no worse than being fully on a VPS, right (and somewhat better if itā€™s only the metadata thatā€™s likely to leak.)

I think so! To me the primary barrier to wider adoption of self hosting is usability - and needing to open ports / do custom DNS are the main factors working against usability. And while the concerns you raise are absolutely real and relevant, I wonder if we couldnā€™t think of ways to engineer against them.

You are absolutely right - apologies for not making that clear from the start, my mistake! Iā€™ve edited the topic title to clarify.
And while Iā€™ve got you both here - thanks so much for all your work on YunoHost! Iā€™m excited about using it, as soon as I resolve my rather puzzling DNS config issue. :wink:

2 Likes
8

At the moment Iā€™ve been on hold for over 20 minutes with my ISP, and when they do finally answer and I ask them to configure my reverse DNS, they will say ā€œWhatā€™s reverse DNS?ā€
Edit: 1 minute after writing the above, my ISP answered, and said ā€œWhatā€™s reverse DNS?ā€ He is now calling his lifeline for expert help.
I post this here not because I need help - but because this issue could also be solved by a reverse proxy tunnel manager to a trusted service. So - I think itā€™s important and worth considering how we make life so much easier for yunohost users by still allowing them to host in their home, but removing all the complexity of router and DNS configuration from the equation, thereby opening up the possibility of self hosting for a massively larger percentage of the population.
And now Iā€™ve just heard back after the second hold, and they donā€™t support reverse DNS changes at all. sigh, oh well.

2 Likes
9

This would be helpful for users on ISPs which block various ports, who prefer to host on their own physical devices instead of ā€œjust put it on a VPS.ā€

Then a VPS which may require substantially fewer resources, and thus cost substantially less per month, can be used to proxy traffic from its public IP to Yunohost.

Every dollar not spent on hosting costs, such as by using oneā€™s own physical hardware, is helpful to users without much income.

1 Like
10

For example, if my ISP fails at some point, I use my phoneā€™s wifi hotspot as a backup.

Iā€™d love to use this kind of automatic public reverse proxy as a backup solution too, so I can connect the raspberry to the hotspot and keep my services online.

11

If the idea is to deploy boringproxy or other reverse proxy/tunneling software on a vps just for one self-hosted machine, why not just setup an openvpn server on this VPS and redirect all the traffic into itā€¦

We already have a vpn client app . We should consider to rework on easy deployment of vpnserver app.

In fact you need proxying only if you want to split one IPv4 onto 2 or more machines. But if you have a vps with 2 ipv4, you can create a ā€œvpn self-hostingā€ for 2 machinesā€¦
Proxying is harder to implementā€¦

12

Very interesting topic thank you.
From my (limited) understanding, OpenVPN is not considered optimal for this use, due to various issues including performance. Wireguard seems much more fit for the task. The developer of boringproxy seems to maintain a list of tunnelling tools on his github.
HomelabOS, another FOSS operating system for self-hosting, has the option to install exactly that sort of solution via what they call ā€œbastion serverā€ and they use tinc as VPN/tunnelling tool. tinc is not mentioned in the list above for some reason, although it seems to work well.

I am currently preparing my (future) Yunohost server setup and I intend to get a cheap VPS to use just for tunnelling. I am not comfortable with opening ports on my home router, among other reasons.

13

The term ā€œbastion serverā€ for something that will drecrease in several context the security is a bit strange IMHO.

I totally agree with you about wireguard or other vpn techno. But currently, vpnclient (openvpn) is the only easy app for vpn self-hosting on yunohostā€¦ The wireguard ynh app is currently a bit more difficult to setup. Contribution on that topic are welcome.

Note, yunohost 4.3 will improve upnp, so we hope in several situation, the user wonā€™t need to open port by itself.

1 Like
14

I donā€™t know how far you got with this but you might find looking at frp or rathole helpful.

15

I definitely second the idea of allowing the incorporation of Boring Proxy as a reverse proxy and tunnel manger within Yunohost on local machine, tunneled from a VPS. I am slowly self-learning the different elements of self-hosting. I have yet to open ports on my router, and have only run services locally, primarily within Docker on DSM7, Docker on Ubuntu, and on RaspberryPi. I have tried with limited success running some services on VPS with Caddy and Docker.

I also maintain my Wordpress website on a managed host which has some Softaculus integration for deploying a few extra apps at your domains/subdomains. I just discovered Yunohost, which has dramatically reduced the burden of setting up a hardware server. I am confident I could set up Yunohost on a VPS, but I donā€™t want to pay for as much VPS storage as I plan on using, ie Nextcloud, Hubzilla, art portfolio Wordpress, XMPP server.

Boring Proxy has made its installation and Tunnel configuration on a VPS and local machine almost as easy as flashing Yunohost to a SIM and deploying apps on a Pi. A Yunohost flavored Boring Proxy to deploy on a VPS of your choosing, or at a minimum, a Boring Proxy app for connecting the tunnel to the local machine after Boring Proxy is set up on the VPS would be amazing for people like me with limited experience and fear of opening ports on router, intimidation of configuring Dynamic DNS, etc. This could all be done as well through Cloudflare/Cloudflared Tunnels (Zero Trust Dashboard) but would violate Terms of Service depending on what you are hosting.

I know one person in this thread said that you already have instructions for doing VPN tunneling, but having personally tried to configure VPS NGINX + Tailscale tunneled to local NGINX + Docker, or a similar VPS Caddy + Tailscale to local server, is all a lot more of a nightmare than just pointing a domain to a VPS, installing Boring Proxy on the VPS, installing Boring Proxy on local server. To my novice eye, it is the missing piece of the puzzle and reduces security concerns of messing with router forwarding, firewall exceptions, etc.

16

Hi all, and a big thank you to @lightnin for raising usability concerns.

I agree that the usability of YH install could be greatly improved, particularly with some clearer instructions/recommendations, and perhaps the tunneling tool mentioned. I donā€™t understand that particular tech, but if it helps YH install be more accessible, then great!

My situation is a good example: I am currently troubleshooting a new Yunohost setup, and my main stumbling block again and again is exactly this point - how/where to set up reverse proxy, dynamic DNS and/or VPN with DynDNS service - so all my currently installed apps and email work. After three weeks and many attempts at workarounds I am still working on it.

To be clear, Iā€™m really enjoying the learning process, generally. And also, I know a lot of folks who I bet would love to host their own/a community YH server, but do not have the time, learning capacity, or functional capacity to navigate the currently circuitous process. I am willing to contribute to this process with the skills I have, should they be useful.

re the point about DNS, Iā€™m comfortable creating and manually editing my DNS records, but I have repeatedly got errors and blocks in my setup process every time the reverse proxy issue comes up.

I am a disabled, hard of hearing UX designer, with only a little server-side experience before this. ā€˜Call your ISP and ask them to set up reverse proxyā€™ is not an instruction that is available equally to all. I cannot call them, or receive calls from them. From my perspective, it seems like a more robust install instruction strategy would be to suggest DynDNS or tunnelling services by default, and provide a note to the effect of ā€˜this step may not be necessary for people to can set up reverse DNS via their ISP.ā€™

Boundary request: Please donā€™t give suggestions or critiques on how I can ā€˜fixā€™ or ā€˜overcomeā€™ my disability. I will not answer questions about my disabled life. If you reply, please stick to the content of the conversation. You are not entitled to school me on my life or functional capacity.

17

Feel free to open a dedicated thread regarding your issues. Have you tried the Redirect app in proxy mode? (though itā€™s quite simple and would require further manual NGINX tweaks to make some apps work)

Good news, we include the Lexicon library, to directly interface with your registrar API (if compatible, obviously) if you supply it with proper credentials.

Any contribution is welcome in the documentation, we have a tutorials section in there, and in the forum too. Core and apps contributors are a tad too focused on making the standard features work to delve into more complex issues right now.

What the fuck. Why would anyone do that. (rhetorical question, no need to answer further).

1 Like
18

Hi @tituspijean

Thank you for your detailed reply!

No! I havenā€™t tried this, and I will add it to my list of workarounds to attempt. I didnā€™t know it was an option, so thank you. :pray:

Again, I donā€™t know what that is, but I appreciate the tip. Currently my domain is registered at namecheap, and I handle my DNS records at Cloudflare.com.

With my patchy regional power and internet (!) it seems wiser that my DNS records are stored remotely than locally, say via pi-hole, where they will/might be wiped each time power cuts out. Cloudflare UX is much better for me than handling DNS via command line, or similar remote DNS services. Itā€™s a free service, and they seem to have been around for a long time (so a hopefully robust option). If there are glaring errors in this logic and/or setup that you see, please let me know!

Super! Iā€™m very keen to contribute to the basic onboarding docs. My only attempt was confusing and not successful.

Are there folks on the YH docs team you could direct me to who can help troubleshoot getting started with contributing?

I have no doubt once I get used to the docs contributing process, I will be able to do so unaided, and can offer many helpful suggestions.

Exciting!

Thanks, will do if I canā€™t get past one of the issues as I work through them.