/e/ Cloud - selfhosting

,

Hello !
je viens de proposer à la team de /e/ de packager leur solution de cloud auto-hebergé pour Yunohost.
Leur solution est vraiment chouette et ça vaudrait le coup, à mon sens (à commencer pour mon propre usage perso, et j’imagine celles et ceux dans ma situation), de voir les efforts de /e/ et de Yunohost se rejoindre sur cette solution.
Mon issue gitlab est ici : https://gitlab.e.foundation/e/infra/ecloud-selfhosting/issues/76

On va voir ce que ça donne :slight_smile:
Mais si certain-e-s d’entre vous, plus opérationnel-le-s que moi sur le packaging d’app sont partant-e-s pour y contribuer, ce serait chouette !

merci !


Hello !
I just posted an issue on /e/ gitlab in order to make their self-hosted cloud solution a packaged app for Yunohost.
I think it would be great (at least for people like me) to see /e/ and Yunohost in a common effort to bring this cloud solution as a Yunohost app.
link to my issue on gitlab is here : https://gitlab.e.foundation/e/infra/ecloud-selfhosting/issues/76

Let’s see what’s coming from this. :slight_smile:
But basically, if any of you, more able than me to package an app for Yunohost, is available to help them get this done properly, it would be great!

thanks!

6 Likes

Hmyea that’s something which was already suggested in https://forum.yunohost.org/t/yunohost-et-e/8787and we did not really continue the discussion because I guess we were too busy with many things …

I still don’t know much about /e/ … my understanding is that it’s a smartphone OS that is able to connect to a cloud of services managed by the /e/ people but there’s a strong wish that you should be able to self host all these services ? And that’s apparently what https://gitlab.e.foundation/e/infra/ecloud-selfhosting helps doing ?

With that said it’s really not clear to me if this is really something that is packageable to YunoHost : it sounds way too “meta” ? I mean this in the sense that it sounds like the repository is not about setting one specific web app, but more about setting up a whole bunch of services including a mail stack for example.

So I really doubt that you can have this as a Yunohost app, because it looks very much like it is itself an ecosystem, and you can’t have two very different ecosystem on the same machine … (even though the software are probably very similar, still, they are probably configured differently)

What I think would make more sense is to make Yunohost compatible with /e/, or viceversa make /e/ compatible with a Yunohost server … I have no idea what it implies though. But that would make perfect sense as we’re regularly thinking about how it’s annoying to install and configure every smartphone apps by hand, and how instead an integration between personal servers with smartphones would really contribute in adding meaningfulness and smoothness to the whole degooglization process to the general public !

In fact, I’m really afraid that this ecloud-selfhosting repository is just reinventing the same wheel as YunoHost, FreedomBox, Sandstorm, Cloudron and other friendly self-hosting distributions … But I don’t know if the /e/ folks realize how time consuming it’s going to be on the long-term to manage the various pieces of config, maintaining/upgrading them and the base system (Debian in our case), app packaging (and upgrades!), DNS configuration and port forwarding tutorials, the mail stack nightmare, security considerations, diagnosis, and making sure that all this is as easy as it can be for the end user (and certainly YunoHost itself is still only easy for tech-savvy folks…).

I don’t know how much thinking went into this self-hosted /e/ cloud or what its true goal is, but imho it’s either going to remain something that only only a few nerds-with-a-lot-of-free-time-and-energy will be able to setup and effectively maintain in the long term, - or - if they truly want to build a user-friendly self-hosting solution (and my understanding is that it’s not the priority of /e/ but I might be mistaking or it could evolve) they are going to have to invest several people full-time on it.

Instead, it would be really nice to see them choose an existing self-hosting project and trying to create compatibility with it… Of course here we’re on YunoHost forum, but any solution they choose among YunoHost, Freedombox or Sandstorm or whatever would make more sense in the long term than reinventing this wheel themselves… Though I do understand that in the short-run, writing your very own stack means having quick control over all the tiny details and therefore allow to quickly make progress.

Anyways … With that said, personally I’m already way too busy with everything going on in YunoHost itself and other related projects. But there’s really something meaningful to work on here if anybody is interested … Like “filling the gap between /e/ and YunoHost” … installing a Yunohost, installing a smartphone with /e/, and trying to see if and how you can connect pieces of one to the other, or what it would take to make it work. I guess if we had a list specific details to implement in YunoHost to make it compatible with /e/ we could work on it as long as it’s compatible with the philosophy of the project :stuck_out_tongue_winking_eye:.

Sorry for the wall of text :man_shrugging:

8 Likes

That’s hell of an answer :ok_hand:t2::rofl:

Thank you for all the precise points you made.
And now I understand way more how those two projects can cooperate at some point, or at least, be helpful for each others.

I definitely would like to administrate my self-hosted /e/ cloud services the way I do with my yunohost.

And yes, I believe in some kind of all-inclusive service to be more efficient that separated apps when it comes to ungoogle the way non-geeks use their smartphones, email services, clouds and everything.

3 Likes

Yes, in fact it can’t be stressed enough but I believe one thing we (the FOSS community) are failing is realizing how much of a killer-feature it is the way Google is able to deliver a set of unified integrated services … I don’t know exactly how that works precisely, but I’m guessing that for example you can easily attach a file from Drive to a mail in GMail ? And open said file in Google Docs ? Also their services have a unified esthetic.

That’s also why Framasoft’s Degooglize campaign was succesful, they did not just installed a bunch of services and wrapped them with some clumsy tape and call it a day : they worked hard on the feel and esthetic and integration on their platform and set of services so that everything looks simple and consistent. Imho the CHATONS / LibreHoster project need to acknowledge and work on this to be succesful especially now that Framasoft’s plan is to close services.

Unfortunately in YunoHost we “only” have the SSO aspect - which is already quite nice - but apart from this the application themselves all have different looks and I’m guessing you can’t attach a file from Nextcloud to an email in Rainloop …

Nextcloud is much better at this since they have a unified / integrated environment of apps which allow this (at least in theory? never tried all this). But the major drawbac is that they have to throw away existing FOSS software and reinvent the wheel in their own framework (e.g. a task manager, a mail client, a social app, …). Sandstorm tried something different with their ‘grain’ system but that was not really straightforward either.

/e/ is also apparently putting quite a lot of effort in creating a consistent and well-integrated ecosystem on smartphone, so that’s very nice to see :+1:

The dream would be to see all those major FOSS software (Nextcloud, Rainloop, Collabora, Matrix, Wekan …) which cover the most fundamental internet services to work much closely together on being able to interface and integrate with each other, both in terms of features and esthetic … :confused:

That’s kinda related to the fact that we (the FOSS community) have a hard time attracing designers/graphisms and having development process where they can easily contribute on the “not-code” stuff

Anyway I’m wandering a bit far away from the original topic :stuck_out_tongue:

6 Likes

But this wandering could be helpful to both communities imo.

I guess /e/ is trying its best to get the best out of those open-source projects to build an entire unified ecosystem, from laptops OS to cloud services through, of course, smartphones ROM.

It is to me, so far, the initiative that understood the most the needs of people like me (no that geek, but not that noob) and our parents (non-geeks at all).

That’s why I’m trying my best to connect both communities and knowledge of Yuno and /e/.

4 Likes

Hello guys, very very interesting topic here. Thanks @Aleks for your long messages. I agree with what is said here.
I can tell you a few things, as I have yunohost server, and an /e/ phone :wink: and moreover I sometimes contribute in translating /e/ contents to french (website, newsletter, and one day I hope apps).

So actually I do connect everything manually :

  • Nextcloud
  • Nectcloud notes
  • Nextcloud Caldav and cardav
  • Nextcloud PhoneTrack/Maps
  • Nextcloud SMS
  • Emails in the K9-fork app
  • Password manager (bitwarden actually, previously keepass+keeweb)
  • Tiny tiny RSS

It is manual but it works.
I think one first step would be to define a kind of “auto discover feature”, like the .well-knows addresses for CalDav and CarDav or like Microsoft uses for exchange in cloud servers, so that an /e/ phone could find with just on URL (and obviously username and password) where are mails servers, nextcloud stuff, contacts and calendars, etc.

I have no time (and probably not enough skills) actually to do that, but I would be happy to see this being done. :stuck_out_tongue:

4 Likes

Exactly the same here.

I guess I’m late to the party but I’m still I’m going to pull this thread all the way up to the top. The reason is because I can totally understand the sentiment behind this ask.

Let me preface by saying that I’ve read every single word in this thread ( yes, all of Aleks’ posts ) and the linked gitlab issue ( though there wasn’t much to read there ).

Now, the reason why I’m jumping in here is not because I understand the sentiment; but because I know this is possible today. I do it myself ( though I’m kind of cheating in a way, but still ). I’m a YunoHost user and a /e/ user myself.

So, brace yourself, this is probably going to be longer than Aleks’ posts; but I’m going to explain how to get this done with minimal to no effort.

Let me start off with some tl;dr :

  1. you don’t need /e/'s self-hosting solution
  2. all you need is just plain vanilla YunoHost with the Nextcloud application and you’re good to go.

The long version :

/e/ Cloud

The /e/ cloud is just a glorified Nextcloud installation with an email provider ( don’t give it too much credit than it deserves )

Email

You already have YunoHost doing a great job with email handling. Let YunoHost continue to do the good job it is already doing.

Nextcloud

/e/ is not doing anything special with their version of Nextcloud other some theming as far as I’ve seen. The reason I’m saying this is because I’m running my own Nextcloud and my /e/ OS works without any issues at all.

All you need is to install the Nextcloud app on YunoHost ( go through the effort of setting it up right; I don’t want to cover that here ) and then go ahead and login in your “/e/ account”.

This is where it kind of gets tricky. There’s a hidden “Server URL” field hidden under that big “Login with an other account” option. All you need to do is put in the URL for your nextcloud instance into that field, put in the right credentials to your user account on Nextcloud ( or YunoHost account; since YunoHost hosts a single LDAP database for the whole server ) and you’re good to go.

All of your default apps start syncing over to your own instance of Nextcloud running on YunoHost :metal:

Caveat

There’s only one issue that you’ll face. It is with the mail app which wouldn’t sync to a custom email server. That’s probably because /e/ uses a different email server than what we use.
That’s an easy fix though. Just clear the Mail app’s data and manually add your email account to it ( more on that below )

Now, as I visited the /e/ forum looking for that post that I linked above, I found this post that says that one would be better off linking WebDav accounts for calendar and contacts and the Nextcloud client for the remainder of the syncing.
However, I haven’t had any issues in the 5 months that I’ve been using the OS. With that said, we should also give them the benefit of the doubt if they have to say that. But they also say that this well known bug will be solved soon. So, I’m just simply going to say - Your Mileage May Vary.

Email Setup - Auto Discovery

Now, what most people find difficult even after having all the heavy-lifting of setting up the mail server right is figuring out how to configure their email clients to actually talk to the mail server.

Enter SRV DNS records. SRV records are DNS records that can be used for service discovery. This can be put to use to set up for your domain.
All you need is a few DNS records and then all of your email clients will just automagically configure themselves when you just put in your email server domain.

  • _imap(s)._tcp.domain.tld
  • _submission._tcp.domain.tld

these are the 2 important SRV records that are required to configure email clients.

Now, you can have other records related to email like _pop(3)(s)._tcp.domain.tld if you have POP3 enabled on your server ( I don’t think YunoHost enables it though; if I recall correctly, YunoHost opens only 2 ports - 465 and 993 ); but those are not required.

Once you have these SRV records in place, any email client should be able to pick up your mail server configurations easily.

Going back to where I left about the email client on /e/, it would be trivial to set the default Mail app up even if you clear the app’s data and have to start over. Just point the client to the right domain and the SRV records will take over from there. The mail client should be able to discover and configure the client with most of the configuration from there.

Cheating

Finally, I did mention in the beginning that I was kind of cheating when I said that I was using this setup with YunoHost. Actually, I don’t use YunoHost for hosting my personal email or cloud. I have them set up on different servers from YunoHost. But that doesn’t mean that I don’t use YunoHost. I use it for everything other than my email and cloud. In fact, I trusted YunoHost with all of my personal email until very recently. It still hosts some other emails on it; just not my personal email.

I got smart with the whole email setup only very recently. Trust me, it definitely is the kind of thing that you put up the big warning DO NOT TRY THIS AT HOME. You can never get all of it right. Even if you get all of it right, you’ll never know for sure because it’ll take months to test the set up in terms of server reputation. And it’s not even about the software that you configure on the server, it’s about the way your server interacts on the network that makes all the difference. And if you get dealt a bad hand ( like an IP from a blacklisted CIDR ), you’ll never know for months.
And that’s why I want to emphasis on how great of a job YunoHost is doing at least on the server software side. You can’t fix the other issues with software, so, that’s out of the scope of YunoHost. I can say this because I’ve tried this ( and failed multiple times ) and am speaking from experience. I cheated even there and compared notes with YunoHost to understand what was going wrong with my configurations.
If you’re adventurous enough, you can go ahead and try setting up your mail server but please don’t route your production emails to it unless you’ve confirmed at least 3 months of consistent good reputation by at least 2 major email providers - Google and Microsoft. You’ll realize how hard it is to get to the right configuration in terms of software.

I did have an instance of Nextcloud on my YunoHost instance but I didn’t use it that extensively though. I think it’s still there too. I set up a fresh Nextcloud instance when I decided to switch to /e/ OS ( and also because I had grander plans ). My YunoHost instance is a little choked on resources; so I didn’t want to choke it any further with all the regular syncing coming from my phone.

The reason why I’m admitting all of this is because I don’t want you to think later that I was a hypocrite to give you all this lecture only to say that I don’t walk my own talk. You can choose for yourself whether you want to take the lecture or not. Your choice.

Conclusion

YunoHost does a great job of hosting your email for you. There’s no reason why you’d need someone else’s solution at all. And since you already have YunoHost, and it lets you host Nextcloud, why not set up Nextcloud there and go ahead, set up some DNS records, flash your phone with /e/ OS, point it to your Nextcloud instance and you have your own /e/ cloud self-hosted.

10 Likes

Thanks a lot for your long answer!

As a /e/OS and Yunohost user, I’m going to try to follow your steps. Not for emails…

when I see this long answer with all the details and caveats that you need to take into account… I’d definitely say that /e/ OS users could use a simple UX solution for /e/ cloud on Yunohost :wink: So, I would be all for if we could have an /e/OS ynh app if that simplifies things.

It seems that they made some modifications.
The one I am thinking about is the sms synchronisation.
It seems that they are using a fork of the version available in nextcloud and a fork of sms sync for Android.
Source : https://github.com/nerzhul/ncsms-android/issues/230

So not everything is that simple.

(But I use /e/ without any /e/ account, just YunoHost (and mails hosted elsewhere)

Quick question about this king of mobile OS : i’m currently using Ubuntu Touch and a bunch of yunohost hosted services (nextcloud, bitwarden, matrix chat, subsonic). I’m pretty much satisfied and it got installed and synced really easily but a few things catched my eye on /e/. Sad thing is i cannot find any demo videos or even a list of available apps, do you have any of that ?

1 Like

/e/ is an Android ROM based on LineageOS.
The main thing is that there is also microG incorporated (an open-source implementation of Google Play Services) so most of Android application that needs the Google Play services works.
The list of available applications is mostly « any Android app » (and they have their own app store with apps from F-Droid and from Google Play Store)
If you want to stay « clean and pure », you can use only F-Droid.
For the main F-Droid repo you will find the available apps here : https://f-droid.org/
But some other repos exist (Bitwarden have it’s own F-Droid repo for example).

I hope this help

3 Likes

Yep thanks, well it’s sold i migrated to /e/ and it’s all easily linked to nextcloud and such.
Ubuntu touch was cool but had a huge power mangement problem and a few other weaknesses.

1 Like

The issue on /e/ gitlab has been closed one year ago.
However, there is active work since 2 months on the docker-based solution. Also an outdated ansible version is still present.

I would like to push the topic /e/OS+Yunohost again through the CHATONS.org collective, thus posted an issue there.

If someone has a clear view on how it should be implemented, feel free to post here or there. Then we will try to contact people at /e/OS again to see if they can help.

1 Like

Small update

1 Like