[Outline] An open, extensible, wiki for your team

Congrats! And thanks again for the great app(s)!

Right now nothing could prevent that unfortunately. We could implement a hook, unfortunately there do not seem to be a pre_app_remove hook. My idea would have been to have Outline create such a hook for Minio and Dex, that would force their removal to ynh_die --message="$app is required by Outline".

That may be something we could cook up for YunoHost 11, but also in the packaging v2 as a “app dependency” in addition to “apt dependency”.

What do you think @Aleks?

zblerg yeah i don’t know what to think about this because we’re talking about essentially creating the whole set of logic about app dependencies and there’s probably much more than just this situation that arises… c.f. also the recent discussions about the new helper that allows to install an app, developed with the context of prosody/jitsi/peertube in mind if i remember correctly

Adding a pre/post_app_remove hook is probably easy, but I’m not sure that this would answer the issue being discussed here ? If I understand correctly, what you’d want is logic to prevent app A from being removed if it’s a dependency for app B as long as app B is installed. Currently, hooks are only designed to perform custom stuff before/after an action (e.g. app install) is performed, but not to prevent the action from being run, afaik.

Hi @Aleks and many thanks for your answer.

Your understanding is correct. And you are right, going too far would basically mean re-developping a full in-house “package manger” with dependancies management which would probably be too complex.

I have a kind of “middle-ground” idea, which would be semi-manual, only assistance to the admin.

  1. Apps could have a standard key, for instance requiredBy, which would have as a value a list of apps.
  2. During the install process, the dependant app (in my case, Outline) would append its name to the requiredBy value of all the apps it depends on (in my case, Dex and MinIO).
  3. We could then have Yunohost checking for that requiredBy key when the admin tries to remove the app : if it’s not set, then the app is just removed as usual. If it contains something, the admin would receive a warning box (just like the one we have when installing an app on the root of a domain) saying like “Outline app is dependant on MinIO. Are you sure you want to uninstall MinIO ? This may break Outline”
  4. Reversely, we could have a hook such that when the admin removes an app, it would scan all other installed apps for requiredBy key. The admin could then receive a warning box saying something like “MinIO was required by Outline that you have just uninstalled. Do you still need it ?”

Anyway it works correctly for now, and I’m happy with the result !
I will migrate to the new helper once it will be official.

With pleasure. I don’t have much merit as I was working for myself as an Outline user :slight_smile:
Waiting for the inline comments now… But for this I don’t have the competence to help… Very heavy typescript / react / node app, way above my simple bash / python knowledge :sweat_smile:

We discussed the matter during last night’s meeting.

The ynh_install_app helper is now merged, and will be available in YunoHost 11. It already adds a apps_dependencies setting to the main app, a comma-separared list of sub apps IDs.

We can now alter the behavior of the app removal routine in YunoHost core to more or less add the checks you were describing above, mainly preventing apps removal when they are needed. This wil be done at a later point.

1 Like

Good to know !
Unfortunately my time zone prevents me to attend these meetings.
I will start having a look at the new helper. Does it mean that if I use it in my install script outline won’t be installable without Bullseye ?

It would be more appropriate to have requires key instead of requiredby. In case the app that is considered as dependency let’s call it app A and we have two apps B and C that depend on A. If one day we add an app D that is also dependent on A, it would be easier to just add the key requires app A to the script of D, instead of going to the app A and append D to the list requiredby B and C.
I hope you understood me.
This will lead to change how yunohost will act when installing and uninstalling an app. It will need to check all apps ids before that.

I guess you can add it in the _common.sh file in the meantime. :slight_smile:


@jarod5001 yes indeed it’s better to list the dependencies in the “mother” app settings. And yeah the future function that will check these will always need to list all apps settings and compute them. But they are already generated in YAML so it would be straightforward.

1 Like

Thanks a lot for your work.
I have been trying to deploy it since weeks and I found out your solution and it finally works!

However, I am not sure how do I configure the https://raw.githubusercontent.com/outline/outline/develop/.env.sample file

I want to enable Google Auth and disable yunohost auth) and allow only a specific domain name.
Also, I would like to setup the SMTP configuration.

Thanks a lot for your time.

Hi @luker and welcome to the Yunohost forum.

Did you discover Yunohost through Outline’s github discussions ?

To enable Google Auth and disable Yunohost auth

  1. 1st of all you have to know that Outline can’t have two different auth methods activated in self-hosted mode at the moment. It should be quickly added as a feature, but meanwhile if you have already added a first user through Yunohost auth then even if you change the .env file, the Google auth won’t appear. You’ll have to install again Outline from scratch and then follow the steps below
  2. Once you have Outline installed on Yunohost, do NOT login yet and go to /var/www/outline to change the .env file and add your google env parameters (and remove Yunohost / Dex parameters which are under the OIDC setup). FYI a standard Google account won’t work. You need a professional account for it to work.
  3. Once the .env file is changed (make sure the ownership of the .env remains outline:www-data) you can either run the command sudo yunohost service restart outline or do it using the web ui
  4. Normally in the future you’ll be able to update your Outline instance through the standard Yunohost process without having to redo the above steps

To setup SMTP server

  • Do you mean an external SMPT server that is not Yunohost one ?
  • To do so I guess you just need to put the right config parameters in /var/www/outline and then restart Outline just like above

Hope this helps.
Don’t hesitate to try other apps to add to your Yunohost !

Thanks a lot!
It is working well now.
Yes, I meant the external SMTP service.
I modified from .env for the same and it working well now.

Thanks man. You helped a lot :slight_smile:

Regarding from where I found about it, then yes, I found you via github outline issue.

great !

With pleasure

Nice if that could help someone self-hosting Outline !

Hello!

I tried installing this but I can’t seem to login, I grand access in Dex and then it redirects to the login screen, it’s stuck in a loop.

What URL’s did you test this with?

I’m using wiki.domain.tld for outline
domain.tld/wikiauth for dex
minio.domain.tld for minIO

I also installed yesterday and am having the exact same problem! Only difference in configuration is I installed Dex on a subdomain, auth.domain.tld.

Hi @sammcf and @hanthor,
Welcome to Yunohost forum !

I’m sorry that installation didn’t work out as expected, let’s try to find what happened.

  1. Did you secure all domains with à Let’s Encrypt Certificate already ?
  2. Did you previously try to install Outline using the same domain ? If yes you should clear your browser cache as I have faced similar loops problems already when the tokens are not up to date
  3. When you are logging in Dex, do you see the page where it shows a green banner and asks whether you agree to share email and name with Outline ?

Thanks for your welcome, @Limezy, and also for the package!

Yes, I had a certificate configured for each domain before installing the app(s).

No, only tried the once! But I reinstalled and cleared browser cache just to see if this would help, no change.

Yes, the process is as follows:

  • select Outline from SSO main page
  • offered option to sign in with Yunohost or continue with email → select sign in with Yunohost
  • redirected to Dex, with sign in form - URL is /auth/ldap/
  • enter Yunohost account credentials and hit login
  • shown “Grant Access” page (Outline would like to view basic profile information, email address), URL is /approval?req=blahblahblah
  • select Grant Access, redirected to outline domain /collections/welcome-blahblahblah then immediately redirected to Outline login page at tld.
  • attempting to access any other Outline pages by manually entering a URL immediately redirects to login page

Hope that provides some clue!

Hi ! OK so it’s not a basic problem.

Looks indeed like a package bug that for some reason is not detected by the CI and that I didn’t detect before. Let’s have a look a bit deeper !

  • Do you know how to access to the web console ? If yes would you mind checking what kind of errors are shown their if there are any ?
  • Do you know how to access the file system of your server ? If yes would you mind copy-pasting here your /opt/yunohost/dex/config.yaml file and your /var/www/outline/.env file ? (removing sensitive information first of course)

On my side I’ll try a fresh install on a new VPS to try reproduce your loop.
Thanks for your help… beta-testing what I thought was now quite stable…

Well, I can’t reproduce, it works well for me !
The URL sequence should be the following :

  1. https://auth.domain.tld/approval?req=qllh5bchzi5h7idq7ueihku75
  2. https://wiki.domain.tld/auth/oidc.callback?code=n4jcq3fo6rmua6qtbf3elpzrt&state=7838d374ac7290c3
  3. https://wiki.domain.tld/collection/welcome-ikIA4k5c06

It seems that the two first ones are working well since you can see the /collection/welcome-xxx one. The bug seems to be in the account creation and provision that happens when a new users logs in for the first time.

Could you please also check the logs found here : domain.tld/yunohost/admin/#/services/outline ?

OK, digging through the console a bit, that URL sequence is happening correctly, but we fall over here:

Failed to load resource: the server responded with a status of 401 ()

with an api.auth response of:

{"ok":false,"error":"authentication_required","status":401,"message":"Authentication required"}

I’m getting a fair payload of js & html, viewable in the sources list and I can see the 200 responses in the network panel, including the welcome collection assets, but not getting authed. Mysterious!

  • /opt/yunohost/dex/config.yaml
root@domain:/opt/yunohost/dex# cat config.yaml
# LDAP connector + Yunohost setup + staticClient as per manifest.json
issuer: https://auth.domain.tld/
storage:
  type: sqlite3
  config:
    file: dex.db
web:
  http: 127.0.0.1:5556

frontend:
  issuer: dex
  logoURL: theme/logo.png
  dir: web/
  theme: light

connectors:
- type: ldap
  name: OpenLDAP
  id: ldap
  config:
    host: localhost:389

    # No TLS for this setup.
    insecureNoSSL: true
    insecureSkipVerify: true

    usernamePrompt: Username

    userSearch:
      baseDN: ou=users,dc=yunohost,dc=org
      filter: "(objectClass=InetOrgPerson)"
      username: uid
      idAttr: uid
      emailAttr: mail
      nameAttr: displayName

    groupSearch:
      baseDN: ou=groups,dc=yunohost,dc=org
      filter: "(objectClass=posixGroup)"

      userMatchers:
      - userAttr: uid
        groupAttr: memberUid

      nameAttr: cn

# Unfortunately the api is too complex to be used here
# As a consequence we have to setup client as staticClient, which means we will need one Dex instance per client app
staticClients:
- id: Outline
  redirectURIs:
  - https://wiki.domain.tld/auth/oidc.callback
  name: Outline
  secret: blahblahblahblah
  • /var/www/outline/.env
root@domain:/var/www/outline# cat .env
# –––––––––––––––– REQUIRED ––––––––––––––––

# Generate a hex-encoded 32-byte random key. You should use `openssl rand -hex 32`
# in your terminal to generate a random value.
SECRET_KEY=blahblahblah

# Generate a unique random key. The format is not important but you could still use
# `openssl rand -hex 32` in your terminal to produce this.
UTILS_SECRET=blahblahblah

# For production point these at your databases, in development the default
# should work out of the box.
DATABASE_URL=postgres://outline:blahblahblah@localhost:5432/outline
DATABASE_URL_TEST=postgres://user:pass@localhost:5532/outline-test
# Uncomment this to disable SSL for connecting to Postgres
PGSSLMODE=disable
REDIS_URL=redis://localhost:6379

# URL should point to the fully qualified, publicly accessible URL. If using a
# proxy the port in URL and PORT may be different.
URL=https://wiki.domain.tld
PORT=2410

# See [documentation](docs/SERVICES.md) on running a separate collaboration
# server, for normal operation this does not need to be set.
COLLABORATION_URL=

# To support uploading of images for avatars and document attachments an
# s3-compatible storage must be provided. AWS S3 is recommended for redundency
# however if you want to keep all file storage local an alternative such as
# minio (https://github.com/minio/minio) can be used.

# A more detailed guide on setting up S3 is available here:
# => https://wiki.generaloutline.com/share/125de1cc-9ff6-424b-8415-0d58c809a40f
#
AWS_ACCESS_KEY_ID=sam
AWS_SECRET_ACCESS_KEY=blahblahblah
AWS_REGION=fr-ynh-1
AWS_S3_UPLOAD_BUCKET_URL=https://bucket.domain.tld
AWS_S3_UPLOAD_BUCKET_NAME=outlinestorage
AWS_S3_UPLOAD_MAX_SIZE=26214400
AWS_S3_FORCE_PATH_STYLE=true
AWS_S3_ACL=private


# –––––––––––––– AUTHENTICATION ––––––––––––––

# Third party signin credentials, at least ONE OF EITHER Google, Slack,
# or Microsoft is required for a working installation or you'll have no sign-in
# options.

# To configure Slack auth, you'll need to create an Application at
# => https://api.slack.com/apps
#
# When configuring the Client ID, add a redirect URL under "OAuth & Permissions":
# https://<URL>/auth/slack.callback
SLACK_KEY=
SLACK_SECRET=

# To configure Google auth, you'll need to create an OAuth Client ID at
# => https://console.cloud.google.com/apis/credentials
#
# When configuring the Client ID, add an Authorized redirect URI:
# https://<URL>/auth/google.callback
GOOGLE_CLIENT_ID=
GOOGLE_CLIENT_SECRET=

# To configure Microsoft/Azure auth, you'll need to create an OAuth Client. See
# the guide for details on setting up your Azure App:
# => https://wiki.generaloutline.com/share/dfa77e56-d4d2-4b51-8ff8-84ea6608faa4
AZURE_CLIENT_ID=
AZURE_CLIENT_SECRET=
AZURE_RESOURCE_APP_ID=

# To configure generic OIDC auth, you'll need some kind of identity provider.
# See documentation for whichever IdP you use to acquire the following info:
# Redirect URI is https://<URL>/auth/oidc.callback
OIDC_CLIENT_ID=Outline
OIDC_CLIENT_SECRET=blahblahblahblah
OIDC_AUTH_URI=https://auth.domain.tld/auth
OIDC_TOKEN_URI=https://auth.domain.tld/token
OIDC_USERINFO_URI=https://auth.domain.tld/userinfo

# Specify which claims to derive user information from
# Supports any valid JSON path with the JWT payload
OIDC_USERNAME_CLAIM=preferred_username

# Display name for OIDC authentication
OIDC_DISPLAY_NAME=Yunohost

# Space separated auth scopes.
OIDC_SCOPES="openid profile email"


# –––––––––––––––– OPTIONAL ––––––––––––––––

# Base64 encoded private key and certificate for HTTPS termination. This is only
# required if you do not use an external reverse proxy. See documentation:
# https://wiki.generaloutline.com/share/1c922644-40d8-41fe-98f9-df2b67239d45
SSL_KEY=
SSL_CERT=

# Base64 encoded private key and certificate for HTTPS termination. This is only
# required if you do not use an external reverse proxy. See documentation:
# https://wiki.generaloutline.com/share/1c922644-40d8-41fe-98f9-df2b67239d45
SSL_KEY=
SSL_CERT=

# If using a Cloudfront/Cloudflare distribution or similar it can be set below.
# This will cause paths to javascript, stylesheets, and images to be updated to
# the hostname defined in CDN_URL. In your CDN configuration the origin server
# should be set to the same as URL.
CDN_URL=

# Auto-redirect to https in production. The default is true but you may set to
# false if you can be sure that SSL is terminated at an external loadbalancer.
FORCE_HTTPS=true

# Have the installation check for updates by sending anonymized statistics to
# the maintainers
ENABLE_UPDATES=true

# How many processes should be spawned. As a reasonable rule divide your servers
# available memory by 512 for a rough estimate
WEB_CONCURRENCY=1

# Override the maxium size of document imports, could be required if you have
# especially large Word documents with embedded imagery
MAXIMUM_IMPORT_SIZE=5120000

# You can remove this line if your reverse proxy already logs incoming http
# requests and this ends up being duplicative
DEBUG=cache,presenters,events,emails,mailer,utils,http,server,processors

# Comma separated list of domains to be allowed to signin to the wiki. If not
# set, all domains are allowed by default when using Google OAuth to signin
ALLOWED_DOMAINS=

# For a complete Slack integration with search and posting to channels the
# following configs are also needed, some more details
# => https://wiki.generaloutline.com/share/be25efd1-b3ef-4450-b8e5-c4a4fc11e02a
#
SLACK_VERIFICATION_TOKEN=your_token
SLACK_APP_ID=A0XXXXXXX
SLACK_MESSAGE_ACTIONS=true

# Optionally enable google analytics to track pageviews in the knowledge base
GOOGLE_ANALYTICS_ID=

# Optionally enable Sentry (sentry.io) to track errors and performance
SENTRY_DSN=

# To support sending outgoing transactional emails such as "document updated" or
# "you've been invited" you'll need to provide authentication for an SMTP server
SMTP_HOST='localhost'
SMTP_PORT='25'
SMTP_USERNAME=''
SMTP_PASSWORD=''
SMTP_FROM_EMAIL='outline@wiki.domain.tld'
SMTP_REPLY_EMAIL='webmaster@wiki.domain.tld'
SMTP_TLS_CIPHERS=
SMTP_SECURE='false'

# Custom logo that displays on the authentication screen, scaled to height: 60px
# TEAM_LOGO=https://example.com/images/logo.png

# The default interface language. See translate.getoutline.com for a list of
# available language codes and their rough percentage translated.
DEFAULT_LANGUAGE=en_US

Hopefully there is some hint in there! Thanks again for your help :smiley:

Wow that’s very strange, everything looks very nominal.

I have checked in Outline source code, and the only case where that error will be given is here :

In other words, it looks like Dex is not producing any token, or Outline is not receiving the token…
Could you check if you see the tokens in the browser storage ? Which browser are you using by the way ? Did you try in incognito mode ?

Could you please share your /var/log/dex/dex.log ?