HTTPs cert not renewing automatically, diagnosis report the domain appears unreachable

[mouse]

My YunoHost server

Hardware: Other ARM board / …
YunoHost version: 11.2.5
I have access to my server : Through SSH
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : yes
If yes, please explain: my_webapps, to run websites/apps with own domains on my ynh.

Description of my issue

Hi again ynh,

So I’m unable to automatically update the https certs for own of my my_webapp websites on ynh. it’s a simple static site with its own domain and some basic nginx config.

the diagnosis error is:

Domain anarchive.mooo.com appears unreachable through HTTP from outside the local network.
It looks like another machine (maybe your internet router) answered instead of your server.
1. The most common cause for this issue is that port 80 (and 443) [are not correctly forwarded to your server](https://yunohost.org/isp_box_config).
2. On more complex setups: make sure that no firewall or reverse-proxy is interfering.

because of this check, certs update fails. if i do updates via ssh/CLI, it works when i add the --no-checks flag.

i don’t know why this check needs to block the update. my box is accessible via http with all the other domains, so it is not a problem with port forwarding.

and for this particular domain, i force HTTPS in my nginx conf, by using:

server {
    listen 80;
    listen [::]:80;
    server_name my.domain.com

# ... other config here...

    location / {
	return 301 https://$http_host$request_uri;
    }

is this the cause of the diagnosis error?

i have had issues with this before:

Unable to renew certs - Domain appears unreachable through HTTP from outside the local network in IPv6, though it works in IPv4

forcing cert renew works, which suggests to me that its only the checks that don’t work. my ports and whatnot all seem to work fine. but this means that ynh auto cert renew doesn’t work, i guess? it’s frustrating that’s its so opaque. dig command run with my domain returns: ;; ANSWER SECTION: mydomain.noho.st. 3600 IN AAAA ipv6:address:here: does that mean one is set but only for ipv6? i’m out of my depth there.

due to them i disabled ipv6 on my router and server. again, i don’t see why i need to have ipv6 working flawlessly merely in order to automate updating HTTPS certs?

if this is only only hiccup, can i permanently disable this particular check before updating certs? i don’t want sites to break for these minor reasons. my site otherwise works perfectly fine.

[Aleks]

@mouse

i don’t know why this check needs to block the update.

It blocks the certificate renewal because, by definition, if this check does not pass, the certificate renewal is very unlikely to be successful

@mouse

and for this particular domain, i force HTTPS in my nginx conf

I’m not sure to understand why you need to enforce HTTPS : YunoHost already enforces HTTPS … actually the snippet you are showing is from the default config so i’m not sure to understand … however there are other specific snippets which should exist in the configuration, namely:

    include /etc/nginx/conf.d/acme-challenge.conf.inc;

    location ^~ '/.well-known/ynh-diagnosis/' {
        alias /var/www/.well-known/ynh-diagnosis/;
    }

and diagnosis and/or cert-renewal won’t work properly without these

Does yunohost tools regen-conf nginx --dry-run --with-diff shows anything specific ?

[mouse]

thanks for your answer Alex.

@Aleks

I’m not sure to understand why you need to enforce HTTPS : YunoHost already enforces HTTPS … actually the snippet you are showing is from the default config so i’m not sure to understand … however there are other specific snippets which should exist in the configuration, namely:

i do have the snippets you mention in my conf, i just didn’t quote them, sorry.

i tried the command you suggest, got a lengthy diff, so i backed up the config file and ran the command minus --dry-run, then restarted nginx. (i avoid updating my nginx files because then when ynh updates it often breaks my sites manual config.)

but web diagnosis still returns the same error.

is there anything else that might at risk of causing the issue? as i mentioned, it’s not port forwarding because all other domains work fine. (i checked my router and 80/443 are forwarded.)

if you say ynh already forces https, should i remove that config?

and as mentioned, ipv6 is disabled on the router.

can i run the http check manually somehow while still redirecting to https?

[Aleks]

@mouse

i tried the command you suggest, got a lengthy diff, so i backed up the config file and ran the command minus --dry-run, then restarted nginx. (i avoid updating my nginx files because then when ynh updates it often breaks my sites manual config.)

If you manually modified files, then you need to use --force to effectively apply the changes, otherwise it just says that the file appears to be manually modified and won’t be updated by yunohost … but it would be nice to know what you did modify in the first place, otherwise we are just brute-forcing the issue without understanding what’s going on, hence the recommendation to look at those with --dry-run --with-diff

[mouse]

i just saw my manual conf is in the domain.com.d/my_webapp.conf subdirectory and file. is it possible to keep all manual config in there and then happily override the domain.com.conf?

i actually think all my manual config is in there. all i needed to do was have an nginx fancy index config for a file listing on my site.

[Aleks]

Yunohost’s “core” regen-conf won’t modify the app’s nginx snippet, it will only apply what’s displayed by --dry-run --with-diff …

[mouse]

i realize the regen-conf file didn’t modify the file because it’s marked as manually modified, so i’ll try again.

[mouse]

server {
listen 80;
listen [::]:80;
-    server_name domain.com xmpp-upload.domain.com;
+    server_name domain.com;

access_by_lua_file /usr/share/ssowat/access.lua;

include /etc/nginx/conf.d/acme-challenge.conf.inc;

location ^~ '/.well-known/ynh-diagnosis/' {
-        alias /tmp/.well-known/ynh-diagnosis/;
+        alias /var/www/.well-known/ynh-diagnosis/;
}

+    
location ^~ '/.well-known/autoconfig/mail/' {
alias /var/www/.well-known/domain.com/autoconfig/mail/;
}
+    



location / {
-        return 301 https://$http_host$request_uri;
+        return 301 https://$host$request_uri;
}


+
+    include /etc/nginx/conf.d/yunohost_http_errors.conf.inc;

access_log /var/log/nginx/domain.com-access.log;
error_log /var/log/nginx/domain.com-error.log;
@@ -53,13 +54,15 @@
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/yunohost/certs/domain.com/crt.pem;
-    resolver 127.0.0.1 127.0.1.1 valid=300s;
+    resolver 1.1.1.1 9.9.9.9 valid=300s;
resolver_timeout 5s;


+    
location ^~ '/.well-known/autoconfig/mail/' {
alias /var/www/.well-known/domain.com/autoconfig/mail/;
}
+    

access_by_lua_file /usr/share/ssowat/access.lua;

@@ -68,50 +71,9 @@
include /etc/nginx/conf.d/yunohost_sso.conf.inc;
include /etc/nginx/conf.d/yunohost_admin.conf.inc;
include /etc/nginx/conf.d/yunohost_api.conf.inc;
+    include /etc/nginx/conf.d/yunohost_http_errors.conf.inc;

access_log /var/log/nginx/domain.com-access.log;
error_log /var/log/nginx/domain.com-error.log;
}

-# vhost dedicated to XMPP http_upload
-server {
-    listen 443 ssl http2;
-    listen [::]:443 ssl http2;
-    server_name xmpp-upload.domain.com;
-    root /dev/null;
-
-    location /upload/ {
-        alias /var/xmpp-upload/domain.com/upload/;
-        # Pass all requests to metronome, except for GET and HEAD requests.
-        limit_except GET HEAD {
-          proxy_pass http://localhost:5290;
-        }
-
-        include proxy_params;
-        add_header 'Access-Control-Allow-Origin' '*';
-        add_header 'Access-Control-Allow-Methods' 'HEAD, GET, PUT, OPTIONS';
-        add_header 'Access-Control-Allow-Headers' 'Authorization';
-        add_header 'Access-Control-Allow-Credentials' 'true';
-        client_max_body_size 105M; # Choose a value a bit higher than the max upload configured in XMPP server
-    }
-
-    include /etc/nginx/conf.d/security.conf.inc;
-
-    ssl_certificate /etc/yunohost/certs/domain.com/crt.pem;
-    ssl_certificate_key /etc/yunohost/certs/domain.com/key.pem;
-
-    
-    more_set_headers "Strict-Transport-Security : max-age=63072000; includeSubDomains; preload";
-    
-    
-    # OCSP settings
-    ssl_stapling on;
-    ssl_stapling_verify on;
-    ssl_trusted_certificate /etc/yunohost/certs/domain.com/crt.pem;
-    resolver 127.0.0.1 127.0.1.1 valid=300s;
-    resolver_timeout 5s;
-    
-
-    access_log /var/log/nginx/xmpp-upload.domain.com-access.log;
-    error_log /var/log/nginx/xmpp-upload.domain.com-error.log;
-}

that’s the diff

[Aleks]

Yes, sounds like it’s been a while since your conf was not updated, so just apply the changes using --force …

[mouse]

done. and error gone.

thanks!

and i guess if that check passes certs will auto update

[mouse]

i see what i have of manual config in this file is a

add_header Access-Control-Allow-Origin

for a domain that i need in there.

can that go in the conf.d/ subdirectory conf file?

[Aleks]

I don’t know, that depends what you are trying to achieve exactly, that syntax doesnt even look like a valid one … but yes, files in domain.tld.d/*.conf are included in the domain conf … or you can also imagine tweaking security.conf.inc too

[mouse]

i just omitted the “domain” from the syntax.

but i’ll keep it in mind, that i should put all mods in conf.d/.

thanks for the assistance! i’m glad i improved my nginx mess.