Have nginx not respond when accessed through http(s)://ip?

My YunoHost server

Hardware: A VPS
YunoHost version: 11.1.19
I have access to my server : Yes, ssh and Web.
Are you in a special context or did you perform some particular tweaking on your YunoHost instance ? : no

Description of my issue

Not really an issue, but my server will serve Yunohost when I access it from https://admin.mysite.com as well as https://myip. I would like nginx not to serve anything when accessed through http(s) and its IP. I can edit nginx configuration files but I am not familiar with Yunohost structure.

I can provide an answer to my problem. Edit /etc/nginx/conf.d/yunohost_admin.conf as follows:

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name _;
    return 444;
#    include /etc/nginx/conf.d/default.d/*.conf;

server {
    listen 443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;

    include /etc/nginx/conf.d/security.conf.inc;

    ssl_certificate /etc/yunohost/certs/yunohost.org/crt.pem;
    ssl_certificate_key /etc/yunohost/certs/yunohost.org/key.pem;

    more_set_headers "Strict-Transport-Security : max-age=63072000; includeSubDomains; preload";
    more_set_headers "Referrer-Policy : 'same-origin'";
    server_name _;
    return 444;
#    location /yunohost {
#        # Redirect most of 404 to maindomain.tld/yunohost/sso
#        access_by_lua_file /usr/share/ssowat/access.lua;
#    }

#    include /etc/nginx/conf.d/yunohost_admin.conf.inc;
#    include /etc/nginx/conf.d/yunohost_api.conf.inc;
#    include /etc/nginx/conf.d/default.d/*.conf;