Cryptpad being deleted by update. How can I recover?

[chmeyer]

What type of hardware are you using: VPS bought online
What YunoHost version are you running: 12.0.9.2
What app is this about: Cryptpad

Describe your issue

I had a working YNH Cryptpad, but the last Update removed it.

1. I updated Cryptpad to 2024.12.0~ynh1 and ran in the sandbox-cryptpad → sandbox.cryptpad problem.
2. I updated my DNS and regenerated the letsencrypt certificate [1]
3. Then I updated YNH from 12.0.9.1 to 12.0.9.2
4. Then I updated my DNS-Entries (and removed the now unused sandbox-cryptpad entry)
5. I Updated Cryptpad again (from 2024.12.0~ynh1 to 2024.12.0~ynh2)
   5.1. Backup went fine [2]
   5.2. Cryptpad Update failed [3], saying: ERROR Domain ‘sandbox.cryptpad.maindomain.tld’ unknown
   (But there is a CNAME entry in DNS and host shows it correctly)
   5.3. CryptPad App has been removed automatically successfully [4]
   5.4. automatic Recovery of CryptPad from archive failed [5]

Now I don’t have cryptpad installed any more, and I don’t want to make mistakes, because there has been important documents in cryptpad.

How can I restore Cryptpad and my documents?

Thanks a lot
Christian

Share relevant logs or error messages

[1] https://paste.yunohost.org/raw/igifogudif
[2] https://paste.yunohost.org/raw/fufinunate
[3] https://paste.yunohost.org/raw/cejilowovu
[4] https://paste.yunohost.org/raw/ezopawiced
[5] https://paste.yunohost.org/raw/egovikuwuv

[emo]

look here, Sandbox domain Certproblem
main problem is, how to do the sandbox c-name.

[chmeyer]

No, I think, it’s different.
I did exactly, what the updatescript told me to do. I created a CNAME entry for sandbox.cryptpad (with dot) and then force regenerated the certificate for it.
So this is, what @rodinux told you:

Sandbox domain Certproblem

Sorry, with the next release, the old domain sandbox-cryptpad.domain.tld is no more used. But the correct « domain » for sandbox is sandbox.cryptpad.domain.tld. There is no domain sandbox.cryptpad.domain.tld created, but a a CNAME must be added. And the certificate must be renew, but perhaps it needs wait a bit to have dns cache and propagation renewed before. I have get similar error trying renew the certificate, but after a while, I could renew it… Also you can remove the old the domain sand…

Sandbox domain Certproblem

Ok I saw your problem… I think you have not edit correctly the CNAME… I see your domain name, so I can test the CNAME here https://mxtoolbox.com/ and it did not found it. In this link, I could verify CNAME for example for my instance cryptpad.linux07.fr. I Choose CNAME Lookup in the orange button and put sandbox.cryptpad.linux07.fr and I see the entry. [Capture d’écran du 2025-01-06 22-34-48] Can you verify on your registar zone DNS if the entry is correct ? You will see it with yunohost …

Yunohost just thinks, that “ERROR Domain ‘sandbox.cryptpad.maindomain.tld’ unknown”.

But it already IS there (okay, it has been there).
Certificate regeneration said before:

2025-01-06 14:29:43,625: DEBUG - + sandboxdomain=sandbox.cryptpad.maindomain.tld
...
2025-01-06 14:29:44,238: INFO - Found domains: cryptpad.maindomain.tld, sandbox.cryptpad.maindomain.tld
...
2025-01-06 14:29:51,224: INFO - Verifying sandbox.cryptpad.maindomain.tld...
2025-01-06 14:30:18,460: INFO - sandbox.cryptpad.maindomain.tld verified!
...
2025-01-06 14:32:34,021: DEBUG - processing pending conf '/var/cache/yunohost/regenconf/pending/nginx/etc/nginx/conf.d/sandbox.cryptpad.maindomain.tld.conf' to system conf '/etc/nginx/conf.d/sandbox.cryptpad.maindomain.tld.conf'
2025-01-06 14:32:34,032: DEBUG - > system conf is not managed yet
2025-01-06 14:32:34,032: INFO - The configuration file '/etc/nginx/conf.d/sandbox.cryptpad.maindomain.tld.conf' is expected to be deleted by regen-conf (category nginx) but was kept back.
...
SUCCESS - Let's Encrypt certificate renewed for the domain 'cryptpad.maindomain.tld'

so sandbox.cryptpad.maindomain.tld already IS verified!.

And when I do an:

$ host sandbox.cryptpad.maindomain.tld
sandbox.cryptpad.maindomain.tld is an alias for cryptpad.maindomain.tld.
cryptpad.maindomain.tld has address 213.xx.xx.xx
cryptpad.maindomain.tld has IPv6 address 2a03:xxxx:xx:xxx:xxxx:xx:xxxx:xxxx
cryptpad.maindomain.tld mail is handled by 10 cryptpad.maindomain.tld.

$ host cryptpad.maindomain.tld
cryptpad.maindomain.tld has address 213.xx.xx.xx
cryptpad.maindomain.tld has IPv6 address 2a03:xxxx:xx:xxx:xxxx:xx:xxxx:xxxx
cryptpad.maindomain.tld mail is handled by 10 cryptpad.maindomain.tld.

So before I did this update things should be all right. Subdomain (with dot) is correctly, and in CryptPad update log there is no word about the dash-subdomain sandbox-cryptpad, so it really seems that it isn’t needed anymore.

I just wonder, why CryptPad update first removes the sandbox.cryptpad by itself and then complains about it:

2025-01-06 20:46:12,854: DEBUG - + grep -q sandbox.cryptpad.maindomain.tld
2025-01-06 20:46:12,854: DEBUG - + yunohost domain list --output-as plain
2025-01-06 20:46:23,071: DEBUG - + yunohost domain remove sandbox.cryptpad.maindomain.tld
2025-01-06 20:46:28,046: WARNING - ERROR Domain 'sandbox.cryptpad.maindomain.tld' unknown

Okay, today I see, @rodinux also told you to change CNAME to A / AAAA, so this should be the first step for me to get a proper CryptPad again…

But this only is half of the story.
After that recovery from backup failed, because:

Domain 'sandbox-cryptpad.maindomain.tld' unknown\n"

so, here it fails because the old sandbox-cryptpad isn’t there any more.

Of course I could set an A / AAAA entry for sandbox-cryptpad.maindomain.tld again and reinstall cryptpad from scratch, but here comes concern number three: I don’t want to loose my backup, so I don’t want to do too many experimental things.

And I don’t see how a
cp -a /etc/nginx/conf.d/pad.mydomain.de.d/cryptpad.conf /etc/nginx/conf.d/sandbox-pad.mydomain.de.d/
could help with the new sandbox.cryptpad subdomain.

[rodinux]

@chmeyer

2024.12.0~ynh1 to 2024.12.0~ynh2

I think the domain sandbox-cryptpad.maindomain.tld should be removed !
I think O also understand the problem with you update… The problem is more because of some scripts… I am correcting them…

So the error was because in the remove, the script remove I see an error in the line
sandboxdomain=sandbox-cryptpad.maindomain.tld,

this is a key in the settings for the app and it should be sandboxdomain=sandbox.cryptpad.maindomain.tld !

So tell me first if you still have these files ? /etc/nginx/conf.d/sandbox.pad.domain.tld.conf and /etc/nginx/conf.d/cryptpad-shared.conf.inc

[rodinux]

Next I see other problems about your DNS… But first tell me that… ( I saw your sandbox.pad.domain.tld is not a CNAME but a IN A )

In your case, I think we need change a line in your backup, change the key to have the correct ine sandboxdomain=sandbox.cryptpad.maindomain.tld in the file /apps/cryptpad/settings/settings.yml, I tell you after how…

[chmeyer]

No, there is no
/etc/nginx/conf.d/sandbox.pad.domain.tld.conf and /etc/nginx/conf.d/cryptpad-shared.conf.inc

I have /etc/nginx/conf.d/cryptpad.domain.tld.conf and an empty /etc/nginx/conf.d/cryptpad.domain.tld.d/.
In this directory there is nothing with sandbox in its name and no inc-file with some name about cryptpad.

But there are files in the archive:

/home/yunohost.backup/archives/cryptpad-pre-upgrade2.info.json
/home/yunohost.backup/archives/cryptpad-pre-upgrade2.tar

For DNS I guess that I have to remove the MX entry.

Thank you for your help.

[rodinux]

@chmeyer

complains about it:

2025-01-06 20:46:12,854: DEBUG - + grep -q sandbox.cryptpad.maindomain.tld
2025-01-06 20:46:12,854: DEBUG - + yunohost domain list --output-as plain
2025-01-06 20:46:23,071: DEBUG - + yunohost domain remove sandbox.cryptpad.maindomain.tld
2025-01-06 20:46:28,046: WARNING - ERROR Domain 'sandbox.cryptpad.maindomain.tld' unknown

Here is a mistake !
from this script: cryptpad_ynh/scripts/upgrade at 3e349feb475dc8eb25c5c27d053a97ef9caf515d · YunoHost-Apps/cryptpad_ynh · GitHub
It should be corrected here
cryptpad_ynh/scripts/upgrade at bdf099a261a900217bb064689fa23168016c8dbf · YunoHost-Apps/cryptpad_ynh · GitHub

@chmeyer

And I don’t see how a
cp -a /etc/nginx/conf.d/pad.mydomain.de.d/cryptpad.conf /etc/nginx/conf.d/sandbox-pad.mydomain.de.d/

No ! this was for the oldest version 5 only !

[chmeyer]

First:
I recently changed sandbox.cryptpad from CNAME to A and AAAA, but I think during the update it has been a CNAME.

Second:
I did a tar --extract -f cryptpad-pre-upgrade2.tar apps/cryptpad/settings/settings.yml and it already looks like: sandboxdomain: sandbox.cryptpad.mydomain.tld

The only place where sandbox-cryptpad appears in this file is checksum__etc_nginx_conf.d_sandbox-cryptpad.mydomain.tld.d_cryptpad.conf.

[rodinux]

@chmeyer

The only place where sandbox-cryptpad appears in this file is checksum__etc_nginx_conf.d_sandbox-cryptpad.mydomain.tld.d_cryptpad.conf.

well, this is normal, the old checksum from the oldest version… it can be removed. but is not the problem…

[rodinux]

@chmeyer

I recently changed sandbox.cryptpad from CNAME to A and AAAA, but I think during the update it has been a CNAME.

It must be a CNAME for cryptpad…

[rodinux]

I am confused, trying understand why the restore backup, in the log restore there is a log

2025-01-06 20:59:38,658: INFO - <strong>Could not complete the operation 'Create permission 'cryptpad''. Please provide the full log of this operation by <a href="#/tools/logs/20250106-195935-permission_create-cryptpad">clicking here</a> to get help</strong>

Can you fond this log ?

yunohost log list |grep cryptpad

Also you can perhaps try verify in the folder extracted if there is somewhere a line with sandbox-cryptpad.maindomain.tld

grep -ir sandbox-cryptpad.maindomain.tld /folder-extract/*

Ok, we can try this if you are OK…

• create the domain temporally sandbox-cryptpad.maindomain.tld
• restore the backup

If it works, wait the fix cryptpad~ynh3 to upgrade or upgrade from testing

[chmeyer]

Okay, so many good ideas.
I will try.

1.No, there is no log with this name.
It has been a long list, so I grep’ed a bit more

$ sudo yunohost log list | grep cryptpad | grep 20250106-19
    name: 20250106-194507-app_upgrade-cryptpad
    path: /var/log/yunohost/operations/20250106-194507-app_upgrade-cryptpad.yml
    name: 20250106-194628-app_remove-cryptpad
    path: /var/log/yunohost/operations/20250106-194628-app_remove-cryptpad.yml
    name: 20250106-195933-backup_restore_app-cryptpad
    path: /var/log/yunohost/operations/20250106-195933-backup_restore_app-cryptpad.yml

2.I did grep -ir sandbox-cryptpad in my untar’ed backup (took a really long time) and this is what I got:

$ sudo grep -ir sandbox-cryptpad
apps/cryptpad/settings/permissions.yml:  - sandbox-cryptpad.my-domain.tld/
apps/cryptpad/settings/settings.yml:checksum__etc_nginx_conf.d_sandbox-cryptpad.my-domain.tld.d_cryptpad.conf: 220ac35b07bd083d194400ee51748eec

so beside the checksum (you said this isn’t important, and anyway the file doesn’t exist in backup) it’s just this single file:

$ sudo head -3 apps/cryptpad/settings/permissions.yml
cryptpad.main:
  additional_urls:
  - sandbox-cryptpad.my-domain.tld/

3.Earlier (in your first answer above) you asked for the two files /etc/nginx/conf.d/sandbox.pad.domain.tld.conf and /etc/nginx/conf.d/cryptpad-shared.conf.inc, and I found them in the backup:

apps/cryptpad/backup/etc/nginx/conf.d/sandbox.cryptpad.my-domain.tld.conf
apps/cryptpad/backup/etc/nginx/conf.d/cryptpad-shared.conf.inc

Are they still usefull for you? I didn’t find any sandbox-cryptpad in it. Everything is sandbox.cryptpad.

4.Where should I create sandbox-cryptpad.maindomain.tld?
In my DNS nameserver or in Yunohost (under Domains - add domain)?
Or both?

Or wouldn’t it better to change the additional_url in permissions.yml to sandbox.cryptpad.my-domain.tld/ ?

[rodinux]

well… why this folder apps/cryptpad/settings/permissions.yml ?? I haven’t such folder… I think this one should be the problem… I am downloading a backup to see what I have… well I have also this !! why ??

@chmeyer

4.Where should I create sandbox-cryptpad.maindomain.tld?
In my DNS nameserver or in Yunohost (under Domains - add domain)?
Or both?

I will said on Yunohost

[chmeyer]

Okay, here I go:

• took a deep breath
• added subdomain sandbox-cryptpad to mymaindomain.tld in YNH
• installed Cryptpad to the cryptpad.mymaindomain.tld subdomain
• restore … ah, okay, that has not been neccessary: uninstall Cryptpad
• sandbox-cryptpad still is an subdomain, even when uninstall said something different.
• oh no, YNH says Error 400: “Domain ‘sandbox-cryptpad.mymaindomain.tld’ unknown”, but it still is listed in the domain list.
• don’t care, re-add sandbox-cryptpad as new subdomain to mymaindomain.tld with an letsencrypt certificate
• restore backup: cryptpad-pre-upgrade2 (2024.12.0~ynh1)
  • Found domains: sandbox-cryptpad.mymaindomain.tld
  • Verifying sandbox-cryptpad.mymaindomain.tld …
  • sandbox-cryptpad.mymaindomain.tld verified!
  • Signing certificate…
  • Certificate signed!
  • restauration successfull

When I open cryptpad.mydomain.tld it looks great.
But when I log in with my username all I see is an (big) error message:

Blocked page
An error occurred when connecting to sandbox.cryptpad.mydomain.tld.

This page has an valid letsencrypt certificate for sandbox.cryptpad.mydomain.tld and cryptpad.mydomain.tld, but it is from yesterday (when I ordered it manually).

When I only want to open a new document the same error page appears.

• Update to 2024.12.0~ynh3, but there are some warnings and errors:
  • WARNING ./15-nginx: line 39: ynh_render_template: command not found
  • ERROR Could not run script: /usr/share/yunohost/hooks/conf_regen/15-nginx
  • WARNING ./19-postfix: line 81: ynh_render_template: command not found
  • ERROR Could not run script: /usr/share/yunohost/hooks/conf_regen/19-postfix
  • WARNING ./43-dnsmasq: line 43: ynh_validate_ip4: command not found
  • WARNING ./43-dnsmasq: line 45: ynh_validate_ip6: command not found
  • WARNING ./43-dnsmasq: line 56: ynh_render_template: command not found
  • ERROR Could not run script: /usr/share/yunohost/hooks/conf_regen/43-dnsmasq
  • Since some changes was done on the sandbox domain, you will need to regenerate the certificate for cryptpad.mydomain.tld and update your DNS config accordingly.
• oh, and I saw you introduced Please, be sure the CNAME is correctly added on your registar and wait for the DNS propagation with commit 6e2ca41ff97faf125c65e883f1d9a44c525df534
  Didn’t you prefer A and AAAA records over CNAME?

So in the end with ynh~3 I’m still at this point with:

Blocked page
An error occurred when connecting to sandbox.cryptpad.mydomain.tld.

[rodinux]

@chmeyer

apps/cryptpad/settings/permissions.yml

Ok your on the good way, now the problem is withe the cname