My server is “somewhat” comparable (4 core, 8 GB RAM, 20 domains over 4 TLDs)
Below is a totally non-scientific and messy log of the same action on my server. In short:
- Just clicking through the domain list
- Adding a domain
- Removing a domain
- Posting logs to yunopaste
- Some screenshots of
htop while performing these actions
I performed all actions through the web admin. The CLI may be more efficient (on a side note: for some accounts I can view, but not create mail aliasses through the webadmin, while it works through the CLI. That may be performance related or a fixed array that is too small; I did not further look into that)
Just opening a domain is not blazing fast. The first time opening the settings for any domain takes a while, after that the first time opening a domain related to a specific TLD takes longer than subsequent other domains for that same TLD. Subsequent openings of the same domain configuration are faster.
It leads me to think that some checks (and after that, cache hits) are responsible for the delay (and subsequent better performance):
- 45 seconds on first opening of a TLD A
- instant return to domain list
- 10 seconds delay on second opening of A
- instant return to domain list
- near instant showing of third opening of A
- instant return to domain list
- 15 seconds delay on opening a subdomain of TLD B
- instant return to domain list
- 15 seconds delay on opening a subdomain of TLD C
- instant return to domain list
- 5 seconds delay on opening of TLD C
- instant return to domain list
- 5 seconds delay on opening of TLD B
While clicking through this list, yunohost-api is relatively heavily used according to htop.
Then I clicked somewhere in the domain list and got the rainbow cat trying to push DNS records to my DNS provider! I cancelled it, because I’m not sure what automatic configuration may break on dns.he.net, and afterwards was not able to find where I clicked to have this popup return.
After that I tried adding a domain
- 5 seconds to start the “Add domain” wizard
- 150 seconds to create the domain, run diagnosis (I suppose) and failing to register letsencrypt
While running, this line mail_in mail_out pops up on top of htop quite often:
Something Nextcloud related also popped up a few times, which it did not beforehand and not afterwards, but that can be coincidence.
Removing the domain took 120 seconds, so about as long as adding it minus timeouts for Letsencrypt registration. During removal the Nextcloud line popped up again, so it seems to get hit one way or another during this proces:
Besiedes that, both MariaDB and synapse are quite a bit busier than when not making any domain changes.
Both these screenshots show a relatively high CPU load; before and after the domain actions the load hovers under 5%
Opening logs:
- 5-10 seconds to open the log list
- 30 seconds 80% CPU load by
yunohost-api to open the first log (accidentally updating DNS settings)
- instant posting to yunopaste
- 40 seconds to open the second log (adding a subdomain)
- seemingly hangs on posting to yunopaste, then 3 new tabs with unique URLs (so the button was still active while processing and not giving feedback)
- This log entry mentions 2 sub operations:
- 40 seconds to open “5_categories”
- 40-50 seconds to post the log to yunopaste
- ? seconds to open oeauaoeu
- 40 seconds to post to yunopaste
- contains 1 suboperation
- ?? seconds to open
- 10 seconds to post to yunopaste
- 30 seconds for twice browser ‘page back’ to return to the main ‘add subdomain’ log entry
- 5 seconds to return to the log list
- 30 seconds to open the remove domain log entry
- 40 seconds to post to yunopaste
- 1 sub operation "Regenerate system configurations “4_categories”
- 20 seconds to post to yunopaste
Second time opening log entry for adding subdomain:
opening the log entry for the sub-operation of the sub-operation:
‘page back’ - ‘page back’ to add subdomain log entry:
opening log entry for deleting the sub domain:
Without knowledge of the matter, I speculate:
- Diagnosis may be involved
- Checking / regenerating of (possibly related) configurations is involved
- Some processes form bottlenecks for other processes
- yunohost-api is Python, as is Synapse. One way or another, on my system, Synapse has higher CPU load while the api is active
- nginx worker processes claim higher CPU load while above processes are running (I failed to notice exactly which processes, the refresh rate on htop is set too high)
- perhaps because of the higher load, php-fpm gets pushed aside and reclaims something after that, causing higher CPU load for PHP processes such as Nextcloud
Sorry for the long and messy post!
