Same behavior on NC15 with php7.3-fpm! So most likely not Nextcloud issue but either php7.3-fpm or the shit ton of other changes that adds the delay…
\o/ found a fix:
Patching in
nextcloud/lib/private/Security/Hasher.phpthe hashing load parameters has tremendously reduced CPU load on a production environment.
/** private $options = []; */
private $options = [
'memory_cost' => 2048,
'time_cost' => 2,
'threads' => 1
];
They had the same issue when upgrading to php7.3-fpm…
Source:
3 Likes
So, if my understanding is correct, to solve that problem, we should adapt the value of hashingMemoryCost, hashingTimeCost and hashingThreads into config/config.php ?
I see on the link that the guy says
[…] Or even better,
threadsshould be something likeCPU cores available / 2andmemory costshould be a fraction (/32?) of max memory assigned to fpm workers.
Defaultthreads = 2does not work on small / cheap virtual machines which have assigned only one core.
That’s information we do have in the package, CPU core is given by nproc and max memory is max children by footprint.
Indeed. But that proposal does not seem to reflect any best practices, which I’m not sure even exist. I’m looking into that now as well and trying to do some tests with different settings.
One other thing to consider is to set this php7.3 in general and not just Nextcloud…
Une question, je me retrouve avec deux listes de dépôts pour les extras de php, mais qui pointent vers la même source, donc j’en commente forcement un pour débloquer apt qui n’aime pas les doublons. Dans /etc/apt/sources.list.d, j’ai ces 3 dépôts: extra_php_version.list onlyoffice.list sury.list yunohost.list.
les dépôts extra_php_version.list et sury.list sont en double et pointent vers:
deb https://packages.sury.org/php/ stretch main
Donc j’ai commenté celui de sury.list. Faut-il supprimer une des listes et laquelle ?
What do you mean ?
Remove at least sury.list.
And personnaly I would removed as well onlyoffice.list…
We can configure the hashing options in PHP or Nextcloud config. But since it’s such an edge case, I think Nextcloud is good enough.
The spec (https://github.com/P-H-C/phc-winner-argon2/blob/master/argon2-specs.pdf) is very unspecific. PHP developers increased the numbers from:
memory_cost = 1024 KiB
time_cost = 2
threads = 2
to
memory_cost = 65536 KiB
time_cost = 4
threads = 1
(see: PHP: rfc:argon2_password_hash, and PHP :: Sec Bug #78269 :: password_hash uses weak options for argon2)
There is no reasoning behind this other than some library mentioning these values, but this is way too heavy IMHO for Pi. The Argon2 spec mentions a 0.5 second time should be enough to get a decent security level for logins. They don’t have recommendations for calculating these values other than try to make them as high as possible.
I’m also not entirely sure which password is being hashed since we use SSO? Or what we are trying to secure here…
I did some testing on my Pi using the following script:
<?php echo 'Argon2i hash: ' . password_hash('nextcloud', PASSWORD_ARGON2I, [ 'memory_cost' => m, 'time_cost' => t, 'threads' => p ]); ?>Where I change m, t, p, and these are my results on the Pi 3:
Default (m=65536,t=4,p=1):
Argon2i hash: $argon2i$v=19$m=65536,t=4,p=1$V3JRdzRJMzJwQm1kUlBWdg$FeQdJQbZemvRexfgP44EoIbn4kktMq2sSc03ZWmmxwA
real 0m5.286s
user 0m5.158s
sys 0m0.100s
Two threads:
Argon2i hash: $argon2i$v=19$m=65536,t=4,p=2$VS96UDdKeGQxVHdxVDBmYw$R1UDEuZmh5Uaulobkqk97IPiJE68MYxkUK/jO6430UM
real 0m2.895s
user 0m5.117s
sys 0m0.210s
Two threads, time_cost=2:
Argon2i hash: $argon2i$v=19$m=65536,t=2,p=2$Rzd6SVV1QldqLi8xQy9Wag$8lmHabLi3L0fuJPN1LSffBfTY3qda8gqfNmuR+Ojgyw
real 0m1.624s
user 0m2.669s
sys 0m0.169s
Two threads, time_cost=2, 32MB:
Argon2i hash: $argon2i$v=19$m=32768,t=2,p=2$WGJ4S01BOW1sQWs5N0N1Vg$ca9z9orBAeuGKVradyeaLXYGNeMDmHrou/w+Km1PfNs
real 0m1.015s
user 0m1.531s
sys 0m0.091s
Two threads, time_cost=1, 32MB:
Argon2i hash: $argon2i$v=19$m=32768,t=1,p=2$U20vU2tDWHZZa3lUeFNNWA$ZUhGptup2hDa5oIFmMjUq+Trh8a32m2xnP29jeFLfxI
real 0m0.641s
user 0m0.818s
sys 0m0.108s
Two threads, time_cost=1, 16MB:
Argon2i hash: $argon2i$v=19$m=16384,t=1,p=2$c1lOOWFKQmM2LmxUSS80cQ$vv/I8aMX6SK6tsGC0G5qpuDKLuYplHJhzAnFnSHHRWw
real 0m0.483s
user 0m0.500s
sys 0m0.112s
So for a Pi 3, setting memory_cost to 16384, time_cost to 1, and threads to 2, I get below that suggested 0.5 seconds. These are the additional lines that I implemented in my nextcloud config.php:
‘hashingMemoryCost’ => 16384,
‘hashingTimeCost’ => 1,
‘hashingThreads’ => 2,
The time to login in with a fresh Incognito session to my production environment went from 45 seconds down to <20 seconds…
Not sure what you should do to fix this in the app script though… You could test for a small system and then use these values, otherwise go to the default. I noticed you have a config panel for sizing, put it in there… I don’t know…
In any case, thank you so much for your support! Highly appreciated!
If I follow those “recommendation” I would have for a Pi 3
hashingMemoryCost = ~500 / 32 = 15625
hashingTimeCost = ?
hashingThreads = 4 / 2 = 2
I do not find any useful info about how to define time cost, maybe a time cost equal to thread could be a good base to define it.
from the spec:
Run the scheme of type y, memory m and h lanes and threads, using different number of passes t. Figure out the maximum t such that the running time does not exceed x. If it exceeds x even for t = 1, reduce m accordingly.
Y = either i, d or id (not our problem)
M = memory_cost
h = threads
t = time_cost
x = amount of time you would want to allow a hash to take (spec suggests 0.5 seconds for authentications)
So if you want to fully follow the spec do a test run with time_cost 1 and increase that until time_taken is >0.5 seconds.
Differently put:
While time_taken < 0.5s do time_cost++ && time_taken = time(doHash($options));
For me would that be:
hashingMemoryCost = 512*1024/32 = 16384
hashingThreads = 4 / 2 = 2
run with hashingTimeCost = 1 takes more than 0.5 sec.
And that’s exactly what I have configured now
To run the test until it reach the expected time sounds like a real brute force method…
I would rather go for a standard value.
What an idea anyway to have such a static config in an app…
I would go for 1 for time_cost. And the best course of action for me would be to open an issue both in the package and nextcloud repo. That’s not our call to apply that patch considering that’s clearly an upstream issue.
Have you tried to turn hashing_default_password to true in the config ?
Looks like you can just bypass that argon2 thing and get rid of it.
Agreed and the blunt security bug fix (https://bugs.php.net/bug.php?id=78269) in php is not helping. Also still no clue what or why we even are hashing here…
Even if the password is in ldap, nextcloud is probably hashing the password you give it to compare it.
Anyway, here https://github.com/nextcloud/server/pull/19203, they’re discussing about that very problem we have here. And to use that option is proposed as a solution to go back to what it was before, or so I guess.
Could be interesting to try, it would solve the entire problem for good.
And it does solve the problem too. Almost exact same speed as the other fix. Much better solution!
Up for another fun rabbit hole?
Try this:
Login into Yunohost, open Nextcloud, go to portal, and sign out from Yunohost. Login as different user, open Nextcloud… Whose account do you see?
Unfortunately a well known issue
I can’t find it, but I believe there’s some work going on about it in the core.
Could you try that branch, https://github.com/YunoHost-Apps/nextcloud_ynh/pull/294, and confirm that it does work for you ?
Did an upgrade from 18.0.2 (using: yunohost app upgrade nextcloud -f /tmp/argon2.zip) and it installed 18.0.3~ynh1, config.php has the ‘hashing_default_password’ => true and the performance is super!
Thank you so much for helping out!
2 Likes
Wonderful 