It’s related to the new configuration regeneration system - see the v2.4 release note. It allows to manage the configuration files’ update of each service - including LDAP and slapd.conf. When you modify by hand one of those configuration files, it will not be automatically regenerated regarding the YunoHost provided one. Instead, your local modifications will be kept. Conversely, if no modification has been made to this configuration file, it will be updated with the next YunoHost upgrade.
So, if you change the LDAP configuration file to set a new password, then regenerate the configuration database - regarding /etc/ldap/slapd.conf content and stored in /etc/ldap/slapd.d-, then put back the original slapd.conf file, it will be as if no local modification has been made to this file. Thus, this file will be updated with the next YunoHost upgrade - without the admin password. But, the configuration database will also be regenerated regarding /etc/ldap/slapd.conf content at the end of the upgrade, and the admin password will be lost again…