ElasticSearch 7, ElasticSearch 8, OpenSearch packaged!

Overview

ElasticSearch and OpenSearch are both distributed, RESTful search and analytics engine. You can use them to store, search, and manage data for logs, metrics, search backend, application monitoring, Endpoint security.

Shipped versions

  • ElasticSearch7: 7.17.8~ynh6
  • ElasticSearch8: 8.6.2~ynh1
  • OpenSearch: 2.5.0~ynh2

Use cases

As an end user, you would probably not worry about these packages, unless you worry about the licenses.

As a packager, you would probably like to use one of them if the app requires using an ElasticSearch or an OpenSearch indexer. You may call the ynh_install_app helper.

What are the differences between them

  • OpenSearch is a fork of ElasticSearch7 and share a common API with it;
  • ElasticSearch7 and ElasticSearch8 are SSPL licensed and thus are not totally free/open-source;
  • OpenSearch is a fork initiated by Amazon…

You may also take a look of why ElasticSearch is not totally free/open-source anymore.

Further considerations

I did not package ElasticSearch 7.10, despite being the last ElasticSearch free/open-source version, as the last patch was released 2 years ago, and I am not confident about the security of it at all.

Repoositories

3 Likes

Hi,
Thank you very much for these packages ! Do they can be used with nextcloud out of the box ?
Something like installing elasticsearch yunohost package, then installing nextcloud’s elastic search and full text search apps, and linking both ? Does anybody have already tested this ?
Edit : any idea of the charge on the server to index files, and to make searches, in relation to the number of files/users ?

I’ve tried to install elastic search, it seems to be working, but I have some doubts. What I did :

  • I’ve installed elasticsearch8 app on yunohost
  • I’ve installed 3 nextcloud apps : full text search, full text search - Files, full text search - Elasticsearch Platform
  • I’ve configured it like described in this tutorial : Nextcloud et Elasticsearch sur VirtualBox – Partie 3 – Elasticsearch
  • I’ve launched the index initialization. It seems to work, but it takes really long time (already 3-4 days, and it’s far from finished). I get this :

My questions are :

  • I’ve already excluded some folders by adding a .noindex file -» that’s working. But is there a way to exclude some kinds of files to index ? Like pictures for example ?
  • I get a huge number of errors, all the same : Elastic\Transport\Exception\NoNodeAvailableException. Does that mean that the indexing doesn’t work ? How to solve that ?
  • when I try to do a full text search in nextcloud, I get no results in files, even if I stop the indexing.

Thanks !

@tomdereub
You have to wait for indexing to be completed to start research in NextCloud.

Can you display the content of your file /etc/systemd/system/nextcloud-fulltext-elasticsearch-worker.service

Modify these two lines

ExecStart=/usr/bin/php /var/www/nextcloud/occ fulltextsearch:live -q
ExecStop=/usr/bin/php /var/www/nextcloud/occ fulltextsearch:stop

by this

ExecStart=/usr/bin/php8.1 /var/www/nextcloud/occ fulltextsearch:live -q
ExecStop=/usr/bin/php8.1 /var/www/nextcloud/occ fulltextsearch:stop
1 Like

The content of my file /etc/systemd/system/nextcloud-fulltext-elasticsearch-worker.service is already :

[Unit]
Description=Elasticsearch Worker for Nextcloud Fulltext Search
After=network.target

[Service]
User=nextcloud
Group=nextcloud
WorkingDirectory=/var/www/nextcloud
ExecStart=/usr/bin/php8.1 --define apc.enable_cli=1 /var/www/nextcloud/occ fulltextsearch:live -q
ExecStop=/usr/bin/php8.1 --define apc.enable_cli=1 /var/www/nextcloud/occ fulltextsearch:stop
Nice=19
Restart=always

[Install]
WantedBy=multi-user.target

And the command I launch to initialize index is : sudo -u nextcloud php8.1 --define apc.enable_cli=1 occ fulltextsearch:index

I don’t know if --define apc.enable_cli=1 affects indexing.

I don’t know, but it doesn’t work without on my server :

root@lowcal:/home/lamonerie# sudo -u nextcloud php8.1 /var/www/nextcloud/occ fulltextsearch:index
An unhandled exception has been thrown:
OCP\HintException: [0]: Memcache \OC\Memcache\APCu not available for local cache (Is the matching PHP module installed and enabled?)

For now, I’m facing 2 problems :

  • I already let indexing for a few days, and even if I stop the first indexing, it doesn’t show any files results in nextcloud. It has finished to index one user, I tried to search logged with this user, I don’t have any result. How to debug this ?
  • I have some external storage that are shared between many users, it the index command will rescan all files with each user… How to avoid this and make a common index for shared folders ?