Replace Current Search with Dependency to Query directly from PHP

1 Initial Problem

ILIAS employs a 'naked' Lucene. There basically is no alternative to Lucene and thus it is set. But running a naked Lucene requires more work than a running a packaged search server like

• Open Search
• Apache Solr
• Elastic Search

We want to get rid of the naked Lucene and replace it by one of the above packages. 
Elastic Search has made a questionable licensing move in 2021. Since then the fork Open Search was created mainly to support Amazon OpenSearch Service. 
Due to the murky licensing move and the closeness of the product we disregard Elastic Search. 

This article is a vehicle to elicit a decision from the Jour Fixe wther to pick Open Search or Solr. 

Open Search has grown out of monitoring roots while Solr document management roots. Both packages are very popular and have huge installed bases. 

2 Conceptual Summary

We need to decide for a dependency on a search package. 

• We need content analysis: detecting and extracting metadata and text from files types (such as PPT, XLS, and PDF) and indexing. The content analysis package is Tika.
  • Tika is an integral part of Solr, it is one server in total. 
  • OpenSearch would require a separate Tika server to be run. This are then two servers in total. 
• We need an interface to the search package from PHP:
  • OpenSearch has a REST API. In addition there is a PHP integration, which does all the lowlevel work. We would propose to add opensearch-php into composer as a dependency.
  • Solr provides to interface. One is the old interface based on query parameters and a newer interface using JSON. Besides directly interfacing with with Solr, there exists a PHP-integration Solarium, which facilitates a lot of the lowlevel handling. We would propose to add Solarium into composer as a dependency. 
• One objective would be to no longer need ILIAS-own Java code for the search. 

3 User Interface Modifications

3.1 List of Affected Views

• none 

3.2 User Interface Details

none

3.3 New User Interface Concepts

none

3.4 Accessibility Implications

not applicable

4 Technical Information

{ The maintainer has to provide necessary technical information, e.g. dependencies on other ILIAS components, necessary modifications in general services/architecture, potential security or performance issues. }

5 Privacy

All data stored and processed in the index is already stored in the database or filesystem of the ILIAS installation. No additional data will be stored. For every bit of data, that gets stored inside the index, it needs to be ensured, that once the source of the this data gets removed, the data also gets removed from the search index.

6 Security

The current ilServer interface will get replaced by the new Search Interface of either Solr or Opensearch. The interface should not be exposed to the public, which correspondes the current state of the ilServer interface. Both Search Interfaces provide additional information how to setup and secure the corresponding systems.

7 Contact

• Author of the Request: Tödt, Alexandra [atoedt]
• Maintainer: Meyer, Stefan [smeyer]
• Implementation of the feature is done by: {The maintainer must add the name of the implementing developer.}

8 Funding

If you are interest in funding this feature, please add your name and institution to this list.

• …

9 Discussion

JourFixe, ILIAS [jourfixe], 02 MAY 2022 : Stefan Meyer gave a short overview of the current status and concept for an improved search for ILIAS 9. Final FR will be presented in one of the next JF.

10 Implementation

{ The maintainer has to give a description of the final implementation and add screenshots if possible. }

Test Cases

Test cases completed at {date} by {user}

• {Test case number linked to Testrail} : {test case title}

Approval

Approved at {date} by {user}.