(Project) ILIAS Resource Storage Service

This is a project page that bundles several feature wiki pages which belong to a larger development activity for the File component in ILIAS.

1 Aim of Project

«All user-uploaded files shall be stored centralized in one Service»

1.1 Conceptual Summary

1.2 The ILIAS Resource Storage Service (IRSS)

The Resource Storage Service was introduced in a first version in ILIAS 6 where it was used to store icons for MainMenu entries. This allowed first experiences to be gathered and the service to be further expanded.
The principle of the service is simple. A file uploaded by a user is forwarded by a component (e.g. forum) to the storage service. In return, the components receive an identification (unique ID of the resource as a string). This ID is stored by the component. If the file is then needed again, the component with the ID can ask the Storage Service for a consumer (e.g. DownloadConsumer, InlineConsumer, ...) and execute it. In this way, the uploaded image is displayed, for example.

1.3 Migrations

Files were historically stored by the components in a directory in the iliasdata or data directory. If a component wants to use the Storage Service to manage files, all existing files of this component have to be moved to the Storage Service once.
This task can take a long time depending on the components and the size of the installation. To make this easy ILIAS 7 offers the possibility to perform migrations in the setup. A migration can be executed in many small parts, for example file by file. Such an implementation has already been done for ILIAS 7 for the file object. But since many other components use the FileObject internally as well this has been rewritten so that the old methods still work. In a further step all these components have to be rewritten so that they directly use the storage service and no longer the file object. In a further step, all other places that work with uploaded files should also be reprogrammed.

1.4 Features in Future

The central administration of all files will enable new features in the future.
For example, it will be possible to reintroduce a quota per user, which will then work globally for all uploaded files.
ILIAS will be able to perform regular actions on all files, e.g. checking for viruses or checking if files are orphaned and delete them automatically.

2 Involved Maintainers and Stakeholders

3 Timeline

Release

Objectives and developments

With ILIAS 6 the following topics were tackled and implemented:

• Implementation of the first Version of the IRSS which was used to store Icons for MainMenu-Items

Status

With ILIAS 7 the following topics were tackled and implemented:

• Migration from the initial Storage-Backend FSV1 to a new FSV2
• The Resource Stoage Service has been implemented and is available via DIC The service is therefore usable for all components and offers all necessary functions to store and manage files. See Introduction of Resource Storage Service
• Migrations are possible: The CLI setup of ILIAS 7 has been extended so that large migrations can be initiated and tracked after an update. See Setup - Add migrate Command
• The first migration was the migration of the file object. Due to the large number of file objects in ILIAS the file object was converted to work with already migrated as well as with not yet migrated files. Thus an installation can be put back into operation immediately after the update. The migration of the file objects can then be done during operation. See IRSS: Migrate File-Object-Files to IRSS

With ILIAS 8 the following topics were tackled and implemented:

• IRSS: Migrate Bibliografic-Files to IRSS

Due to the major refactoring work in the ILIAS 8 / PHP 8 project, there were unfortunately too few resources available to achieve further goals in the IRSS project.

With ILIAS 9 we accomplished to following steps:

• The following Components or Use-Case have been migrated to th IRSS with ILIAS 9
  • Exercise: Instruction-Files IRSS: Migrate Exercise to IRSS (Instruction Files)
  • Exercise: Sample Solutions
  • Exercise: Tutor Feedbacks
  • Forum: Posting Files
  • TestQuestionPool: Upload-Question User-uploaded Files
  • TileImages
  • User-Avatar: IRSS: Migrate Profile-Files to IRSS
• The following general Improvements have been introduced to the IRSS
  • IRSS: Flavours
  • IRSS: Container-Storage
  • IRSS: Resource-Collection User-Interface
  • IRSS: "Draft"-Revisions
  • IRSS: Resource Overview

• The migration of HTML learning modules and SCORM modules to container resources
  • IRSS: Migrate HTML-Learning-Modules to ContainerResources
• Migration of export files to resource collections
• As many components as possible should be converted to the Resource Storage Service. The main focus is on all those components that have been using the file object internally so far. We will gradually ask the respective maintainers whether we or they would like to carry out the conversion themselves. See the List in IRSS: General Overview of Components to Migrate (ongoing)
• Not directly related, but to facilitate the use of the service, additional UI file upload inputs will be written. For example a photo upload with preview function, a multi-file upload and a selection field of already uploaded resources
• Additional Features for Container-Resources:
  
  • IRSS: Container-Resource User-interface

• Virus-Scanning of Resources: 
  • IRSS: Virus-Scanner for Resources as Cron-Job
• Delete Resources in Resource-Overview:
  • IRSS: Delete Resources in Resource-Overview

4 Related Feature Requests and Status

5 Further Results

6 Additional Information

…

7 General Discussion

Please discuss specific questions of feature requests on the related feature wiki pages. This discussion section is only for a general discussion of the project and its realisation.

Schmid, Fabian [fschmid] 2019-03-01: A basic version of a File-Storage-Service should be implemented for the feature Customisable Main Bar - Icon Upload. The service won't be rolled out to the whole ILIAS base with 6.0. 

JourFixe, ILIAS [jourfixe], 11 MAY 2020 : We highly appreciate the suggested project. The project manager will provide a template for such project pages, soon. And he will help to adapt the content to this structure. All related feature requests and pull requests should be listed here but discussed separately. (Done in the meantime.)

JourFixe, ILIAS [jourfixe], 23 NOV 2023: Fabian presented the project at the Big Projects Jour Fixe. No additional comments.

11 DEC 2023: The product manager sees this as a very important project for ILIAS, and not just for improving the security of the software. It would be very pleasing if the project that started with ILIAS 6 could be largely completed with ILIAS 10. I would therefore ask all developers involved to help with the planned conversion work.