Show Advanced KnowledgeHide Advanced Knowledge

IRSS: Migrate File-Object-Files to IRSS

1 Initial Problem

The File-Objects in the repository acts as a Module but also provides some service like functions to other component. Within the framework of this FR, the file object is cleaned up internally so that it temporarily makes the service-like functions available to the components as a wrapper, but the file objects itself only uses the file storage service. the settings are also streamlined, see Splitting up File Object into Object and Service

2 Conceptual Summary

This is a spin-off feature request from:

• IRSS: General Overview of Components to Migrate (ongoing)
• Project: ILIAS Resource Storage Service (IRSS)

as the first component, the file object in the magazine is converted to use the file storage service to store the files.
this is a mainly technical feature request with no visible change to the object.

3 User Interface Modifications

3.1 List of Affected Views

none

3.2 User Interface Details

none

3.3 New User Interface Concepts

none

4 Technical Information

The migration will use Setup - Add migrate Command. All technical details of the service and the migration possibilities will be discussed using a pull-request.

Questions/decisions for the JourFixe:
- Is a partial migration of the data possible, i.e. with the help of the setup but additionally ad-hoc migration of single files during operation?
- How to deal with files/file versions that cannot be migrated (anymore)? Write a log, list it in the administration? Error messages to users? Delete objects?

5 Privacy Information

The File-Storage-Service stores the user_id of the user, which uploads a file.

6 Security Implications

• There are no new enpoints or libraries introduced.
• Files are stored in the internal (iliasdata) directory and are not accessible directly from the web

7 Contact

• Author of the Request: Schmid, Fabian [fschmid]
• Maintainer: Schmid, Fabian [fschmid]
• Implementation of the feature is done by: Schmid, Fabian [fschmid]

8 Funding

If you are interest in funding this feature, please add your name and institution to this list.

• sr.solutions
• ...

9 Discussion

JourFixe, ILIAS [jourfixe], 11 MAY 2020 : Fabian gave a first presentation of this feature request which is still in progress.

Schmid, Fabian [fschmid] , 11 MAY 2020: The two questions mentioned above could be discussed at the JF:

- We prefer a dedicated logfile for errors that occur during migration. this makes it easier for system administrators to find such errors. if possible, add a permalink to the log to non-migratable objects (in the case of file-objects this is possible).
- errors during ad-hoc migration should be reported to the user. if no file version can be migrated at all, the file-object should remain, but then with a message that no version is available.

it was also discussed that files that no longer have any reference in the database should not be migrated and remain on the hard disk. administrators can then delete the old directories manually. As soon as the PRs for the service and further elaboration of the migration are ready, they will be presented again at the JF for final acceptance.

JourFixe, ILIAS [jourfixe], 26 OCT 2020: We highly appreciate this suggestion and schedule the feature for ILIAS 7. Fabian will implement an additional parallel mode where no ad-hoc migration is executed. This might prevent performance problems at huge installations where >1000 concurrent users are requesting file objects. A dedicated setting will be implemented in the ini file.

10 Implementation

The migration uses a current version of the Setup - Add migrate Command, which is not yet merged. There will be some adjustments before this will me be merged.
In general, a migration is diveded to multiple steps, in case of the FileObject-Migration, every single file in ilias/ilFile/... is one step.
With a migration helper, other components will be able to use it, too:
- Iteration of a directory and reveal next file to migration
- Regex to get ILIAS Object-ID out of a path
- Regex to get version (if applicable) out of a path
- Method to reveal existing StorageIdentification out of ILIAS Object-ID (to append versions, if applicable)
- Handling of deletion of empty directories (if all files have been removed)
- Handling of moving/copy files during migration

Test Cases

Test cases completed at {date} by {user}

• Tackled by all existing Testcases of File Object

Approval

Approved at 2020-11-12 by Amstutz, Timon [amstutz].