Show Advanced KnowledgeHide Advanced Knowledge

IRSS: General Overview of Components to Migrate (ongoing)

1 Initial Problem

The central file handling is to be reorganized with the introduction of the File Storage Service.  For more information, please check the related project page : [[[Project] ILIAS Resource Storage Service]]

2 Conceptual Summary

This list shows all the directories that are currently available on a test installation, for example, and gives a realistic picture of what may still need to be migrated. However, many directories are only used for exports, and these can all be migrated. Other types still have to wait, e.g. structured files, as the basis for this will only be implemented with ILIAS 9 (presumably). 

Main-Directory	Directory	Main-Type of Files	Status ILIAS 9
iliasdata	HTMLPurifier	Unknown
iliasdata	PersistentCertificates	Single Files
iliasdata	assessment	Unknown
iliasdata	auth	Unknown
iliasdata	bibl	Single Files	Migrated
iliasdata	bibl_data	Single Files	Migrated
iliasdata	blog_data	Export
iliasdata	chat	Unknown	was dropped
iliasdata	chatroom	Unknown	was dropped
iliasdata	chtr_data	Export
iliasdata	cmix_data	Export
iliasdata	copa_data	Export
iliasdata	course	Export
iliasdata	dcl_data	Export
iliasdata	embedchat	Unknown
iliasdata	exercise	Unknown
iliasdata	feed_data
iliasdata	files	Unknown
iliasdata	fold_data	Export
iliasdata	forum	Unknown
iliasdata	glo_data	Export
iliasdata	group	Unknown
iliasdata	grp_data	Export
iliasdata	htlm_data	Export
iliasdata	iass_data	Export
iliasdata	ilAdvancedMetaData	Unknown
iliasdata	ilCategory	Export
iliasdata	ilCategoryReference	Export
iliasdata	ilCourse	Export
iliasdata	ilCourseReference	Export
iliasdata	ilExercise	Single Files	Partly migrated
iliasdata	ilFile	Single Files	Migrated
iliasdata	ilForum	Single Files	Partly migrated
iliasdata	ilGroup	Unknown
iliasdata	ilRestFileStorage	Unknown
iliasdata	ilSession	Unknown
iliasdata	ilUser	Export
iliasdata	ilUser_personal_data	Unknown
iliasdata	ilVerification	Unknown
iliasdata	import	Unknown
iliasdata	itgr_data	Unknown
iliasdata	lang_data	Unknown
iliasdata	lm_data	Export
iliasdata	lso_data	Export
iliasdata	magpie_cache	Unknown
iliasdata	mail	Single Files
iliasdata	map_workfiles	Unknown
iliasdata	mcst_data	Export
iliasdata	mep_data	Export
iliasdata	orgu_data	Export
iliasdata	poll_data	Export
iliasdata	prtf_data	Export
iliasdata	prtt_data	Export
iliasdata	qpl_data	Export
iliasdata	role_data	Export
iliasdata	sahs_data	Export
iliasdata	scorm-import	Unknown
iliasdata	skmg_data	Unknown
iliasdata	spl_data	Export
iliasdata	sty	Unknown
iliasdata	sty_data	Unknown
iliasdata	svy_data	Export
iliasdata	tst_data	Export
iliasdata	upload	Unknown
iliasdata	user_import	Unknown
iliasdata	usrf_data	Export
iliasdata	webr_data	Export
iliasdata	wfe	Unknown
iliasdata	wiki_data	Export
iliasdata	xtst_data	Plugin Files
data	IASS	Single Files
data	LSO	Single Files
data	assessment	Unknown
data	certificates	Structured-Files
data	chat	Unknown
data	chatroom	Unknown
data	container_data	Single Files
data	course	Unknown
data	courses	Unknown
data	css	Unknown
data	custom_icons	Single Files
data	draw	Unknown
data	exercise	Unknown
data	ilBookingManager	Unknown
data	ilDidacticTemplateIcons	Single Files
data	ilExercise	Single Files
data	lm_data	Structured-Files
data	lti_data	Unknown
data	mobs	Structured-Files
data	obj_data	Unknown
data	openidconnect	Unknown
data	orgu_data	Unknown
data	previews	Single Files	Migrated
data	prg	Unknown
data	prg_data	Unknown
data	pub_badges	Single Files
data	scorm	Structured-Files
data	sec
data	sec/ilBadge	Single Files
data	sec/ilBlog	Unknown
data	sec/ilIndividualAssessment	Unknown
data	sec/ilPoll	Single Files
data	sec/ilPortfolio	Unknown
data	shop	Unknown
data	srv	Unknown
data	sty	Unknown
data	sty_*	Unknown
data	thumbs	Unknown
data	usr_images	Single Files	Migrated
data	xnob	Plugin Files
data	xqcas	Plugin Files

This feature request is used to get the agreement to modify all affected components in release 7. Further feature requests are required. Please note the following overview of the affected components.

Currently (at least) the following modules use ilObjFile and must be modified to use the new file storage service:

• File
• Exercise
• DataCollection
• Bibliography
• Glossary
• Learning module
• SCORM
• Test
• Calendar
• Page Editor
• Weblink
• Session
• Forum
• Profile

Modules that use the page editor must also be modified:

• Wiki
• Blog
• Content Page
• Learning module
• DataCollection
• Containers (Course, Folder, category, group, item group)
• Glossary
• Media pool
• Portolio
• SCORM
• Test
• Authentification

There are also many components that manage the files themselves and store them in the file system. They have to be modified as well. Basically all components that internally use \ilUtil::moveUploadedFile() are also affected.

3 User Interface Modifications

3.1 List of Affected Views

none

3.2 User Interface Details

none

3.3 New User Interface Concepts

none

4 Technical Information

in a first step the file object itself is modified, see: IRSS: Migrate File-Object-Files to IRSS
in this step a wrapper is also built to keep the above components working.
these components should rebuild internally and use the File Storage Service directly

5 Privacy Information

see IRSS: Migrate File-Object-Files to IRSS

6 Security Implications

see IRSS: Migrate File-Object-Files to IRSS

7 Contact

• Author of the Request: Lorenz, Katharina [klorenz]
• Maintainer: Schmid, Fabian [fschmid]
• Implementation of the feature is done by: {The maintainer must add the name of the implementing developer.}

8 Funding

If you are interest in funding this feature, please add your name and institution to this list.

• sr.solutions
• …

9 Discussion

JourFixe, ILIAS [jourfixe], 11 MAY 2020 : We highly appreciate this suggestion and schedule the feature for ILIAS 7. We will try to implement this new service for as many components as possible for ILIAS 7 and complete the project with ILIAS 8. This feature request covers the implementation also for components that are not listed above. We ask all maintainer to list their components that have already implemented the file storage service (in chap. Implementation).

Schmid, Fabian [fschmid] Implementd for File-Object in ILIAS 7, see IRSS: Migrate File-Object-Files to IRSS

10 Implementation

Please see the linked per-component requests above.

Please read the following text concerning Umlauts in different components (which came up in Mantis Ticket 29510):

With ILIAS 7 the ResourceStorageService was implemented, which now - from the components that use it - handles the storage of files and their metadata and takes care e.g. of special characters such as german umlauts. That means, on the hard disk no more files with original names are created, but only a resource and the metadata, like also the file name, are exclusively in the database, which can handle special characters very well. This means that the base now provides all the tools to be able to fix the special character problem. In principle, this works in such a way that ILIAS objects can e.g. submit uploads in the storage service and do not have to remember e.g. metadata such as the file name themselves. In the case of the File object in the magazine, this is now the case. For historical reasons, the File object is not only used for the magazine object "File", but other modules also use the File object in the background to store files, including the following modules:

• Datacollection
• Mediaobject
• Forum
• Glossary
• LM
• Mediacast
• Mediapool

With ILIAS 7 we have adapted the FileObject in such a way that it behaves towards the outside - i.e. towards the other modules - as possible as before, but internally it does the storage via Resource-Storage-Service. More or less the only thing that the FileObject still stores itself is the title of the object (which does not have to be the same as the file name). You can see this e.g. in the versions tab, there are the filenames of the uploaded files, the object itself can have a completely different name.
Now it depends in the modules, which use the file object internally, whether they store the file name themselves, or whether they set the file name to the file object as a title or just use the information of the respective uploaded file, when it comes to displaying or downloading the file again. Let's assume that the DataCollection receives an upload, stores it as a File-Object and sets the title of the file from Bär.jpg to Baer.jpg and every time the file is downloaded, the DataCollection again takes the title (i.e. Baer.jpg) as the name for downloading, then the DataCollection has to adapt this.
In the medium term, the modules should no longer use the File object to store files, but instead use the Resource Storage service directly. But this needs adaptations to each of these modules, which we are now tackling for ILIAS 8 piece by piece. Then each module can implement this in such a way that, for example, the file name is not set as a title and special characters are replaced, but the module can confidently simply give the upload to the resource storage service and no longer worry about such things. I took the DataCollection as an example earlier, because that's exactly what we're tackling as the first object to come into the JF over the next few weeks. And after that all other objects that currently still use the File object in the background. For the developers of the respective modules it becomes much simpler after the reorganization and a coupling to the file object is thereby also gone, which gives the file object again more clearance, in order to develop itself - with features - further.
But there is still one point to consider. Even if all modules are adapted and use the service and thus actually have the correct file name available when the file is to be downloaded or displayed, there are still cases in which the file name of a downloaded file does not correspond to the original: When delivering file names, ASCII should be used in each case according to RFC https://tools.ietf.org/html/rfc2183#section-2.3, i.e. ILIAS should convert characters in the file name to ASCII and replace unknown (special) characters with "_".