Feature Wiki

Information about planned and released features

Tabs

Revision of ILIAS Test Question Creation Process

Page Overview

[Hide]

This feature request is part of (Project) Revision of Test Question Creation.

1 Initial Problem

Currently, for every one of the 15 question types in the ILIAS core, the creation works differently in sometimes very subtle ways. This is the product of the long history of the Test & Assessment. Some questions types are older than others. Features have been added over time and changing paradigms have never been backported to existing questions.

This culminates in many problems:

  • Not every form is reachable by keyboard.
  • The inputs must be sanitized individually and often this goes wrong (I look at you --> Cloze Question).
  • We have 3 different editor options: Unformatted input field, TinyMCE and IPE.
  • We have different ways to integrate pictures into options for answering. * aso.

Maintaining this is nearly impossible.

Additionally, the coupling of the Test- and the TestQuestionPool-Components and the questions themselves is far to strong, requiring changes in many places for every improvement and creating doubt about the places where the actual "source of truth" is. This makes it currently impossible to implement specialised Components for different use cases, e.g. a Quiz-Component for formative assessments or an Exams-Component to only run exams, leading to a Test-Component that is over complex and conducive to configuration- and other errors.

Further, there are many feature requests that have been around for a long time and would implement understandable needs, but cannot be implemented as implementation would be very complicated and could only be done in a way that would further weaken the stability and integrity of the system.

Finally all forms in the test are still based on the unmaintained Legacy-Forms and are getting harder and harder to keep in synch with the look and feel of the rest of ILIAS. We won't be able to change this in one fell swoop, but we would like to start this process by moving the creation- and edit-forms of the questions to UI-Forms.

1.1 Timeline

It is our hope that we are able to migrate all question types with ILIAS 12 and that we then can start using the new questions in the Test, the Test Question Pool, and the Learning Modul in ILIAS 13. This is rather optimistic planning though and it is very possible that we will need an additional version to get things done.

2 Conceptual Summary

2.1 Objectives

In general the proposed changes aim at future-proofing the questions. To achieve this it the following objectives are pursued:

  • Simplifying the handling of user input when creating/editing questions. This will allow us to be sure about the type of input we receive and how to present it to the user.
  • Decoupling the three different aspects of questions in a test:
    • The question itself.
    • The place to manage questions implemented in the component "TestQuestionPool".
    • The tool to present the questions to participants implemented in the component "Test".
  • Reducing redundant code.
  • Providing an infrastructure for questions that allows to add additional capabilities to questions, so that currently proposed features can be implemented easily and in a structured way.
  • Providing a clear-cut way to add question types outside of the ILIAS-Core.

2.2 Guiding Concepts

We are trying to provide a concept that is about "questions" in general not about "questions for assessment" and we believe it is possible to do so.

The ILIAS-Community has spent a lot of time discussing the question what a question actually is. We stipulate that this question is actually not very relevant, as a question is just a message that solicits an answer and as such isn't very interesting in our context. Additionally, on a more technical level, the question is independent of the way to answer it. We propose to put the answering of the question in the center of our attention.

We thus stipulate that the question is just a way to provide stimuli to a user so that she can react to them. We use the singular "question" for convenience and clarity, but stipulate that a question can be more complex and consist of multiple stimuli and multiple opportunities for the user to react to them. This is already the case today, e.g. in the cloze-question or on a more technical level in the KPrim question and will be expanded through the proposed changes. We conclude that the presentation of the question profits from a rich environment where different types of stimuli can be provided and it is mostly the responsibility of the creator to make this input clear and concise, i.e. to manage the extraneous cognitive load of the participant.[1] The ILIAS Page Editor is the logical tool in ILIAS to provide an interface for creating this stimulus.

We thus focus our attention on the options for answering a question. These options need to be created, presented, and handed in. The answer handed in needs to be processed, persisted, and it might trigger a reaction like showing a feedback or proposing content for further learning. The answer a participants tries to hand in might also be unacceptable and thus need an instant reaction, even before it is processed. From a technical perspective we are always dealing with form-inputs, the creation of them, the presentation of them, and the reaction to their submission. Additionally, many of these actions may need to be processed in the client of the participant without being able to rely on the availability of an internet connection.

We do not believe that all types of answer form need to or even can provide all the capabilities. One type might not offer a way to calculate points as it is only relevant in surveys, another one is unable to run without an internet connection and thus cannot be answered purely in the client of the participant.

2.3 Structural Changes

  • Questions are moved to their own component called "Questions". The questions in this component are modular, i.e. they can have points for answers or not, they can have feedbacks or not. They will in this way be designed to make them usable across multiple scenarios.
  • The available types of answer forms will not be configurable on a system level. If a consumer wants to make the available types of answer forms configurable, as is the case for the test, it needs to implement its own configuration interface.
  • The units will be moved to component "Questions".
  • All code concerning the import and export of questions will be remove from the component "QTI". The aim is to also remove the rest of the code, but this depends on further changes in the component "Test".
  • The question types "Cloze", "Longmenu", "Numeric", and "TextSubset" will be merged to one answer form type.
  • The question types "Multiple Choice (Single Answer)" and "Multiple Choice (Multiple Answers)" will be merged into one answer form type.
  • The question types "Ordering Horizontal" and "Ordering Vertical" will be merged into one answer form type.
  • The administration of skills in questions is moved to the component Questions and linked to the editing of the answer forms.
  • All code about presenting questions will be centralized in the component "Questions", including code used to render a question client side currently distributed across multiple components.

2.4 Changes in the Backend

  • The ids of questions will be moved to `UUID`s, this will allow to identify questions more or less uniquely across different platforms.
  • Adding a question to a test will always create a copy with a new `UUID` the original `UUID` will still be held in the field `original_id`.
  • The field `external_id` is dropped and will not be kept any longer.
  • Only questions that have been copied into a test will have an `original_id`. The field will be emptied when the two questions diverge.
  • With this changes questions will stop being of a certain type. They may contain answer forms of different types. It is thus possible to add multiple answer forms to one page containing answers of different types. Each of the answer forms will be treated independently when handed in.
  • A question is explicitly allowed to not contain any answer forms. It then does not award any points and does not show up on e.g. the manual marking views of the test.
  • Feedback and Skills will be optional properties of answer forms. Content for Recapitulation and Taxonomies will still be an optional property of the question.
  • Consumers will not receive any information on the question upon creation or copy. The component "Question" will store the `obj_id` of the object a question belongs to and, if configured accordingly, the position of the question.
  • Consumers will receive a `response_id` upon handing in a response and are responsible for safekeeping. The component "Question" will delete responses if the corresponding question is deleted.

2.5 Changes in the Interface

  • The creation- and edit-forms of the questions will be moved to UI-Forms. If new Inputs are needed to achieve this, they will be implemented and coordinated with the UI-Coordinators.
  • The way to edit questions will be unified, replacing the three different editors by a single one per use case.
    • The ILIAS Page Editor will be used to create the question itself and to input the question-text.
    • For all other inputs Markdown will be used.
  • The current input field "Question" will be removed from the create/edit form.
  • Clicking on the title of a question in a list of questions will enable the question-editing-mode.
    • All controls to leave the screen will be removed:
      • The Meta Bar will only show the help.
      • The Breadcrumb will be disabled.
      • The Main Bar will be emptied and will only present two buttons:
        • A first one "List of Questions" toggling a list of questions in the slate.
          • Clicking on the question titles in the list in the slate will let you move from one question to the other.
        • A second button labeled "Create question" leads to a view showing a minimal form to create a question in the main content:
          •  a text-field "Title",
          • a text-field to enter the author pre-filled with the name of the current user,
          • a dropdown to select the life-cycle set to "Draft",
          • a textarea "Remarks", containing the information currently in the text-field labeled "Description"
          • and a radio input "Create Mode" offering two options "Simple" with the byline "Only unformatted text can be added for the question text and exactly one answer form will be created." and "Full" with the byline "All options can be used for the question text and as many answer forms can be added as needed.". The selected option will be remembered for future creation processes. The default will be set to "Simple".
          • Creating a question in the mode "Simple" will lead to a new form offering two inputs, a textarea "Question Text" and a select to choose the type of answer form. Creating a question in the mode "Full" will open the edit-view of the ILIAS Page Editor without leaving the question-editing-mode.
          • The mode does only apply to creation! Editing a question will always happen through the ILIAS Page Editor.
    • The main content area of a question will contain:
        • Two Tabs "Question" and "Statistics" same as today
        • A form to edit the basic properties of the question containing the same fields as the form to create a question
        • Underneath the form a preview of the question in a panel; the actions of the panel will contain the entries:
          • "Edit" that will open the question-page for editing (analogous to the current button "Edit Page")
          • "Reset Preview"
    • Inserting the answer forms and configuring them will be similar as it currently is in the Learning Module.
    • All the above views are presented in question-editing-mode and the Main Bar always contains the buttons to toggle the list of questions, to show the create form and potentially a tools button to toggle the controls of the Page Editor.
    • You can always close the question-editing-mode through the Mode Info to get back to where you started.
    • The view to edit answer form will have a "back"-button, labeled "Page" as in other page elements.
    • The option to directly add a question to a pool when creating it in a test will be removed.
    • Questions may contain answer forms of different types. It is thus possible to add multiple answer forms to one page containing answers of different types. Each of the answer forms will be treated independently when handed in.
    • Feedback and Skills will be edited in tabs accessible from inside the view to edit the answer form. This means that the view to edit the answer form will contain three tabs "Answer Form", "Feedback", and "Skills".
  • For filtering in Test and TestQuestionPool the options 'Contains only', allowing to select a single type of answer form, 'Does contain', and 'Does not contain', both allowing to select multiple types of answer forms, will be provided.
  • The element "Question" in the ILIAS Page Editor used in ILIAS Learning Modules will need to be adapted:
    • It will not be possible to create or edit questions directly in a page of a ILIAS Learning Module. It will be possible to add questions directly to a ILIAS Learning Module (without the need for a "QuestionPool") on the Tab "Questions". For this a button will be added to the page and each row receives actions to edit and delete the question.
    • The element "Question" will then be used to add a question to the page either from a "QuestionPool" or from the ILIAS Learning Module itself. It will display the same table as is currently used in "Add from Pool" and "Add from Test" in the component "Test". The input "Question Pool" will be replaced by the new repository picker input set by default to the ILIAS Learning Module itself. If no questions are present in the current ILIAS Learning Module an Info-MessageBox is shown informing the user that questions can be created in the tab "Questions".
    • The "Number of Tries"-input will be moved out of the question and into the form to insert the question into the page.
  • For complex forms for example the ones to create/edit "ErrorText" or "Cloze" questions, the create form will be staged (i.e. a wizard/roundtrip) and you will never be able to edit the interdependent sections of the question simultaneously. Editing will differ fundamentally from creating the answer forms. An orientation will be provided to show you what step you are in. This still needs to be specified and it should be part of a stepped/multi-step/wizard-form container in the UI-Framework. This container still needs to be developed.
  • If a complex form is used, every step of the form will be directly accessible from the start/overview screen used to edit the question.
  • Every form containing fields that could previously be entered with the tinymce or the ILIAS Page Editor and that now are Markdown fields, will provide a button above the form to fill in the previous texts stripped of all html.
  • The component "Questions" will provide a node in the administration under "System Settings and Maintenance". It will show a tab "Questions" listing all questions in the system and a tab "Units" where the units can be configured.

2.6 Migration

  • Current texts will be frozen and kept for the foreseeable future, i.e. it will be displayed as long as no changes are required, but will need to be moved to the new implementation to be updated. They will be used for presentation when no migrated version is available and deleted as soon as the text has been migrated. We will only keep rendered html-versions of the texts. The current database structure for legacy text will be restructured and simplified and thus the texts will still need to be moved around as the database is cleaned up.
  • The legacy texts will only be shown in the preview and the view for participants, but not in the forms to edit answer forms, as it would be close to impossible to display this information in a consistent way in the edit-forms without huge modification to the UI-forms and an astronomical chaos on the screen.
  • The legacy question text will be shown in a custom Page Editor element until they are deleted. The only action available on it will be to delete it.
  • We will move all questions types to answer form types in one go in order to keep a consistent interface and not have to bother with presenting both versions in parallel. This will mean that the migration to the version this is introduced in (we aim at ILIAS 13), will be long and resource intensive, if you do not run the migrations in advanced (see next point).
  • We will provide the corresponding question types in in the code as soon as possible also offering migrations for the question types that are already implemented. These migrations will not delete any data but copy them into the new component "Questions".
  • We propose to add a very rudimentary viewer for the questions, so that the migration can be tested on real data without causing any loss or more fundamental pain.
  • Tests, Question Pools, and Learning Modules will NOT work without the migration being run, they will just show a message that the migration must be run.
  • One version after adapting the Test, the Test Question Pool, and the Learning Module to use the new component a migration to drop all legacy tables will be added and the migration to migrate question types will be removed.

2.7 Additional Remarks

  • We will not change the functionality of the Test or the TestQuestionPool as part of this request.
  • The function to manually mark questions will not become a function of the component "Questions", but will stay part of the component "Test".
  • All rendering of questions will be centralized in the component "Questions". Corresponding code for asynchronous rendering currently present in other components will be moved there.
  • The element "Questions" in the ILIAS Page Editor will be left as part of the component "COPage" and will not be moved to the component "Questions". The corresponding changes will be provided as a PR as will the changes to the ILIAS Learning Module.
  • An additional page element will be created to display the legacy "question".
  • A change of the scoring system for all types of answer forms that would allow to separate the correctness of an answer from the points was considered and researched (see this feature request). The move was deemed to be a step backwards as it would decrease usability but increase complexity. It is insufficient to move to a system that tags answers as right or wrong, as we believe the current implementation that provides for answers to be partially correct to be the right one and it would be impossible to migrate existing questions to a purely binary system, even if we didn't. This move would work form some question types, but for most it would call for a lot of additional input fields (imaging the answer form type "Cloze" where we would need weights on the level of the question, the answer form type and possibly also the gap) and it would sometimes completely unclear what an index of correctness would even mean.
  • We will need two additional icons: One for the "List of Questions" one for "Create Question".

3 User Interface Modifications

3.1 List of Affected Views

  •  `ilAssQuestionPreviewGUI`
  • `assQuestionGUI::edit()`
  • `assQuestionGUI::suggestedsolution()`
  • `ilAssQuestionFeedbackEditingGUI`
  • Pages of the ILIAS Page Editor offering to add questions

3.2 User Interface Details

Base Page to Edit a question, currently the List of Questions is implemented as a Panel in a Legacy Slate. This implementation will need to be discussed with the UI-Coordinators.
Base Page to Edit a question showing the create form
Simple create mode form
Full create mode with a single answer form and the option to add another

3.3 New User Interface Concepts

  • We would like to expand the KS-Forms with the option to add multiple buttons, defining the action happening after the form hast been saved. This would hugely improve the user experience e.g. for creating questions as it would allow to add an action "Save and Create new" and it would allow to keep the current behavior that allows for questions being saved with a return to the previous page or staying on the edit form.
  • We would like to add an input-container for forms with multiple steps to the UI-Components.
  • We propose to expand the KS-Forms with the option to send them asynchronously. This would allow to reduce to dynamically build forms depending on parts of their input without having to reload the whole page, we could thus e.g. show the form to edit the gaps in a "Cloze" question only after the cloze text is input.
  • We propose to allow an action on a panel to be defined as primary and to display it directly if we are not on a small screen or if it is the only one.
  • There will surely be a need for adaptations to inputs in the KS-Forms. These will be specified in the individual feature requests specifying the changes for each question type.

3.4 Accessibility Implications

The forms to create questions are based on legacy-ui-forms with a lot of custom inputs that have a huge amount of accessibility issues. By moving these forms to KS-Forms a lot of the issues should be resolved. Additionally it will be easier to address future accessibility issue.

4 Additional Information

4.1 Involved Authorities

If this request is related to multiple components, please list both authorities for all related components.

4.2 Technical Aspects

  • The component should use the new integration mechanism provided by the component "Component". This will need some additional preparatory work in other components. We aim at providing this as PRs, but depend on the collaboration of other code authorities for it to succeed.
  • Logging:
    • The Test-Logger has been refactored with ILIAS 10, but there is still some coupling of questions with the test logger. The generation of logs needs to be completely decoupled by creating a `LogInformationGenerator` for questions, that would than be injected into the `AdditionalInformationGenerator` in the test, and by moving the creation of the `TestUserInteraction`s out of the question. This will also allow other components to implement their own logging facilities with access to information from editing and answering questions.
  • Current Question Plugins will need to be adapted for the new structure. Custom answerforms can be implemented in additional components by implementing the corresponding interfaces and will be collected by the components Questions. Corresponding information will be available in a proper documentation inside the component.

4.3 Privacy

This feature request doesn't change the data that is collected and also doesn't change how it is presented. It clarifies the storage of answers by users answering questions, by doing so it will improve the understandability and the structure of the data thus making it easier to control the data's lifecycle and to inform the user correspondingly.

4.4 Security

By moving to KS-Forms and restructuring and clarify the storage of data, the data will be easier to control and user inputs in the KS-Forms will be more consistently validated and sanitized on output.

4.5 Contact

Person to be contacted in case of questions about the feature:  Kergomard, Stephan [skergomard]. For questions about the funding, please contact: Strassner, Denis [dstrassner]

4.6 Funding

Funding status and funding parties are listed in the block 'Status of Feature' in the right column of this page.

If you are interested to give funding for this feature, please get into contact with the person mentioned above as 'Contact'.

5 Discussion

Seiler, Yvonne [yvseiler] & Weber, Jens [jens.weber], 26 SEPT 2025:

The University of Bern supports this project and welcomes the concept that has been developed and presented here, as well as in Marburg in September. Many thanks to Stephan Kergomard and everyone else involved! Given the situation in T&A (number of issues, code structure, lack of future sustainability, etc.), it is clear that something needs to change.

The concept described here is a well-thought-out and far-reaching approach. With the planned standardization and centralization, we believe it to be extremely promising and sustainable. It also opens up new possibilities, such as the long-cherished desire for multiple questions on one page or the simple use of existing questions in plugins or other ILIAS objects. 

In order to get an even better impression of the planned workflow in terms of usability and user experience, we think it would be useful to tackle a prototype in one of the next steps, which would allow us to determine how the creation process “feels.”

In our view, this concept is a big step forward in finally getting a better handle on the existing problems associated with test question creation and maintenance. We look forward to the comprehensive improvements and opportunities that this concept will bring.

Jackisch, Ingo [jackisch], 2025-10-02; This will be a big step forward for many question- related topics. Many thanks for the approach. Please keep in mind, that there are many questions asl choice or cloze type, which use LaTeX in the question texts and in the answers (single line, too). Since we anticipate changes in the LaTeX- handling in ILIAS core, it woud be a good move if answer forms support LaTeX natively, so that the need for a plugin to keep the functionality of existing questions. There are many of these in the optes-results which are used by several institution an d I am not sure if all of them are aware oft the inpacts of the changes in question handling and LateX streamlining.

Kergomard, Stephan [skergomard], 02 OCT 2025: Currently we can wrap the content to render MathJax. This should work already. Mathjax will not be rendered, as far as I can see, in the preview of the markdown editor, we will need to see how we adress this. I will keep this in mind, especially in regard to legacy content in questions, and will check for it during implementation. Thanks for reminding me to ensure the support is working.

Gorgas, Martin [gorgas], 06 OCT 2025: I need to strongly object to removing the external_id. Our entire test and exam workflow (IMS → ILIAS → Examinator → Statistics) relies completely on the external_id for cross-system traceability and reporting. The new ILIAS-UUID is generated too late in the process. It is unknown in our upstream (IMS/Authoring) and downstream (Examinator/Klausursystem) systems. Without the external_id acting as the original item reference, we completely lose the ability to match ILIAS results back to the original source question.

Sturm, Johannes [sturm], 06 OCT 2025: The University of Freiburg is experiencing the same problems as Martin Gorgas with importing and exporting with IMS. We need the external ID to be able to exchange data between ILIAS and the third-party system IMS, and this would significantly impact or even prevent us from conducting medical tests (summative and formative). 

Kergomard, Stephan [skergomard], 06 OCT 2025: Thank you very much Gorgas, Martin [gorgas] and Sturm, Johannes [sturm] for your feedback! If I understand you correctly, your problem is based on a misconception, though: If I'm getting it right you are running a scenario where you import your questions from some system and then export your questions to another system. Upon import with the proposed system you would need to provide a UUID for the question, it would not be generated for you (apart from one ILIAS-version at the time of introduction to ensure backward compatibility of imports). So the UUID would be under your control anyway. There are a few interesting possibilites when it comes to collisions of UUIDs, but these we will need to discuss as part of the Import-Export-Concept done by Joußen, Thomas [tjoussen]. I can imagine that we keep some kind of external_id in this context, but we need to define it very precisely, explicitely, and consistently for all scenarios, and I see this, as said, as an issue of import-export. From the point of view of the question service, in a first approximation, I think the intent to get rid of all secondary IDs is the right one. Again: I'm very glad to have this here in the comments, to keep this issue in our field of vision.

Seegers, Christina [csee], 06 OCT 2025: 3 Questions: I) Will it still be possible (with the combination of the question types into this new groups) to enable/disable them separately (within the adminstration settings) for each installation, or will this lead to the situation where either all are allowed or none? II) And what will happen if a test is copied, will all questions get a new UUID and keep the original_id of the question of the original test? And III) What will happen if questions need more than a text as input. I am thinking about formula questions (and STACK). How will such (randomized) inputs for the text field be part of the workflow?

Kergomard, Stephan [skergomard], 06 OCT 2025: Thank you very much for the questions Seegers, Christina [csee]. Let us see if I understood everything:

  1. The component "Questions" will not have an option to enable or disable types of answer forms, but the current settings in the test will remain. This is so to make the questions useful across different components as you might want to allow a certain type of answer form in a future component for formative quizes, but maybe not in the one for exams.
  2. If a test is copied all questions will receive new UUIDs and loose all connection to the place where they were copied from as these two places have no link to each other whatsoever. The links we have today are purely fictional: Two questions that are linked through an id -> original_id - connection, might today have nothing to do with each other anymore. The changes proposed here aim at making sure that this connection actually has meaning.
  3. I don't understand this question: As I understand it the answer forms of the type "Cloze" described in the corresponding feature request need far more than an input of the type text and are structurally widely equivalent to a formula question. The current question of the type "STACK" has a complexity to it that we cannot program against without loosing sight of the bigger picture. I currently think that it should be possible to create an answer form of the type STACK inside this framework, but moving the current forms in the question of type "STACK" to UI-KS-Forms will not be easy. This cannot be avoided though and is independent of this projects: All forms will have to move to the new forms as the legacy forms are not maintained by anybody anymore. I would suggest for the interested parties to make sure they get the move of STACK to the new framework started asap after the first versions of the component "Questions" is available.

Slotosch, Sven [sven.slotosch], 07 OCT 2025:

After reviewing the feature request, I have a few more questions

1. Timeline

Would it be possible to specify the timeline more precisely? I think it is important to know a more precise time frame so that, on the one hand, the developers of question plugins have the opportunity to adapt their questions to the new structure and, on the other hand, third-party systems such as IMS from UCAN can also be adapted to the new question architecture if necessary.

2. Migration

I'm a little unsure how the migration of legacy questions will work. Although you write "Current texts will be frozen and kept for the foreseeable future, i.e. it will be displayed as long as no changes are required", you note further down that "Tests, Question Pools, and Learning Modules will NOT work without the migration being run, they will just show a message that the migration must be run." In addition, you write "The legacy texts will only be shown in the preview". Does this mean that the questions have to be migrated manually by users via copy and paste? Currently, there is a similar process for the introductory and executive texts of the test. The texts are still available, but must be transferred to the new format by copy and paste. If I understand correctly, this would mean that several thousand questions would have to be handled individually by the users in order to transfer them to the new format so that old questions can be made usable in the test at all. And that in the shortest possible time, since you plan to "move all questions types to answer form types in one go in order to keep a consistent interface and not have to bother with presenting both versions in parallel." So there will be no way for users to continue running their old tests during the semester, while in the background they have to go through the potentially time-consuming process of (manual?) migration.

3. Multiple questions on one page, randomisation and question pools

With the new system, you hold out the prospect that it will finally be possible to display several questions/question types on one page. How will it be possible to draw answers (to stay with your structure) randomly from question pools? Will it even be possible to compile these pages from a question pool or do I have to create such a page anew or rewrite the answers every time I want to exchange/update individual answers?

Kergomard, Stephan [skergomard], 09 OCT 2025:

  1. Timeline: I'm sorry, this is completely impossible to say right now as it does not only depend on us, but on many external factors (mainly on funding: how much and when). What can be said: If nothing earthshattering happens, I  will have a first version of the component "Questions" running with ILIAS 12. Maybe it will only contain the Cloze-Question, but I hope there will be more. This should be enough for you to start the move of plugins, but maybe it is also worth to first ensure we are fully funded to implement the full component, first.
  2. Migration: "Migration" refers to the "Migrations" in the Setup. So: Once we have all question types you will need to have run the "Migration" in the Setup of ILIAS to be able to use any questions. Questions will work without all texts having been updated manually, but the old texts will not be visible in the forms to edit the questions. This is what was meant be "only in the preview". I rephrased it above to be clearer. So two different levels. You will have to have run the "Migrations" as the structure of the database will change and we will need to render the texts a last time. I hope this is now clearer, as there are multiple levels, this is not so easy to describe.
  3. Multiple Pages and Randomization: You will still be able to randomize the questions, but clearly not the answer forms. So: If you want to randomize everything, you need to create individual questions with only one answer form. This is actually exactly the desired behavior, as this allows to define blocks of answer forms that have a logical link. Nobody will stop you from just creating questions with a set of random answer forms in them, though. This would just not be the most fruitful use of the new infrastructure as it is actually designed as a way to bundle related answer forms not just to control the presentation.

Dette, Christian [dette], 09 OCT 2025:

Thank you very much for the concept and your presentation. The technical challenges are indeed tremendous. From a user’s perspective, most of the issues discussed often remain unnoticed, as the focus is primarily on the user interface and workflows. Therefore, the practical testing phase is always the greatest challenge; in fact, most problems only become apparent once the transition actually occurs. We have discussed this issue on several occasions, and it is essential that all parties complete test runs well in advance.

We would also like to reiterate previous comments supporting the development of an early prototype. Such a prototype would inevitably raise additional questions and highlight unforeseen challenges. It is important to ensure that sufficient resources are allocated for any necessary adjustments.

While the use of the page editor is a logical technical step, it may not be the most efficient solution due to the increased number of required clicks. Whether it ultimately meets the practical needs should be thoroughly evaluated through practical testing.

Nishino, Kenji [kentoni], 10. OCT 2025 | From the CompetenceCenter E-Learning, University of Cologne (Anja Löwe, Andrea Klein, Kenji-T. Nishino):

We would like to thank you for the carefully prepared and forward-looking contribution.
The project represents an important step toward modernizing the Test & Assessment area, but several conceptual clarifications are still required.

Open Points:

  • Test- and Question Pool-Relationship: The impact on the storage of questions, answers, and points is still unclear and needs to be modelled and documented.
  • Migration: The procedure (automated or manual) and the effort required for large question banks have not yet been defined. Dependencies between ILIAS objects need to be assessed.
  • Funding: Overall funding has not yet been secured. A project of this magnitude should only proceed after a binding financing commitment has been made.
  • Organization & Transparency: It is unclear who has been involved in the process so far and who will be engaged in the future. We recommend establishing a dedicated project coordination team to align stakeholders and ensure transparent communication.

Recommended Approach:

We suggest a four-phase model to ensure technical and organizational feasibility:

  1. Prototype: Implement 2–3 exemplary question types, use real test data and validate the migration process and performance.
  2. User Testing: Conduct UX and workflow tests with pilot institutions.
  3. Implementation: Integrate lessons learned, define realistic timelines, secure funding, and develop a go-live strategy.
  4. Jour Fixe Decision: Make decisions based on validated results.

Recommendation for the Jour Fixe:

We suggest that no final decision on implementation is made on Monday. Instead, the stepwise approach outlined above should be approved first. This will ensure that the project’s feasibility, funding, and practical usability are thoroughly evaluated before development work begins.

Closing Remark:

A phased, transparent, and financially secure approach will strengthen both the quality of the project and the community’s trust in this critical development step.

Mersch, André [amersch], 10. OCT 2025 | From #SeLL - Services für digitales Lehren und Lernen, Bielefeld University of Applied Sciences and Arts (Mahnke, Milva [m_mahnke], Becker, Matthias [matthias.becker], Kaiser, Sascha [skaiser], Mersch, André [amersch] ):

We support the proposed concept and thank Stephan Kergomard and everyone involved for their excellent and forward-looking work. The initiative offers a promising and much-needed foundation for the future of Test & Assessment in ILIAS.

We strongly support the suggestion made by Yvonne Seiler to develop a prototype, ideally in the form of an early-access test system, which we also see as a necessary prerequisite for approval by the JF. This would allow real question sets to be uploaded, migration behavior to be simulated (including error handling), and feedback to be incorporated directly into the redesign.

We also share the concerns already raised regarding the relationship between tests and question pools, the migration strategy-especially with regard to large-scale legacy content-and the general handling of dependencies. These areas require further clarification to ensure a smooth and feasible transition for all institutions.

In this context, we understand that developers may not be able to provide or develop a dedicated migration tool, so we emphasize the importance of clearly defining practical strategies to manage migration safely, including rollback options where feasible.

Furthermore, we would like to inquire:
How have usability-related issues been considered so far, and how will usability and user acceptance be systematically integrated into the process in the future?
Is there a migration concept for plugin developers, such as guidelines or compatibility layers, particularly for complex question types like STACK?

We recommend a more structured and transparent project setup, including broader community involvement e. g. via SIG-EA, to support successful implementation.

Kergomard, Stephan [skergomard], 10 OCT 2025: Nishino, Kenji [kentoni], Mersch, André [amersch] could you please clarify what you mean by "Test- and Question Pool-Relationship: The impact on the storage of questions, answers, and points is still unclear and needs to be modelled and documented." respectively "We also share the concerns already raised regarding the relationship between tests and question pools"? I'm having a hard time understanding what is unclear here: What do you see changing? Additionally: Could you please quickly explain what is unclear about the migration strategy?

Strassner, Denis [dstrassner], 10 OCT 2025: We will provide a prototype and an installation to accompany the progress as soon as there is some reliable code after the JF decision.

Rabah, Rachid [rabah], 10 OCT 2025 |From eCampus , University of Bonn : Thank you for the comprehensive concept. We recognize the necessity of addressing the current technical debt in Test & Assessment. To better assess the practical implications for our high-stakes examination environment, we would like to raise several questions.

1. Migration & Workload
1.1 Migration represents our most significant concern. We operate examinations in large scale with substantial legal implications, and any disruption could have serious consequences. Will migration tools assist with bulk conversions, or must enriched content be manually recreated?
1.2 What specific technical safeguards prevent migration failures? Is there a rollback mechanism if critical issues emerge post-migration?
1.3 What monitoring and logging mechanisms track migration progress in real-time?
1.4 Who is the point of contact for migration issues? What happens with skipped ILIAS versions or failed updates?
2. Data Integrity & Interoperability
2.1 The removal of external_id concerns us and other institutions relying on cross-system workflows. Our examination pipeline for the Medical Faculty  (IMS → ILIAS → Examinator → IMS) depends on this field for traceability, reporting, and grading. Those Worklflows are established and well tested. We need this ID. While UUIDs offer technical advantages, they're generated within ILIAS and unknown to upstream and downstream systems.
2.2 How is the connection between question text and answer blocks technically maintained? Can questions still be edited after being added to a test?
2.3 What impacts will the changes have on scoring logic and test statistics with multiple answer forms in one question?
3. Usability & Workflows
3.1 Most of our instructors are unfamiliar with Markdown. Have other alternatives been tested?
3.2 The wizard mode removes familiar navigation elements (breadcrumbs, back buttons). Have empirical usability tests been conducted to validate that this doesn't increase cognitive load or fear of data loss?
3.3 Our instructors' mental model centers on "one question = one complete unit." Their predominant use case involves creating discrete, independent questions. Please indicate which steps will be needed to create "several questions on one page"?
4. Accessibility
4.1 How accessible are the creation and navigation experience? From a user perspective, how accessible are interaction types such as matching or ordering questions?
5. Cross-object use
5.1 If questions are also to be used for learning modules, polls or surveys: Have all relevant authorities participated and agreed to the concept?

Conclusion
We recognize this modernization effort addresses fundamental technical challenges and fully support the University of Cologne's proposed phased approach (prototype → user testing → implementation → decision) as a pragmatic pathway to ensure both technical stability and secure funding. A phased, transparent, and financially secure approach will strengthen both the quality of the project and the community's trust in this critical development step. Thank you very much!

Kergomard, Stephan [skergomard], 13 OCT 2025: Thank you for your feedback Rabah, Rachid [rabah] and here I think II can give you some answers:

  1. Migration and workload:
    1. You are, as stated above, mixing two levels: There will be an automatic migration into the new structure and the questions will then simply work. The moment you need to change a text that previously allowed for formatting, you will need check this text and, depending on your needs, reformat it manually. As written above, there will be a button to at least transfer a pure text representation of your original input to the new forms.
    2. What do you mean by this? E.g. it is very well possible to have a broken question today that does not work in the system. It is also possible to have leftovers of questions in the database aso. So there will be no prevention of migration failures. As written above we will provide a process that will allow you to have enough time to check your migrations. We will have one version of ILIAS where the full infrastructure will be present, but will not be used yet, to give you enough time to do some checks and report issues that then will lead to improved migrations. This migrations will clearly fix already migrated questions. As they will not be in use yet, this gives us time to get things right. Currently, I would believe that we will also add temporary test cases to the test suite in Testrail, but this considerations are outside of the scope of this Feature Request (as they are for all of them). The new questions will also be strongly typed and have Unit Tests, so there are more guarantees that the new questions will work, as there are today for existing questions.
    3. As we are not deleting anything during the version of ILIAS previous to the full implementation we have a lot of time. In a first step, I would suggest to simply log to the console (as we do for all other migrations), if a migration fails and give the best estimate we have about the reason it failed. As always we have Mantis to track issues (bugs) in migrations. I don't kown what you mean by "point of contact for migration issues" beyond that.
  2. Data Integrity & Interoperability:
    1. Please see above.
    2. Nothing changes here.
    3. Nothing changes here. The only difference to today: There might be more than one answerform that can be scored on one page/in one question. Statistics are very rudimentary today, if you need more in this regard, this would be a Feature Request anyway. We aim at minimal changes in the presentation of the statistics.
  3. Usability & Workflows:
    1. The beauty of Markdown is that it also accepts pure text, so people can use the questions without any knowledge of Markdown. Additionally the input for Markdwon helps you out in a first step. But for a full answer, I think, your question needs to be split up into three separate ones: What type of input are we dealing with? Considering the  type of input, are there alternatives for its representation? Considering the type of input, are there alternatives for the interface? I will try to quickly tackle one after the other: In the context in which we are working here and within the framework of ILIAS we can distinguish three different types of content that can be provided by a user. Users can create a page with rich content that contains a lot of different elements ranging from pure text, to interactive images (imagemaps), videos, overviews and more. This is done with the ILIAS Page Editor (using XML for its representation). Then they can create what was defined as text during an earlier investigation by Klees, Richard [rklees] (See: How to Represent Text in ILIAS? and this comes again (simplifying slightly) in two types: "Text with Structure" and "Text without Structure" that I will call here "pure text". We do not have any discussion for pure text and pure text can also be added by the input for Markdown just ignoring the additional options or by simple inputs of the type testarea. Looking at the previous usages, through many discussion over the last few years, and from our own experiences we have come to the conclusion that what was previously added in the field "Question" is more than text. This is already rather obvious by the additional use of the ILIAS Page Editor in question creation. I do not believe we have working alternatives for this case. So we have covered two out of the three types. So the question is what we want to do with "Text with structure". Moving to the second question above, the one about the representaion of this text, we see that most of our current problems are due to us dealing directly with html and never nowing which part of it  is for the user and what is formatting. Clarifying and simplifying the subset of formatting we are working with and abstracting this structure is clearly going to do wonders for the reproducability and reliability of our outputs. So we can safely assume that a move to a `Text` object as provided by the facility implementing the concept in the document above that internally codifies a set of valid Markdown will bring us a huge step in this direction. Now to the final question of the type of input used to create this type of text. I believe the options are in turn very clear: We can either have  a WYSIWYG editor or an editor leaning more to a WYSIWYM paradigm. But this question is actually completely independent of the decision to use a Markdown, Markup, Word, or Pie-in-the-sky representation. We were using the WYSIWYG option with html as its representation form many years now by relying on TinyMCE. We found many, many issues (see above) with it and actually supporting it well is a pain. Currently TinyMCE is inofficially supported by Jansen, Michael [mjansen] with help by Kergomard, Stephan [skergomard] providing just the necessary. The last usages are slowly being removed from ILIAS and it is also loosing functionality step by step. This is not going to be a way forward. On the other hand the Markdown editor has seen continously increasing usage across the code-base and it is part of the UI Components. It right now tends rather towards a more spartan WYSIWYM paradigm and there are very good reasons to keep it this way, among many others its maintainability, but there are many Markdown editors around that tend towards a more WYSIWYG paradigm (e.g. the editor of Nextcloud, Zettlr, QownNotes and many more). So, yes, we have considered all the options we saw around us and we have consiously decided, that this is the right way forward, then allowing us to concentrate our efforts on steadily improving the one input type we use across the whole code base. And again: Your users CAN use Markdown, they don't have to use it.
    2. No, but could you give a theoretical reason why removing distractions, complexity, and opportunities to make a mistake will increase cognitive load?
    3. There is no option to create several questions on one page. You can create a Question and this Question is one page, then you can can create multiple answer forms on this question/page. With your question you are actually exactly pointing at the reason for implementing it this way: A Question should be a unit even if there are multiple answer forms on the page. So if your goal is to create multiple questions, please do so. The ILIAS Learning Module will be an exception to this rule, but this needs to be tackled there not here.
  4. Accessibility:
    1. I reitarate: The person taking a test will not see any changes to the interface as we are not tackling the presentation of content to her. The accessbility of the creation process is a question of implementation. We are relying fully on elements in the UI Components and as we are going to look closely at the accessibility (although I'm still rather unsure, what can go wrong there) thus ensuring the most accessible implementation currently possible within ILIAS.
  5. Cross-object use:
    1. The questions are created so that they CAN be used in these contexts. The move to this component is up to the corresponding authorities. The only exception is the ILIAS Learning Module that currently fully relies on the questions from the Test and for which the authorities of the Test & Assessment have been keeping the code running for the last years. We have proposed an approach  and would provide a PR as stated above, but then it is up to the authority to accept the PR. This is the same procedure we have e.g. used for updating the export in ILIAS 10.

Killing, Alexander [alex], 12 Oct 2025: Thanks again for the initiative to move forward with this important issue.

This request includes a number of different aspects regarding question workflows, general concepts and technical component related developments. Thus we should make very clear what any JF decision will target specifically.

On the top level, we have to decide how the development of a final implementation should be organized. We have various instruments for this: Projects, Workshops, Feature Wiki Requests, Pull Requests.

Trying to move forward with a sparse project page and feature requests only does not seem to fit the overall scope of the endeavor. Especially regarding a question service component, that should be integrated into multiple core components and could also be integrated into multiple plugins (live voting, interactive videos).

Larger projects like the page editor or the removal of legacy UI do not only come with a project page, but are lead by a dedicated project management. This includes the planning of open meetings, a financial overview and resource planning. It would be advisable to do the same here, too.

This will allow us to organize

  • Open Workshops for everyone being interested to work out the details of user interactions, e.g. creation processes for individual question types.
  • Overview on funding from different parties and having estimations on missing funding.
  • Early feedback from stakeholders.
  • A way to plan and organize usability tests and accessibility reviews an any prototype or implementation.
  • Workshops for the technical aspects of a question component involving other maintainers (interface design, planning of integrations and so on).

Even if this seems to be a higher investment to begin with, getting early feedback, reviews and test in a planned way will help to reduce risks in finding issues late. Taking care of any conceptual or technical issues late in the development process usually increases the costs to get rid of them. We also risk acceptance, if we do not open up the discussion. This is what a project management and broad involvement of stakeholders tries to avoid.

[TatjanaCB], 13 OCT 2025: We agree with the opinion that a prototype should be made available as early as possible. Furthermore, as mentioned above, we believe that, in addition to the manual scoring feature, it is particularly important to consider an effective post-correction feature of the questions ('corrections' / 'Nachkorrektur') for the new scenario.

JourFixe, ILIAS [jourfixe], 13 OCT 2025: We highly appreciate this suggestion and accept the feature for trunk.

As a support for implementing not only this feature request but the entire (Project) Revision of Test Question Creation the ILIAS society will take over the project management in consultation with the contractor Uni Hannover. This project management will support the responsible authorities to organise and run specific meetings and to communicate the latest results of the project and to collect feedback also from other developers at an early stage.

6 Implementation

Feature has been implemented by {Please add related profile link of this person}

6.1 Description and Screenshots

{ Description of the final implementation and screenshots if possible. }

6.2 Test Cases

Test cases completed at {date} by {user}

  • {Test case number linked to Testrail} : {test case title}

6.3 Privacy

Information in privacy.md of component: updated at {date} by {user} | no change required

6.4 Approval

Approved at {date} by {user}.

[1] This consciously ignores the system-environment as it is out of scope for the proposed changes.

Last edited: 13. Oct 2025, 14:49, Kunkel, Matthias [mkunkel]