Feature Wiki

Information about planned and released features

Tabs

Revise Cloze and Long Menu Question Creation Process

Page Overview

[Hide]

This feature request is part of (Project) Revision of Test Question Creation.

1 Initial Problem

The set of questions available has grown organically over many years. There are redundancies and inconsistencies and the forms to edit the questions are based on a lot of custom code based on the legacy forms featuring many unique concepts, like forms that are rendered by javascript on the client.

There is a clear need to bring this process back in alignment with the current way of doing things in ILIAS, to remove redundancies, and rectify inconsistencies.

Having too many question types that do not offer solutions to clearly distinguishable problems is confusing and makes using the ILIAS harder to learn.

For the named question types:

  • The Cloze and the Long Menu question are actually both Cloze Questions and both even offer to create select inputs to select the response to a gap, but the editing is fundamentally different and even the way gaps are inserted into the corresponding text and the tags used to represent a gap are different.
  • The Text Subset Question is also only a special case of a Cloze Question and it is already possible today to create a form that asks for different answers to the same question with a Cloze Question.
  • And finally the same is true for the Numeric Question, as there is a gap type "numeric", but even there the handling is inconsistent, as the Numeric Question only offers an upper and a lower limit for the correct response, while the Cloze Question also offers a correct answer.

2 Conceptual Summary

  • We propose to unify the question types Cloze, Long Menu, Numeric, and Text Subset into a single answer form type "Cloze".
  • The general rules formulated in Revision of ILIAS Test Question Creation Process will be followed. So all text editing will either be made in markdown or as plain text and the forms defined by the UI-Components will be used.
  • This step is only concerned with question creation and structure. The form to answer a question will basically stay as is and only undergo some minor tweaks.
  • The tag used to represent a gap will be unified and harmonised with "Mustache". "{{GAP}}" will be used upon creation, "{{GAP_<uuid>}}" for existing gaps. This way we don't rely on the order of gaps once they are created. When not editing the cloze text itself the tags will be replaced with a representation wrapped in square brackets and the uuid will be shortened to its first 4 characters, so the gap "{{GAP_0d439578-a36d-4eb2-8308-beb17ed381e3}}" will be shown as "[GAP_0d43]". This should give enough entropy to avoid collisions in most cases, and in the rare case it fails, humans are really good at parsing order.
  • There will be no protection of the tags for gaps. When editing the user is herself responsible for keeping them intact.
  • The types "Text", "Numeric", "Select", and "Long Menu" will be offered for gaps.
  • The type "Numeric" will not offer the input "Value" anymore, but only "Lower Bound" and "Upper Bound". Only the "Lower Bound" will be required. If only the "Lower Bound" is entered, the "Upper Bound" is assumed to be the same and the result presentation will state that the correct value is equal to "Lower Bound". This also happens if two identical values are provided. If both values are provided and not equal the result presentation will state that the value has to be between "Lower Bound" and "Upper Bound". A byline will be added to inform the user.
  • The complex process to create the gaps and then edit their properties will be broken up in steps and it will not be possible to edit the text containing the gaps and the properties of the gaps in one go.
  • The process will be different upon creation, where a process guiding the user will be presented, from the process for editing where direct access to change the different parts of the form will be provided.
  • The entry view to edit the answer options will not provide any form inputs, but present the currently available information. The first element will be a panel with the basic answer form properties "Cloze Text", "Text Matching Method", "Minimal Characters for Suggestions", "Scoring of Identical Responses", "Text Field Length", "Enable Gap Combinations", and a button leading to a form to edit them on a new view. The second element will be a `DataTable` listing the current gaps, with a text column showing a shortened version of the uuid, a text column showing their types labeled "Type", a column of the type List labeled "Answer Options Awarding Points" and a column showing the available points. If gap combinations are enabled an additional boolean column labeled "Used in Combination" is shown. Gaps that are used in combinations will have no values in the columns "Answer Options Awarding Points" and "Available Points". The action to edit gaps is presented as `StandardAction` allowing to edit one or more gaps in one go. As the forms to edit the gaps can be very long the process is shown as a sequence of pages and not in a modal.
  • The markdown-editor should be enhanced by a button to insert a gap. This is still being discussed with the corresponding authorities.
  • If the "Cloze Text" is changed on an existing question and new gaps are added the same process as upon creation is started, but presenting only the newly created gaps. If a gap is removed a corresponding warning is raised in a modal asking for confirmation.
  • Gaps of the type "Long Menu" will allow to award different points per answer instead of only right or wrong as today and thus make it the same as the text and select gaps.
  • Only gaps of the type "Text" will have an option to explicitely set their length. For the types "Select" and "Long Menu" the length of the field will be derived from the longest possible entry. Gaps of the type "Numeric" do not really have a length, but might offer an option to set the step-size probably called "Precision".
  • The process to create the gaps will be constituted of three steps that will always show all gaps that are to be changed/added:
    • A first view allowing to set the type of the gaps.
    • A second view to edit the answer options.
    • A third view presenting the answer options and allowing to assign points to them.
  • Combination of gaps will be added and edited in a separate sub-tab labeled "Gap Combinations". The main form will then be shown in a sub-tab "Properties".
    • The page will contain a `DataTable` showing the current combinations with the selected gaps as a column of the type List, a column of the type List labeled "Combinations Awarding Points" with a list of answer option combinations with the corresponding points (answer options will be truncated to a maximum of 10 characters), and a column available points. Three `SingleAction`s will be available at most: One for adding more combinations of values awarding points labeled "Add Values Awarding Points", one removing combinations of values awarding points labeled "Remove Values Awarding Points" will only be visible if there is more than one combination of values awarding points, and one to remove the combination as a whole labeled "Delete Combination". It is not useful to show an option to edit a combination as removing or adding gaps to a combination leads to everything needing to be reconfigured. A button will be shown at the top of the page to add a gap combination.
    • Adding a combination of gaps will be presented as a `RoundtripModal` with the following two steps:
      • Select the gaps to be added on a form showing all gaps as `Checkbox`es.
      • Add a combination of values and points on a view showing a `Select` input per selected gap and a `Numeric` input for the points .

3 User Interface Modifications

3.1 List of Affected Views

  • All views of `assClozeTestGUI`

3.2 User Interface Details

First view of the creation process allowing to set the general answer form properties. Only a back-link to cancel is shown in the tabs and the answer options will not be saved at the end, avoiding to create broken answer forms. This view is directly accessible through the button "Edit Basic Answer Form Properties" from the entry view of the question. In this case, if the text is not changed the form will be saved and the user will be returned to the entry view. If a gap is removed a confirmation will be shown, if gaps are added the reset of the process will be shown (see next images), but will only show the added gaps.
Second view of the creation process allowing to set the types of the gaps. Only a back-link to cancel is shown in the tabs. To improve orientation the rendered cloze text is shown above the form. The the answer options will not be saved at the end. This view is also directly accessible from the entry view of the question when invoking the "Edit gap(s)"-action on the table showing the gaps.
Third view of the creation process allowing to add the answer options to the gaps. Only a back-link to cancel is shown in the tabs. To improve orientation the rendered cloze text is shown above the form. The answer options will not be saved at the end, avoiding to create broken answer forms.
Fourth view of the creation process allowing to add points to the gaps. Only a back-link to cancel is shown in the tabs. To improve orientation the rendered cloze text is shown above the form. Clicking on "Finalize" will save the answer form.
The entry view of an existing question. Showing first a `StandardPanel` containing the rendered cloze text and the other "Basic Answer Form Properties". A button "Edit Basic Answer Form Properties" leads to a view to edit the properties (see first screenshot). Below a `DataTable` shows the answer options. It contains `StandardAction`s to edit the gaps leading directly to the corresponding views.
Alternative setup for the panel containing the basic answer form properties in a descriptive listing.
View to edit the basic answer form properties. It is identical to the first screenshot, but showing the selected values.
View to edit gap combinations. To reduce complexity, it is only available if the gap combinations are enabled for the question. It shows a button to add a combination opening a modal and a `DataTable` showing existing combinations and allowing to add additional values awarding points to a combination, to remove values awarding points from a combination, if there already is more than one combination, otherwise it is not available, or to delete the whole combination. All these actions will happen in a modal.
First view to add a gap combination allowing to select the gaps to be combined.
First view to add a gap combination allowing to select the values for the gaps and the points awarded for entering this combination correctly.
Modal to remove values from a gap combination. At least one combination of values must be left otherwise an error will be displayed.

3.3 New User Interface Concepts

None for this specific request. See Revision of ILIAS Test Question Creation Process for overarching changes.

3.4 Accessibility Implications

The forms to create questions are based on legacy-ui-forms with a lot of custom inputs that have a huge amount of accessibility issues. By moving these forms to KS-Forms a lot of the issues should be resolved. Additionally it will be easier to address future accessibility issue.

4 Additional Information

4.1 Involved Authorities

If this request is related to multiple components, please list both authorities for all related components.

4.2 Technical Aspects

4.3 Privacy

This feature request doesn't change the data that is collected and also doesn't change how it is presented. It clarifies the storage of answers by users answering questions, by doing so it will improve the understandability and the structure of the data thus making it easier to control the data's lifecycle.

4.4 Security

By moving to KS-Forms and restructuring and clarify the storage of data, the data will be easier to control and user inputs in the KS-Forms will be more consistently validated and sanitized on output.

4.5 Contact

Person to be contacted in case of questions about the feature:  Kergomard, Stephan [skergomard]. For questions about the funding, please contact: Strassner, Denis [dstrassner]

4.6 Funding

Funding status and funding parties are listed in the block 'Status of Feature' in the right column of this page.

If you are interested to give funding for this feature, please get into contact with the person mentioned above as 'Contact'.

5 Discussion

Seegers, Christina [csee], 6. Oktober A question regarding the answer options. It seems to me that it will be possible to use multiline answers or maybe even pictures for the select gap type. Is that correct? And it seems as if the option to set the length of the field for the numerical gap will be removed. Is that correct? Or is that part of some subpage that i missed (they seem to have increased)? If the idea is to remove that option, i would argue that setting a length gives a clear hint to the participant to choose a suitable format for answering and thus would be nice to keep.

Strassner, Denis [dstrassner], 06 OCT 2025, Hi Christina!
Thank you very much for your questions!
It will not be possible to add multiline answers or pictures to the select gap type. This part will not change compared to the actual implementation. 
The numeric gap no longer has a specific length setting, as it should be implemented as a numeric input, that does not have a length. 
Best,
Strassner, Denis [dstrassner]

Dette, Christian [dette], 09 OCT 2025

Thank you very much for the concept and your work. We welcome the unification of the question types. There are some well-known but undesirable behaviors with cloze questions—for instance, issues with LaTeX interpretation, the handling of special characters that are interpreted as HTML, or the automatic removal of spaces that are part of the correct answer. Certain disciplines have managed to "break" this question type rather consistently in the past. If these specific issues are beyond the scope of this project, we are happy to address them separately, but perhaps there is an opportunity to make this area more robust in the future. Are there also plans to change or improve the post-correction functionality for cloze questions?

Kergomard, Stephan [skergomard], 09 OCT 2025: Thank you very much for the comment and question! One of the big problems we try to solve with the revision of questions in general is the inconsistent rendering as we never really know what type of text we are dealing with. This should improve the situation. There is a next step to this process though: The revision does consciously not tackle the presentation of questions to learners (or in previews) as its scope is already big enough. To achieve full consistency this will also need to be addressed. We hope to start working on this soon, though.
The functionality to edit questions in a running test will not be changed for the time being. We will move it to the new forms and we will also improve consistency, but we will not change the changeable elements. We already hope to improve understandability and handling by defining clear levels of editability instead of providing a whole different functionality as it is today. Questions will in the future either be "Fully editable", "Partly editable", or "Not editable".

Killing, Alexander [alex], 9 Oct 2025: Thanks you for this proposal. I have a question regarding the workflow. E.g. the numeric question type will be replaced by a cloze question with a numeric gap, is this correct? So the interaction workflow would be like:

  1. Click create (starting on the screen here Revision of ILIAS Test Question Creation Process)
  2. Enter Title
  3. Click Save
  4. Enter Question
  5. Select Type (Cloze)
  6. Click Save
  7. Click/Add Gap Tag
  8. Click Next
  9. Click Numeric Gap
  10. Click Next
  11. Enter lower/upper Limit
  12. Click Next
  13. Enter Points
  14. Click Finalize

Would this be the workflow for a numeric question?

Strassner, Denis [dstrassner], 09. OCT 2025, I would say you have described the workflow for a question with a cloze answer form and numeric gap correctly. 

Killing, Alexander [alex], 9 Oct 2025: And this will replace the current workflow for creating numeric questions? This is currently:

  1. Click Create
  2. Select Type (Numeric)
  3. Enter Title
  4. Enter Question
  5. Enter Points
  6. Enter lower/upper Limit
  7. Click Save

Strassner, Denis [dstrassner], 10 OCT 2025: Yes.

Killing, Alexander [alex], 10 Oct 2025: There is a report by Ferdinand adressing the usability in the question editing here: Report: Creating Question. It mainly addresses the high number of interactions that are necessary to create questions. It suggests to reduce the number of clicks/interactions and views/page reloads. Did you ask Ferdinand to give some feedback on this proposal? I wonder if we can achieve a better usability by adding more views and more clicks. Is there any worflow that would end up with a lower number of interactions?

Klees, Richard [rklees], 10 Oct 2025: Hi Alex, I don't think that it is actually correct, that the main "high number of interactions" is the core problem according to Ferdinands Report. Have a look into the section "Core Issues". The first issue there is "No flow during question creation". If I understand Ferdinands report correctly, this is not about more or less clicks, but more about where these clicks have to be done, if the next click is in a logical position, or, in his words: "The goal should be to allow a flow state, where the interface is an unobtrusive room for the user's thoughts, rather than causing them to think about the interface itself.". There is more to this than counting clicks.

Do you feel that this quality ("flow") has decreased as well?

Killing, Alexander [alex], 10 Oct 2025: I am not judging here, I just analyse the new workflows and counting steps (like the report does). Regarding the flow of the simple numeric question type, main changes are:

  • 14 instead of 7 interactions
  • The user has to select the "Cloze" type first to create a numeric question. (In other LMS this is an explicit question type)
  • The user has to create a "Gap" ( this was not necessary before)
  • The user has to understand labels like "Gap_g0d4" being a name representing the input (and select a type a second time, making it numeric).

Do we think this increases the quality of the flow? Do we think this will reduce users "thinking about the interface itself"?

Klees, Richard [rklees], 10 Oct 2025: Hi Alex,

I actually can't say what "we" think, because I don't know what you think. It would be interesting if you'd voice an opinion instead of asking suggestive questions.

From my perspective is already is a win w.r.t. "flow" that all questions will use a similar workflow, i I understand correctly. I also feel that having a guided workflow with a consistent way to trigger "Next" step over all creation workflows and steps will be a win too (compared to the current state of the art, where users will need to navigate a complex form with these "underworlds" and make up their workflow themselves).

I don't doubt that there will be room left for improvements, though. But you might be missing the bigger picture by focusing on details of this one question.

Kind regards!

Killing, Alexander [alex], 10 Oct 2025: It is hard to tell anything about the big picture currently, at least for me. There is no question we want to revise the question editing. But getting an idea of the future workflows is difficult, if we don't have a full specification for the most popular question types like MC questions. Ultimatively we should try to have some common criteria for working out alternatives (like nr of user interactions or workflow consistency) and then do some user testing leading to a final decision and implementing the details.

Tödt, Alexandra [atoedt], 2025 Oct, 12th: If using the mark down element, the screen reader currently cannot access the preview. Please reconsider using this element. Accessibility is an important issue but besides this the Report: Creating Question (ILIAS Test & Assessment) lists Cons for using mark down: "Cons: While markdown is popular among programmers, most ILIAS users will most likely never have heard of it. Even if you know markdown, you still have to learn additional syntax, especially for scoring and feedback logic. Maybe some good auto-complete suggestions can nudge new users in the right direction, but some of it will not be self-explanatory. While writing the markdown might feel frictionless, having to hunt for errors later when markdown parsing fails might be a very frustrating experience."

Kergomard, Stephan [skergomard], 13 OCT 2025: The problem with the preview in markdown can be solved very trivially, it is a missing "aria-live='polite'". Parsing simple markdown is a far more managable problem then the one we have today: ensuring valid and secure html from user input.

Rabah, Rachid [rabah], 13 Oct 2025 Thank you very much for this comprehensive proposa. We appreciate the work done! While the technical motivation is clear, some questions arose regarding the consolidation approach:
1. Technical vs. Didactic Abstraction
1.1 The proposal states that Cloze, Long Menu, Numeric, and Text Subset are "actually all Cloze Questions" from a technical perspective. Does technical similarity justify conceptual unification from an instructor's perspective?
1.2 Are these didactically the same question type, or do they serve different pedagogical purposes? For example: Do instructors conceptualize a numeric calculation ("5 × 7 = ?") as a "cloze text" or as a distinct assessment type?
2. Standards and Interoperability
2.1 Other LMS platforms maintain separate question types (Numeric, Short Answer, Fill-in-the-Blank, etc.) despite technical similarities. What advantages does ILIAS gain by diverging from this industry standard pattern?
2.2 How will this consolidation affect import/export functionality, particularly regarding cross-system compatibility and the use of standards like QTI for data exchange?
3. UX Testing and Validation
3.1 Will this concept be tested with actual instructors before implementation?
3.2 Could pilot testing compare task completion times and error rates between current and proposed workflows?
3.3 Are there plans to gather feedback from instructors who frequently create numeric, long menus ...?
4. Conclusion
Thank you again for the detailed concept and the provision of mockups. We certainly recognize the technical advantages for long-term maintainability. However, due to the absence of user testing, we remain uncertain about the impact on user perception and acceptance.

JourFixe, ILIAS [jourfixe], 13 OCT 2025: We highly appreciate this suggestion and accept the feature for trunk.

6 Implementation

Feature has been implemented by {Please add related profile link of this person}

6.1 Description and Screenshots

{ Description of the final implementation and screenshots if possible. }

6.2 Test Cases

Test cases completed at {date} by {user}

  • {Test case number linked to Testrail} : {test case title}

6.3 Privacy

Information in privacy.md of component: updated at {date} by {user} | no change required

6.4 Approval

Approved at {date} by {user}.

Last edited: 13. Oct 2025, 15:11, Kunkel, Matthias [mkunkel]