Extended post-correction with asynchronous recalculation in tests

If you need any help in filling out this wiki page, please visit our ILIAS Community FAQ. And please complete the metadata information in the right column after having created the page.

1 Initial Problem

When managing tests, especially after they have been completed, teachers often face the challenge of having to adjust the assessment. Reasons for this may include incorrect or ambiguous questions that require a fair reassessment.
The current mechanisms in ILIAS have two major limitations in this regard:
1. Inefficient recalculation: Any change to the evaluation of a question triggers a synchronous recalculation of all test results. For tests with a large number of participants and/or questions, this leads to long waiting times, system overload, or even timeouts. This makes it impossible to work smoothly.
2. Limited grading options: There is a lack of flexible and differentiated options for re-grading a question for all participants across the board. For example, completely removing a question from the grading or awarding points for a question across the board is cumbersome or even impossible. These limitations make post-correction a time-consuming and error-prone process.

2 Conceptual Summary

The implementation of an extended post-correction functionality is proposed, which solves two core problems:
Asynchronous recalculation: The recalculation of test results should take place in the background (asynchronously). Teachers can make adjustments and receive a notification once the process is complete. The system remains responsive during the calculation, which massively improves usability and performance, especially in tests with large amounts of data (participants and questions).
Extended evaluation modes: There should be a dedicated interface for post-correction that provides teachers with tools to adjust the scoring of a single question for all participants.

The following modes should be available:
Remove question from scoring: The question is completely neutralized. The maximum achievable score for the question and the points achieved by the participants for this question are set to 0.
Assign fixed score to all: All participants are assigned a defined score for the question. The maximum achievable score for the question is also set to this value.
Count question as bonus task: The maximum achievable score for the question is set to 0, but the points achieved individually by the participants are retained. This makes it possible to award bonus points without increasing the total score for the test.
This combination of asynchronous processing and flexible modes would fundamentally improve the post-correction process.

2.1 Use Cases

Incorrect question: One question in the test was incorrect in terms of content. Instructors remove the question from the scoring for all participants to ensure fairness. The recalculation for 300 participants runs in the background while they continue working.
Flat-rate scoring: One question was worded ambiguously and participants interpreted and answered it differently. To adjust the results, teachers decide to give all participants full marks for this question.
Bonus question: A question is to be retroactively counted as a bonus task because it was unexpectedly difficult. Instructors use the corresponding option so that the points earned are included as bonus points in the overall score without affecting the maximum achievable score (and thus the passing grade).

2.2 Details of the procedure

Procedure (from the perspective of the evaluator)
1. The evaluator navigates to a completed test. There they will find a tab labeled “Post-correction.”
2. The post-correction view displays a list of all test questions.
3. Clicking on the question title opens a page or a modal for the selected question. Here, in addition to the existing question-specific options (e.g., renomination of answers in the cloze question), the new evaluation rules can be defined.
4. The interface offers the following options (e.g., as radio buttons):
- No change: (default)
- Remove question from scoring: Sets the maximum achievable score for the question and the scores of all participants for this question to 0.
- Assign fixed score: An input field appears in which the new score (e.g., “5”) is entered. This score is set as achieved for all participants and the maximum achievable score for the question is adjusted accordingly.
- Evaluate as bonus task: Sets the maximum achievable score for the question to 0, but retains the points achieved by the participants so far.
5. After selecting and confirming the action, a “Recalculate” button appears in the question overview.
6. Clicking on the button starts the recalculation in the background.
7. The evaluator can continue to use ILIAS. Once the process is complete, they will be informed of this in the post-correction view.

2.3 Mockup

Not currently included. Can be created if required.

3 User Interface Modifications

3.1 List of Affected Views

3.2 User Interface Details

{ For each of these views please list all user interface elements that should be modified, added or removed. Please provide the textual appearance of the UI elements and their interactive behaviour. }

3.3 New User Interface Concepts

{ If the proposal introduces any completely new user interface elements, you might consult UI Kitchen Sink in order to find the necessary information to propose new UI-Concepts. Note that any maintainer might gladly assist you with this. }

3.4 Accessibility Implications

{ If the proposal contains potential accessibility issues that are neither covered by existing UI components nor clarified by guidelines, please list them here. For every potential issue please either propose a solution or write down a short risk assessment about potential fallout if there would be no solution for the issue. }

4 Additional Information

4.1 Involved Authorities

If this request is related to multiple components, please list both authorities for all related components.

4.2 Technical Aspects

{ Necessary technical information have to be provided here, e.g. dependencies on other ILIAS components, necessary modifications in general services/architecture, potential security or performance issues. }

4.3 Privacy

{ Personal data that will need to be stored or processed to implement this feature have to be listed here. For each date give a short explanation why it is necessary to use that date. }

4.4 Security

{ Does the feature include any special security relevant changes, e.g. the introducion of new endpoints or other new possible attack vectors. If yes, please explain these implications and include a commitment to deliver a written security concept as part of the feature development. This concept will need an additional approvement by the JourFixe. }

4.5 Contact

Person to be contacted in case of questions about the feature or for funding offers: Holger Markus holger.markus@uni-goettingen.de, Andreas Zahn andreas.zahn@uni-goettingen.de

4.6 Funding

Funding status and funding parties are listed in the block 'Status of Feature' in the right column of this page.

If you are interested to give funding for this feature, please get into contact with the person mentioned above as 'Contact'.

5 Discussion

6 Implementation

Feature has been implemented by {Please add related profile link of this person}

6.1 Description and Screenshots

{ Description of the final implementation and screenshots if possible. }

6.2 Test Cases

Test cases completed at {date} by {user}

6.3 Privacy

Information in privacy.md of component: updated at {date} by {user} | no change required

6.4 Approval

Approved at {date} by {user}.