Show Advanced KnowledgeHide Advanced Knowledge

Is-dirty Mode for Tests

1 Requirements

The project "Spearating submission and navigation" was a first step towards better data upon quality of test questions: we want better statistics. 
It allowed ILIAS to understand whether a question was answered and saved or just looked at and leaved-over. 

This suggestion aims at retting rid of a lot of buttons and ensure, that a question understands whether or not it was touched and tempered with. 

A question should know it's original state and remember it. When a participants clicks on Next, the question compares its orginal state and its current state.

• If the states are the same the question will know it was not answered but leaved-over and only looked at.  
• If the states differ the question will know it was answered and save the answer. 

This would rid us all of the buttons "Save Answer and Next" and "Do not answer and Next". 

To arrive there we have to ensure that Ordering questions are never initially displayed in the correct order. Additionally we have to ensure that the issue of Multiple Choice Questions whose correct answer is to leave all answers uncheked are somehow transfered to managable version. 

We have to either abandon Flash and Javaapplet question or find a way to deal with them. 

Then we have a smart next button, that knows if it triggers navigation-only or navigation-and-save AND we can start thinking about improving the quality of our test questions based on empirical data as opposed to hunches. 

2 Additional Information

• Idea / concept: (please add your name when creating this feature request)
• Interest in funding: (please indicate if you are interested/able to fund this feature)
• Maintainer: (will be set by Jour Fixe / maintainer)
• Implementation of the feature is done by (will be set by Jour Fixe / maintainer)
• Testcases by: (please add your name if you want to create the testcases for this feature)

3 Discussion

Neumann, Fred [fneumann], May 30, 2016:
This posting overlapped Improved Separating Submission and Navigation. That request describes the “is-dirty” recognition and its consequence for the test navigation in detail. The main difference here is the way how ordering questions and MCs with a correct empty solution are treated.

I strongly disagree to prevent empty MC questions being correct or prevent the correct order being displayed initially. This will reduce the space of possible answers (thus affecting statistics) and can lead unfair conditions for test participants when choices are shuffled in an online exam. As an example, consider the following question which is, of course, a constructed case for demonstration:

“Please order the following planets by their volume (increasing): Jupiter, Uranus, Neptune”.

Most people will know that Jupiter is the biggest planet. But do you know that Uranus is slightly bigger but having less mass than Neptune? Now let’s assume that choices are shuffled and students know that the initially presented solution is never correct.

• Student A gets the presentation “Uranus, Neptune, Jupiter”. He knows that this solution is wrong but Jupiter is on the correct position. So the correct solution must be “Neptune, Uranus, Jupiter”.
• Student B get the presentation “Jupiter, Uranus, Neptune”. He knows that this solution is wrong for two reasons, but that does not help him to get Uranus and Neptune in the correct order.

The request Improved Separating Submission and Navigation handles that case by adding a checkbox to that question types that allows a student to take the presented solution as his own answer,  without the need to modify it.

Regarding Flash and Java applet question I believe there is a possibility to apply a dirty recognition. But even if that is not possible, I guess these questions do not produce records in ‘tst_solutions’ if they are not answered. So we can at least distinct the cases “not answered” and “answered".

Kiegel, Colin [kiegel], 2016-05-30: The problem Fred Neuman describes could be circumvented as follows:

• If the states differ the question will know it was answered and save the answer. 
• If the states are the same
  • AND the state is INVALID (e.g. single choice without selection), the question will know it was not answered but leaved-over and only looked at.
  • AND the state is VALID (e.g. multiple choice without selection), the user will be prompted whether he wants to save this answer despite any changes

This could be a compromise with smart behaviour in most cases and fallback to prompt the user only in uncertain cases. However a free navigation in the test (without confirmations) is not possible for all question types.

Neumann, Fred [fneumann], May 31,2016:
Hi Colin, your last point „AND the state is VALID (e.g. multiple choice without selection), the user will be prompted whether he wants to save this answer despite any changes” would mean that for each MC and ordering question a conformation box will appear when the student just “scans” the questions in his exam.
Please have a look to the new prototype at http://www.labs.odl.org/ilias-testnav51 which implements the behavior described at Improved_Separating_Submission_and_Navigation. MC and ordering questions got an additional checkbox “use unchanged answer” that is only shown when no answer exists. When a student checks it for an unchanged answer then this default will be taken as his answer. It will be disabled as soon as the student changes the answer.
That implementation has three benefits:

• Confirmation popups are completely avoided when questions are just viewed.
• Answering a question is consistent for all question types. A student answers a question by doing something “inside” the question either by checking choices or checking the “use unchanged answer” option. The question status is immediately set to “edited” which is shown for the student.
• Navigation and saving are consistently handled for all question types. An untouched question will not be saved. An edited question will be saved after a confirmation that can be suppressed with “do not show again”.

In think that is a very explicit and easy to understand behavior and it comes rather near to what we had in 5.0. But it still allows separating answered and not answered questions clearly.

Kergomard, Stephan [skergomard], 2022 Mar 9: We close this as part of the cleaning up effort undertaken in the Splitting-Up Test & Assessment Workinggroup.

4 Implementation

{please give a description of the final implementation and add screenshots if possible}

Test Cases

Test cases completed at {date} by {user}

• {Test case number linked to Testrail} : {test case title}

Approval

Approved at {date} by {user}.