Show Advanced KnowledgeHide Advanced Knowledge

Scoring Revision

1 Description

We must clearly define how the scoring in tests is working, especially the scoring of questions that have been submitted, postponed, not presented at all.
 
The following remarks reflects the behaviour as we (the core team) currently understand it (see also Mantis bug 8450).

• Users can postpone questions. Currently when a question is "postponed" the answer is saved (and scored) and the answer is put to the end of the test. Unfortunately, the order of questions on the result screen is changed accordingly so that comparision of test answers between two users can be difficult (due to changed order of questions).
• Questions are considered as being submitted, when they have been displayed and the user hits at least one button on the question page (next question, postpone question, list of questions, ...)
• If a question is postponed (via postpone button), the answer is saved and scored anyway.
• Currently you can give points per answer (checked/unchecked). As soon as the question has been displayed to the user and a button has been pressed by the user (next quesiton, postpone, list of questions, ...) the question is scored. Only questions that have not been displayed are (currently) scored with 0 points.
• We agree that the behaviour (questions are scored as soon as one button on the question page has been pressed), could be discussed.

One problem of the current presentation of test results is that it is not indicating if a question has not been answered or not been presented. In both cases ILIAS indicates the question as not been answered and shows a red x for each possible answer (while the page "Detailed Evaluation of..." in "Statistics" is showing this information for every question: « The question was not answered ».

2 Status

• Scheduled for: Not scheduled yet
• Funding: Required / Partly funded by / Funded by ...
• Development: Feature is to be developed by

3 Additional Information

• If you want to know more about this feature, its implementation or funding, please contact: your name / your e-mail

4 Discussion

MB, 01 Jun 2012: In my opinion, the test should follow some basic rules which I think reflect a common understanding of a test:

1. Questions in a pass that was started should be rated, regardless if they were presented or not, postponed or intentionally answered. Whenever a pass ends, for whatever reason, the actual result should matter. Should the test be ended prematurely, due to technical issues or other disasters, it should be up to the user to call for relief and up to the tutor to decide on a case by case basis. An accurate detection if a question was "presented" or not is almost impossible. I.e. hardware could fail and so far we have no means to detect a burning screen with the browser as runtime.
2. The number of questions answered should be determined by the existence of saved solutions (in table tst_solutions) , which are (to be) created when the user navigates within the test after the particular question was sent. There is no way to determine the intention of a user clicking the  "next question" button.
3. Should the test results be called with not all answers saved - for whatever reason - the unanswered questions must still be evaluated due to the possibility of non-cutted negative scores.

BH 25 September 2014:
 
The problems discussed on this page are all solved by the feature Separating Submission and Navigation allready scheduled for 4.4. We do not need this discussion, unless we want to abandon this scheduled feature.

5 Follow-Up