Show Advanced KnowledgeHide Advanced Knowledge

Straightforward Configuration of Result Presentation

1 Requirements

1.1 Status in 5.0

This chapter focuses on the settings for the views „Table of Detailed Test Results“ and „'List of Answers' for Printing“ in Test > Settings > Scoring and Results.
These two settings compete: Settings labeled as specific views carry sub-settings that affect other views. The two views treated in this chapter are redundant in certain aspects. It is very confusing. This section outlines the issue using screenshots.

• Checking „Access to Test Results“ provides users with access to test results at the selected point of time.
• Checking „Show 'Table of Detailed Test Results' for each Test Pass upon Selected Point in Time“ provides users with a table with additional information on each test Passes in the test results.

• Checking „Scored Answers of Participant“ will add a link to the rated answers to each question in the 'Table of Detailed Test Results' but it is very far away from the 'Table of Detailed Test Results'-setting
• Checking the sub-setting „Best Solution“ adds the best possible solution to the same screen
• Checking „Additional 'List of Answers' for Printing“ has unintelligible effects:
• A new button, „List of Answers“ is shown in the participant’s info tab.
• The content of the table 'Table of Detailed Test Results' gets changed despite the respective setting has not been touched: 'Table of Detailed Test Results' suddenly includes the information from the 'List of Answers'. This information is appended under the 'Table of Detailed Test Results' (see Mantis #14275).
• As of 5.0, participants can see the best results compared to their own answers in two different views: Either answer per answer ("Scored Answers of Participant" > „Best Solution"), or in one single list ("Additional 'List of Answers' for Printing" > „Show Best Solution in List of Answers").
• Checking the sub-setting „Show Best Solution in ‚List of Answers'“ (47b) does not affect the ‚List of Answers‘ accessible on the info tab, but ONLY affects the 'Table of Detailed Test Results' (see Mantis #15252).

These two views („Table of Detailed Test Results“ and „'List of Answers' for Printing“) compete and provide similar functionality.
In addition, we observe a certain functional similarity with the setting „Overview of Answers Given“: Just like the „'List of Answers' for Printing“, this overview shouldn’t provide any information on test results, but rather serve as a review (for students) or documentation (for institutions).
 
 

1.2 Underlying User Requirements

Summing up, we can discern three different requirements supposed to be met by the settings mentioned above:

1. Review and Archive: ILIAS provides a view of high-stakes exams that includes the answers given by a participant. This view serves as a review for participants before they submit a test. After submission, it can be printed and signed to be archived as a document by the institution administering the high-stakes exam.
2. Communication of Test Results: Like a bulletin board, ILIAS informs students about their passed status, mark and ranking.
3. Review and Learn from Mistakes: In addition to the basic communication of results, ILIAS helps participants to review their test and deepen their understanding of the materials by providing access to scored answers, feedback, a comparison to the best possible solutions, and additional content for recapitulation. This view needs to be easily accessible.

In our solution, each of the views and the settings that control them should be awarded a clear purpose, so that each of them really adds value. They should be clearly labeled and not influence each other.
 

1.3 Suggested Solution

Review and Archive: Extending „Overview of Answers Given“ in ‚General Settings‘

We suggest meeting the requirement review and archive by extracting the settings providing the respective functionality from the ‚Scoring and Results‘-tab.
They should be added to the ‚General Settings‘-tab, extending the functionality of the currently existing setting, „Overview of Answers Given“ (under ‚Settings‘ > Finishing the Test) in a meaningful way.
Crucially, results, best solutions et cetera should not be revealed in this view to prevent mistakes in choosing options.
 

47b (Show best solution in list of answers) is intentionally not included, as this view should never provide results. It is provided with the test results, though!
 

Communication of Test Results & Review and Learn from Mistakes: Tweaking „Access to Test Results“ in ‚Scoring and Results‘

If access to test results is granted, then the ‚Results’-tab is displayed to participants showing and opens with the "Test Results" sub-tab initially. The "Test Results" sub-tab shows the well known pass overview.

If access to detailed test results is granted, then the "Review of Scored Answers" sub-tab is shown as well. The participant can switch to this sub-tab  to get detailed information about the scored pass (detailed information on non scored passes won't be available anymore).

This allows for the scenario of  Berne: quick comparison of own answers with best solutions, accessible with one click immediately after finishing a test pass.

The advantage of our solution would not only be greater clarity, but the possibility to adjust this view as needed, providing more (or less) information.

The setting „Item Analysis Export for Pure Single Choice Tests“ [51] does not belong to the controlling of result presentation views, so it should not get involved in the new form section "Test Results". It is to be moved to the section "Misc Options" of the scoring and result settings screen.
 

2 Additional Information

• Idea / concept: Alexandra Tödt
• Funding: Required 
• Maintainer: Björn Heyser
• Implementation of the feature is done by (company, developer)
• Tested by / status: (name, e-mail), (status information set after implementation)

3 Discussion

BH 13 April 2015: The maintainer supports and strongly recommends to accept this request.

JourFixe, ILIAS [jourfixe], July 13, 2015: We support the general request and schedule it for 5.1.

• "Link to Review of Answers": The info text of the first checkbox should add where this feature appears: "... on the "Info" screen and at the end of a test."
• "Archive Answers": The info text should state who is able to access the archives: "... by a test administrator."
• "Access to Summary Test Results":The info text of the first checkbox should add where this feature appears: "... on the "Info" screen and at the end of a test."
• "Access to Summary Test Results" Form: The problem is that this form may use three indented subforms (1: checkbox on top, 2: "Access to Detailed Test Results" > 3: "Include Scored Answers" > 4: "Single list for...". ILIAS does not support this currently (only one level of subforms is "officially supported"). Maybe the kitche sink group can give an advice. Maybe we give checkboxes the possibility to hide/show not only its own subforms, but also other parts of the form, too. But this may require lots of tests (the same is true for three-level-subforms). Please make a suggestion before implementing this.

4 Implementation

...