Feature Wiki

Information about planned and released features

Tabs

Extract Test Results from the Participant tab and make a separate Tab of it

Page Overview

[Hide]

1 Initial Problem

1.1 Tabs

Test results are scattered accross the test object.  In ILIAS 5.0 the results of a test are presented

  • to participants in the ‘Info’-tab:  A button leads participants to view their results at some point in time. It is very awkward place: Something very important is ill-labeled and hidden. Participants will look for something called ‚Results‘.
  • to test administrators in the ‚Participants‘ tab: Test administrators look up the results in the tab they use for administrating participants. It is similarly awkward to present the results to the test administrator in the ‚Participant‘-tab. They too will be looking for something labeled ‚Results‘.
  • there is a prominent ‚Statistic‘-tab that offers a rudimentary item-statistic. This tab’s label seems to indicate something like results, but this is not the case. This tab is presenting very simple quantitative data on how the questions performed / on the quality of questions.

1.2 Permissions

  • A dedicated permission ‘Statistics’ that is used to control access to the tab ‘Statistics’.
  • Access to the test results of participants in tab ‘Participants’ tab is controlled by ‘Edit Settings’. 
The does not reflect the real world division of labour:
We often see a clear distinction in responsibilities between those persons involved in creating and editing tests and those who need information about the results and performance of their learners.
The second group should not be allowed to change settings of the test while the first group does not neccessarily need to get access to the personal data of learners.

2 Conceptual Summary

2.1 Tabs

  1. A dedicated ‚Results‘-tab should be created communicating results to participants and test administrators. ‘Results’-tab should display
    • individual results to an individual participant and
    • overview of the results of all participants to administrative personnel.
  2. The ‚Participant‘-tab is only displayed if the option 'Manually Selected Participants' is checked. This tab then is dedicated to monitoring participants in an ongoing e-exam. [ See JF decision! ]
  3. The ‚Statistics‘-tab presents information about the quality of test questions. The sub-tab ‚Evaluation of all Users‘ plainly dies.

2.2 Permissions

  • We abandon the permission ‘Statistics’ and introduce a new permission ‘Access to Results’. [ See JF decision! ]
  • The ‘Access to Results’-permission controls access to the new tab ‘Results’ as well as to the old tab 'Statistics'. 
  • 'Edit Settings' will not longer give access to results. 
When migrating to 5.2, all roles with ‘Edit Settings’ (test administrators) and 'Read'  (participants) get ‘Access to Results’ as well.

3 User Interface Modifications

3.1 List of Affected Views

  • Tab 'Results' → NEW
  • Tab 'Participants'
  • Tab 'Statistic'
  • Tab 'Permissions'

3.2 User Interface Details

3.2.1 Results Tab

This tab should bear the following sub-tabs: 

Test Result: This sub-tab comprises information on passed / failed status and mark.  Participants are presented with their own test result only. Test administrators are presented with a list of all participants that completed the test.

Results and Grade for individual participant
Test Administrator with individual results as well

Sub-tab Competence Results: This sub-tab presents the assigned competence levels of a participant derived from his or her test results. Since there is no view for administrators showing the competence results of other participants with the current implemenation, this sub-tab will not behave in a different way for administrators and participants, it simply shows the own competence results.

Competence results for individual participant
Competence Results for testadministrator

Sub-tab Ranking:  This sub-tab shows individual participants their own ranking. The two tables "By Score" and "By Working Time" will be placed underneath each other.

Sub-tab Review of Scored Answers: Here the information on own answers, best answers, feedback and content for recapitulation relating to the scored pass should be presented. This detailed information will not be available for non scored passes any longer. Again: Participants get their personal data while test administrators get an overview over all participants.

3.2.2 Participants Tab

  • Tab is dispalyed only if 'Manually Selected Participants' is activated in Settings-tab. [ See JF decision! ]
  • Tab and table heading should be labeled "Exam Dashboard" to clearly communicate it's purpose (thanks Fred for pointing this out). [ See JF decision! ]
  • Action 'Show Test Results' is removed. 
  • Multi-Actions are reduced to 'Save' and 'Remove as Fixed Participant' (should be named 'Remove as Selected Participant')

3.2.3 Statistics Tab

  • 'Evaluation of all Users' is removed . The information shown there is also shown in the new 'Results'- tab where it actually belongs to.
  • The results export that was available in 'Evaluation of all Users' is provided in the Export tab, too. No loss. 
  • The only infomation that is available on this tab only is the information 'Mark of Median'. This info is moved to the ranking sub-tab: the mark assigned to the median participant is to be displayd next to the mark granted to an individual participant.

3.2.4 Permissions Tab

  • Permission 'Statistics' is abandonned. [ See JF decision! ]
  • Permission 'Access Results' is introduced.

3.3 New User Interface Concepts

none

4 Technical Information

No technical aspects need to be considered.

5 Contact

  • Idea / concept: Alexandra Tödt
  • Maintainer: Björn Heyser
  • Implementation of the feature is done by (company, developer)

6 Funding

If you are interest in funding this feature, please add your name and institution to this list.

7 Discussion

Matthias Kunkel, 28 Feb 2015: This is a very good suggestion. I wonder why we haven't changed it yet. We already have a concept for "Statistics" in ILIAS. But the "Statistics" tab in the T&A is following another idea. A tab called "Statistics" should show the same kind of information in the entire system. Streamlining it is very important. But I am not sure about your suggestion "Quality of Test". It promises something we do not offer today - a screen with detailed and qualified information about the quality of this test. IMHO we just can make quantitative statements.

AT, 2015-03-09: the quality of a test is evaluated by quanitative information: The number of times a distrator is selected is the very sound information about it's quality as a distractor.  So one would expect quantitative information in a tab called "Qaulity of Test".
I talked this over with the maintainer and am aware of community members discussion how to improve the availability of this kind of information on test quality.

BH 13 April 2015: The maintainer strongly supports this request and was similarly overtaken by this great idea (like Mattias).

I have to second Alexandra regarding the quality of test discussion, quantitative answering information is the base for test quality statements. Sure, we do not provide as much or detailed aggregations as we could, but the existing minimum represents a minimal possibility to derive test quality statements.

I am sure, we will extend this quantitative information, so the precision of the derivable quality statements gets increased.

JourFixe, ILIAS [jourfixe], 20 July 2015: We highly appreciate this improvements and schedule it for 5.1.

Kunkel, Matthias [mkunkel], September 03, 2015: The feature didn't made it into 5.1 due to missing time. But there is still a strong need for this revision. Therefore, I hope we get funding for it. I have added a suggestion for the permission handling for this new tab which should be considered when scheduling the feature again.

Slotosch, Sven [sven.slotosch], 07.03.2016: After discussion the feature request at the SIG-ea-Meeting at Bremen, we (Heads of SIG-ea) appreciate the implementation of the feature. Possible interferences with the "Item-statistics-Plugin" developed by Fred Neumann should be discussed with Fred.

JourFixe, ILIAS [jourfixe], May 09, 2016: We highly appreciate this feature request and schedule it for 5.2. The new "Settings" sub-tab for administrating the invited test participants should be named "Invitations".

Neumann, Fred [fneumann], March 20, 2017: I apprechate the request in general, but I see problems with the following proposal:

The ‚Participant‘-tab is only needed to actually manage participants in case the option „Manually Selected Participants“ is checked. This tab then is dedicated to actually managing participants. This dedicated ‚Participant‘-tab should be displayed not as a major tab but as a new sub-tab / link in the ‚Settings‘-tab.

In E-Exams we use this tab to monitor a running test in real-time: to see which participants already started and when their last access happened. This allows us to see if everything works  as planned when a test is simultaneously written in different rooms and buildings.

The tab provides also the functions to set an extra time for single participants, to delete the data of participants and (since 5.2) to finish the running passes of all users. All of these functions have an administrative character and they don’t require a view of the actual answers, reached scores or passed/failed status. I propose to keep this tab and not to move it below the “Settings”, as is has a purpose on its own.

Regarding access control and privacy, I see a potential need to distinct between the permission to access individual data (‘Results’ tab) and aggregated data (‘Statistics/Quality’ tab).

Neumann, Fred [fneumann], March 27, 2017: Thanks for leaving the "Participants" list on the top level. But I still have a little problem. I don't know how other institutions organize their exam participants but we just add them by import as members to closed groups that contain the tests. So we don't set "manually selected participants"  and this would hide the "Participants" tab.

JourFixe, ILIAS [jourfixe], March 27, 2017: We highly appreciate this suggestion and schedule it for 5.3. We would like to have the following changes:

  • The tab 'Participants' shall be named 'Dashboard' (not only used for exams) and can be triggered not only by 'Manually selected participants' but also in case of allowing additonal time for test participants.
  • We also want to have the action 'Finish Test' on the 'Results' tab.
  • We keep the permission 'Statistics' to have the option for giving access only to anonymised test statistics.

JourFixe, ILIAS [jourfixe], 18 DEC 2017: We schedule this feature for 5.4 and hope it will make it into this release. No changes required compared to last JF decision.

8 Implementation

Any access to the own test results has been removed from the info screen, the only remaining button is about starting/resuming the test. The Participant tab that handled the access to the participant's test results as well as manually selected participant at one place has been removed.

8.1 Tab: Results (for Participants)

As long as participants do not have any test results or when the access to test results is not yet granted depending on the setting for score reporting no results tab is shown at all.

Participants without Access to Test Results / having no Test Results

When the access to test results is enabled the results tab is available for participants with existing test results. Depending on the test configuration it comes with the following subtabs:

8.1.1 Subtab: Test Results

This subtab provides the test results of the participant that were formaly availale behind the Show Test Results button on the info screen. This screen shows the list of all passes the participant has been done. The pass details are accessible behind a link that opens a tab-less underworld with a back link in the tab bar.

Test Results for Participants

8.1.2 Subtab: Competence Results

This subtab provides the competence results of the participant that were formaly availale behind the Show Competence Results button on the info screen.

Competence Results for Participants

8.1.3 Subtab: Ranking

This subtab provides the ranking information for the participant that were formaly availale behind the Show Ranking button on the info screen.

The Ranking for Participants

8.1.4 Subtab: Review of Scored Answers

This subtab provides the ranking information for the participant that were formaly availale behind the Review of Scored Answers button on the info screen. This screen shows the list of all passes the participant has been done. The pass details are accessible behind a link that opens a tab-less underworld with a back link in the tab bar.

Review of Scored Answers for Participants

8.2 Tab: Results (for Admins)

For test admins with the RBAC permission Test Results the results tab shows up all existing test participants that have been taken the test. The RBAC permission Edit Settings does not longer grants access to the test results for all participants.

The screen provides the functionality for finishing test passes and the deletion of test results as well as the access to test results. When opening the test results for any participant they are shown in a tab-less underworld with a back link in the tab bar. 

All Participipant's Test Results for Admins

When the admin has taken the test himself his own test results are presented within the results tab as well. The results for all participant's test results is than accessible within a subtab called All Participants.

Results Tab for Admins with own Test Results
Results Tab for Admins with own Test Results

8.3 Tab: Dashboard

The Dashboard tab comes with two subtabs that are available depending on the test configuration. The permission Edit Settings is required to have access to the Dashboard tab.

8.3.1 Subtab: Manually Selected Participants

The subtab Manually Selected Participants provides the functionality for managing manually selected participants. This subtab only shows up when the corresponding setting was activated in the configuration. When the setting is not activated a subtab participants is available, that shows the very same information without any interaction related to handle participant invitations.

Handling of Manually Selected Participants

8.3.2 Subtab: Extra Time

The subtab Extra Time provides teh functionality for granting additional time for participants that just take the test. It was formally available in the Participants tab that has been removed. This subtab only shows up when the test is restricted in time and limited to one pass only.

Handling of Extra Time for Single Participants

Test Cases

Test cases completed at 05 April 2018 by Heyser, Björn [bheyser]

Existing and changed Cases:

Newly introduced cases:

Approval

Approved at 2018-05-07 by Samoila, Oliver [oliver.samoila].

Last edited: 2. Apr 2019, 20:30, Samoila, Oliver [oliver.samoila]