Show Advanced KnowledgeHide Advanced Knowledge

Randomized Assignments

1 Initial Problem

In the field of project- and competence-oriented teaching, especially where programming is involved, students have to hand in written code examples. In order to prevent copying of other students solutions, students are given individual assignments/exercises. At the moment this is done via ILIAS test-objects since they support the random selection of questions from a question pool. However, the tests configuration settings concerning time limits do not fit due to the longer time needed to complete an exercise (full days/weeks):

1. The assignments should be completed within the semester so a global start and end date is needed.
2. Students should be able to access their submissions even after the semester has ended.
3. Students should only receive the full working time for an assignment if they start their assignment early enough to promote self-organization and responsibility. This requires individual deadlines and in addition: Restrict Relative Deadlines by Latest Submission Date.
4. Students should be able to move the workload of assignments around during the semester:
   1. Start assignment 1 in the first month
   2. Start assignment 2 in the second month and so on and so forth

The availability settings of the test could be used to define the general time period in which the assignment has to be completed, but after the finish time of the availability has been reached the test results cannot be accessed by students anymore. In addition to this problem, the availability is a global feature that cannot be changed without affecting other objects as well. Access settings of tests could be expanded and a relative deadline to finish the test could be implemented. However, this finish time would be connected to the test run as a hole and not a single question and so above number 4) isn't possible with only one test object. Instead multiple test objects (and question pools) would have to be used, even if all assignments/questions have the same workload.

With ILIAS 5.4 the exercise object covers most of what we need, but the assignments/work instructions cannot be randomized, each student has to complete the same assignments (all mandatory assignments contained in the exercise).

2 Conceptual Summary

A new pass mode for exercises is implemented: "Random Selection of Assignments". When selected, students will receive a random selection of the assignments contained in the exercise object as their personal assignments to complete. The number of assignments specified to be randomly selected must be eqal or higher than the number of mandatory assignments of the exercise (same as with the pass mode 'Minimum Number of Assignments').

Peer-review needs to be blocked for this new pass mode, because we need to prevent assignments from being assigned only to one participant. To do so ILIAS can check if assignments with active peer-feedback exists upon activating (saving) the new pass mode setting. In case the random pass mode is active, the peer-feedback option can be disabled in the edit view of assignments.

To keep transparency for participants, they should be informed about how assignments are selected and how many, before the actual selection happens. To solve this a new start screen for exercises is implemented that contains all the required information and allows the participants to trigger the selection of assignments.

3 User Interface Modifications

3.1 List of Affected Views

• exc/settings/edit -> new setting "Random Selection of Assignments" under block Passing the Exercise.
• exc/content/content
• (exc/info -> could be used as a start screen)

3.2 User Interface Details

3.3 New User Interface Concepts

No new user interface concepts.

4 Technical Information

Killing, Alexander [alex], 19 Feb 2019: I think the option "Assignments are enhanced so that they can contain multiple work instructions..." is the one that will add the least amount of complexity to the code. So I think this would be the best option. Instead of introducing a Pool object we might consider to simply add the possibility to copy assignments from one exercise object to another.

5 Contact

• Author of the Request: Sesterhenn, Fabian [sesterhenn]
• Maintainer: Killing, Alexander [alex]
• Implementation of the feature is done by: {The maintainer must add the name of the implementing developer.}

6 Funding

If you are interest in funding this feature, please add your name and institution to this list.

• TH Köln

7 Discussion

Sesterhenn, Fabian [sesterhenn], 08 JAN 2019: Feel free to discuss the request here. We are always looking to improve the core of ILIAS as long as our requests are not too special. For this reason I'm interested to hear if this request would be interesting to more people.

Sesterhenn, Fabian [sesterhenn], 14 JAN 2019: Added another possible solution (conceptual summary).

Sesterhenn, Fabian [sesterhenn], 15 MAR 2019: General revision of concept: New pass mode "Random Selection of Assignments".

Sesterhenn, Fabian [sesterhenn], 19 FEB 2019: Updated description of Initial Problem.

Tödt, Alexandra [atoedt] 2019-04-01: Could you please elaborate why you do not consider making changes to the test thus is covers these requirements. The test seems well equipped to cover a lot.

Specifically this one I do not grasp: "However, the tests configuration settings concerning time limits do not fit due to the longer time needed to complete an exercise (full days/weeks)."

Why not use "Administering the Test: Access" > "Starting Time" and "Finishing Time". This can easily be set to full days and weeks. Could you please point out why this is not sufficient? 

JourFixe, ILIAS [jourfixe], 08 APR 2019: We highly appreciate this suggestion. We prefer option 2.1 "New Pass Mode - Random Selection of Assignments" and ask Fabian Sesterhenn to finish the feature request accordingly. We prefer only to show those assignments to a user that have been selected for this user (and hide the others). Additionally, a start screen that notifies the user that n assignments have been selected for him randomly should be implemented This helps user to understand why there list of assignments might be different from the one of another user. Peer review should not be supported when this pass mode is selected to avoid additional complexity.

JourFixe, ILIAS [jourfixe], 03 JUN 2019 : We highly appreciate this suggestion and schedule the feature for 6.0.

Sesterhenn, Fabian [sesterhenn], 24 SEP 2019:
The current implementation randomizes the number of assignments which are marked to be mandatory for the user, but will always assign all assignments. That latter part wasn't my idea. The idea was to give tutors the option to assign x out of N assignments (where N is the total number of assignments contained in the exercise) randomly. I.e. 5 out of 10. The other 5 assignments should be hidden (not assigned) to the user. This leads to some hassle with the Mandatory flag for the assignments, which I tried to solve in the same way it is done with the pass mode Minimum Number of Assignments.

My mockup from above 'Content View After Random Selection Of Assignments' was meant to show:

• the exercise contains any number of assignments > 6, lets go with N = 10
• 3 of N assignments are marked as mandatory by the tutor
• the tutor selects a number of assignments to be randomly selected, where this number must be equal or greater than 3, because 3 is the number of mandatory assignments
• the user then is given the 3 mandatory assignments (always) and in addition 3 more assigments from the total of 10, showing 6 assignments

That would leave 4 more assignments that are not selected in my example and not shown. This is important, because these assignments then are not accessible for all students.

I like the current implementation in that it solves the conflict between the new pass mode and the mandatory flag for an assignment so elegantly! All that is required now is to hide the assignments that are not selected as mandatory from the user.

Killing, Alexander [alex], 21 Oct 2019: The proposed final behaviour is: The random selection determines the mandatory assignments. Only these mandatory assignments are shown to the user. @JF: Is this behaviour ok?

JourFixe, ILIAS [jourfixe], 21 OCT 2019 : We highly appreciate the suggested behaviour "The random selection determines the mandatory assignments. Only these mandatory assignments are shown to the user." and accept it for this feature request.

8 Implementation

Test Cases

Test cases completed at 2019-12-06 by Tödt, Alexandra [atoedt]

• 32119 : Übung mit mehreren Übungseinheiten anlegen
• 32129 : Übungseinheit mit zufälliger Auswahl einreichen

Approval

Approved at 21 OCT 2019 by Sesterhenn, Fabian [sesterhenn].