Show Advanced KnowledgeHide Advanced Knowledge

Different test parts for randomized tests

1 Requirements

We would appreciate the following feature for a randomized test situation:
There should be the possibility to create different randomized test parts in only one "global" test. The order of the different test parts is fixed for the "global" test, the order of the questions for one test part is randomized.
 
Current test situation:
The students have to be tested in several topics. We create one randomized test for each topic, so the students have to work on several randomized tests.

• Students first have to finish one test (=topic) to work on another test (=topic). If they finish one test (=topic), they won’t be able to change answers in this test.
• We could also create only one randomized test with all topics and questions: this is not very comfortable, students risk to get "lost in the mass of questions/topics".

Advantages:

• Clearly represented test parts: One test part = one topic; different topics and their questions can be shown in one test; the questions of one topic are presented randomized within one test part but not mixed up with the questions of other topics.
• All questions are always accessible: Students can work on one topic, than on another. But they can always jump back/forwards to another test part without leaving/finishing the "global" test.
• Easier test administration: There is only one test to administrate, not two or more tests.
• Easier test reporting: You don’t have to calculate with different scores of different tests to get one "global" result for each student.
• Better navigation: Students can navigate through the test parts, topics and questions very easily.
• Possibility to create a test with randomized AND fixed questions, e.g.: You can fix some difficult/important questions as a test part at the beginning of the test to guarantee that each student has to work on it.

2 Status

• Scheduled for: Not scheduled yet
• Funding: Required / Partly funded by / Funded by ...
• Development: Feature is to be developed by

3 Additional Information

• If you want to know more about this feature, its implementation or funding, please contact: Christine Berggold / christine.berggold@uni-hohenheim.de

4 Discussion

5 Implementation

Florian Suittenpointner, 9 Dec 2012:

• Actually, users can already now leave one test unfinished, jump to another one and return to the first test later.So, this seems to be a rather weak argument for this suggestion.
• Also the argument "only one test to administrate, not two or more tests" seems questionable when at the same time you want to have different settings for different test parts ("randomized AND fixed questions .... fix some difficult/important questions as a test part at the beginning of the test"):This would require that every test part has its own settings, and so, in the end you would have to check the settings of every test part separately.

However, I also have often been asked by customers whether there are plans for creating such "modular" tests.
Therefore, I would like to make an alternative suggestion:
Instead of "test parts", you could introduce "test groupings":

• A test grouping can aggregate results from different tests.
• It can overwrite settings in all tests belonging to it (technically spoken: it is a setting template that can be applied to all of them).
• It can offer a "List of Tests", analogously to the "List of Questions" in an individual test object. This list can be called when ever you process one of the tests belonging to the grouping.

If you make even one more step back, such groupings could be interesting for other object types, too.
By now, the new repository object "Item Group" is only used for sorting but it could be developped in a direction like the one discussed here.

Christine Berggold, 10. Dec 2012:
I understand the argument of test administration; it still remains difficult to administrate all “test parts” and settings together.
 
I think to leave a test and to return to it, when the maximum duration of a test is fixed, is more complex: if you stop test 1 to open test 2, time keeps running for test 1. But you don’t see the remaining time of test 1 while working on test 2, so you don’t know your entire remaining time for all tests…
 
I also support your suggestion of “test grouping”. This would be really great and helpful for the test authors and the aggregation of results!
 
But I’m often asked by our examiners, if there is a possibility to combine fixed questions AND randomized questions in only one test. The argument is, to fix some difficult/important questions for ALL students and to randomize the rest of the test. Could you imagine a solution for this idea?

Florian Suittenpointner, 10 Dec 2012:
The fixed/randomized topic fits well into my vision of test groupings because you can have different settings in the tests in the grouping (the grouping only offers the option to manage the settings centrally).
The problem is the maximum duration topic you mention. I think, it's no good usability, anyway, that the max time is running out even if you "suspend" a test.
This is not what I expect when I read "suspend".
If this was fixed (in the framework of the big, big test revision that is desired by the core team since years), this problem would be eliminated, too.
 
However, if you need a quick solution, here's a workaround: Take a question pool A for the difficult/important questions and pools B, C, D for the rest. Then let the test draw as many question as there are from pool A (so they will all be drawn) but only subsets of the questions in pools B, C, D.

Christine Berggold, 11 Dec 2012:
Thank you for the workaround, I think this will help us for some test situations!

Are there already articles in the Feature Wiki concerning your idea of "test groupings" and the solution for the "maximum duration/suspending a test" problem? Or are these ideas fixed as parts of other features in the wiki? We would be very interested in the development of these features and perhaps we could also take part in the funding...

JF 18 Feb 2013: Please get in contact with our test&assessment maintainer Björn Heyser, bheyser (at) databay.de to discuss the concept and a possible implementation.

Florian Suittenpointner, 21 Feb 2013:
I have gathered my ideas concerning this feature in a graphical overview:

Fred Neumann, 28 March 2013:
I see a justification for both approaches. The test groupings of Florian would fit for a scenario at our university: three tests are written at three times in a semster, and the final result is a combination of their result. The test parts proposed by Christine are somehow related to the feature request Several questions on one test page. This feature would need an introduction of test parts, too. These could get their questions by a fixed or ramdom selection.

Christine Berggold, 2 April 2013:
I think the test groupings of Florian are helpful concerning easier test reporting/aggregated results (for examiners) and better navigation within the test grouping and between the tests (for students). So we also appreciate this idea!
The combination of fixed and randomized questions fits well with the feature request "Several questions on one test page" mentioned by Fred. The settings for the time limit should be optional: each section has its own time limit OR there is only one time limit for the whole test.

Florian Suittenpointner, 4 April 2013:
I don't think it makes sense to add complexity in both directions - within the object AND outside of it.
The main reason for my proposal was to "protect" the test module from getting too complex.
Avoiding complexity is not only a usability issue but also of a stable and properly working system, especially in connection with functionality that may have as serious consequences as tests.
Therefore, my idea was to rather offer additional complexity outside of the normal test module, so people who have requirements like Christine can meet them while all the others can keep the functionality as it is (and works!) by now.
However, after I read Christine's last post, I thought that maybe those pure navigation aspects (like what is on one page) aren't really a burden for the test module (unlike things such as rendering who has passed a test) ...
Maybe the module maintainer can make a statement about that?
 

BH 20. October 2014:
 
The Uni Freiburg funded the creation of a concept handling handling the requirement of this feature request.
 
See Test-Parts and Question-Groups.