Show Advanced KnowledgeHide Advanced Knowledge

Cloze question: Points on gap combinations

1 Requirements

In language tests with cloze questions it is sometimes neccessary to not give points on answers for single gaps but for combined answers in different gaps. It should be possible to define such cominations additionally to the points per gap. The total points given for the question will be the sum of both.

Examples (German):
* Er [wird] seine Hausaufgaben zuverlässig [erledigen].
* Er [hat] seine Hausaufgaben zuverlässig [erledigt].
 
* Am Montag [schicke] ich das Paket [ab].
* Am Montag [gebe] ich das Paket [auf].
 
Example (English): "He will be doing", "he will do",  but not: "he will be do".
 
The QTI standard allows such complex point calculations as "Response processing":
(see: http://www.imsglobal.org/question/qtiv2p1/imsqti_implv2p1.html#section10007)

This topic is also related to the separation of two definitions:

• The definition of the question as it is presented to the user. This should not be changeable after a test is written.
• The definition of the points given to specific answers. This may be changed after a test is written (see implemented feature Nachkorrekturmodus). The screen for that shoul not allow to change the gap types or edit the values of "select" gaps.

For cloze questions this can be done by separating gap definitions from result definitions. It may look similar to the following example:

2 Status

• Scheduled for: Release 5.0
• Funding: Funded by Universität Erlangen-Nürnberg
• Maintainer: Björn Heyser, Databay AG
• Development: Feature is to be developed by Guido Vollbach, Databay AG
• Contract settled: yes
• Tested by: Fred Neumann, fred.neumann@fau.de / (testing completed)

3 Additional Information

• If you want to know more about this feature, its implementation or funding, please contact: fred.neumann@fim.uni-erlangen.de

4 Discussion

JF 15 Apr 2013: Since the editing in the T&A is already quite complex: Could you provide a mock-up screen for the editing view, please? If this can be edited without too much additional complexity, we could schedule it for 4.4. Please put the topic on the JF agenda again.

FN 29 Okt 2013: added example and updated the funding status

JF 31 Mar 2014: We support the general idea and schedule it for 4.5.
 
Naming: We would prefer to use the term "Gap Combination" instead of "Result".
 

• The usual interface should look like 4.4.
• Only a button "Add Gap Combination" should be presented on the bottom of the screen.
• If pressed a "one-line" gap combination is added. The preselected "Gap"-Drop-Down value is "-- Please Select --". As soon as a gap is selected the "Points" column it its original definition section is hidden (values preserved until submission) -> It is not possible to assign points to the same gap in the original definition section and the combination area at the same time.
• We do not introduce a "Matches Case Sensitive" selection in the combination areas, since there is no possibility to do this on single-gap-basis yet.
• If the "-" button of "the last line" in a combination area is pressed, the whole gap combination is deleted.
• Every click on "-" the "Points" columns of some gaps may re-appear, if they are not used in combinations anymore (former values should re-appear, if the user did not "Save" the complete question in the meantime).
• Scoring: The points of all fulfilled combination rules are summed up in the scoring, even if multiple rules inlcude the same gaps. (Every gap combination is evaluated separately)

BH 09 July 2014: During the development some questions came up:
 

• In the screenshot the reference value for a select gap is defined with a text input in the gap combinations definition, although the selection list is defined in the gap defitinion section. We prefer to provide this defined selection list for choosing the reference value in the gap combinations definition, instead of a text input.
• With text gaps used in gap combinations the definition of reference values has been moved to the gap combination section completely, but we would like to keep it at the gap definition section. We would prefer to provide a selection list for choosing the reference values in the gap combinations section.
• The screenshot does not show how to handle numeric gaps in gap combinations. We could imagine three different ways:
  • Definition of Value and Lower/Upper-Bound at the level of gap definition. The reference values selectable in gap combinations would be "in range" and "out of range" (or something similar).
  • Definition of Value and Lower/Upper-Bound at the level of gap definition. The reference values are defined with a lower and an upper bound that must be in the range of the boundaries defined at the gap definition level.
  • Definition of Value and Lower/Upper-Bound are completely moved to the gap combinations definition

Fred Neumann 10 July 2014: My mock-up was inspired by the idea of strictly separating the question definition (what is shown to the participant) from the definition of how the user input is scored.
However, according to the last Jour fix comment from March 2014 the gap combinations should be more treated as an “extended feature” of the existing question definition.
 
This is fine for me because the core requirement of the feature is just to allow an extra scoring for combinations. But then I propose to follow a simplified “additional setting” approach for the combinations consequently. That would mean:

• A combination is always created with two lines and must be saved with at least two completed lines.
• No fields in the gap definitions are hidden when a combination is added or a gap is selected in a combination.
• The scoring for a combination is added to the scoring of single gaps (combinations give “bonus points” or “malus points”).
• The reference values for select gaps and text gaps are selected from the values listed in the gap definition.
• The reference for numerical gaps in a combination is “in range” or “out of range”.
• It should be possible that single gaps have no points assigned – just the combinations. The question is valid if every gap has either points assigned directly or is used in a combination with points.

If the adaptive interface proposed by the JF should be kept, then my preferences for Björns questions are:

• Agree with the proposals for text gaps and select gaps
• Numeric gaps should have “in range” and “out of range” options in the combination

JF 21 July 2014: It is that the author needs to explicitly specify a "best" solution and to derive the "max. points" from this best solution. If a learner enters "an even better solution" (s)he will not get more points than the specified max. points.

JF 10 Nov 2014: We highly appreciate the solution worked out by Fred, Björn and Guido.

5 Implementation

BH 18 September 2014: Feature is implemented and ready for acceptance.

BH 05 JAN 2015: Implemented changes requested by Fred Neumann on 7th Nov 2014.

6 Testing

Fred Neumann, 22. Dezember 2014:

• Added the slightly modified test cases for the gap combinations to TestRail (section Test&Assessment > Spezielle Fragentypen > Lückentext-Fragen).
• Tested the new fetures succissfully.

7 Test Cases

Testing Cloze Question
 
1 New UI 1.1 Essential Informations

1. The editing of this question type now expects JavaScript to be activated in your Browser

1.2 Prerequisites

1. Create a “Question Pool Test”
2. Create a “Cloze Question”
3. Enter a Title and Question

1.3 Simple Text Gap

1. Start with 1.2 Steps 2-3 in your Question Pool
   1. Insert one Gap with the “Insert Gap” Button below the “Cloze Text” field
   2. Click in the “[gap 1][/gap]” field in the “Cloze Text” box
      1. Enter one answer in the first “Values” field in the overlay
      2. Click on the green plus button
      3. Enter the second answer in the second “Values” field
      4. Add a third values field and remove it afterwards by clicking on the minus button next to it
      5. Enter the value 1 in the points field for the Correct answer value
      6. Click on the “Close” button to get rid of the overlay
   3. Check if your entered answer values are in the gap field in your “Cloze Text” box
   4. Check if your entered answer values in the form below with the title “Gap 1”
2. Save the Question

1.4 Simple Select Gap

1. Start with 1.2 Steps 2-3 in your Question Pool
   1. Insert one Gap with the “Insert Gap” Button below the “Cloze Text” field
   2. Click in the “[gap 1][/gap]” field in the “Cloze Text” box
      1. Change the “Type” of the Gap to “Select Gap” field in the overlay
         1. Instead of the “Text Field Length” Text field there should now a “Shuffle Answers” option
      2. Enter one answer in the first “Values” field
      3. Click on the green plus button
      4. Enter the second answer in the second “Values” field
      5. Add a third values field and remove it afterwards by clicking on the minus button next to it
      6. Enter the value 1 in the points field for the Correct answer value
      7. Click on the “Close” button to get rid of the overlay
   3. Check if your entered answer values are in the gap field in your “Cloze Text” box
   4. Check if your entered answer values in the form below with the title “Gap 1”
2. Save the Question

1.5 Simple Numeric Gap

1. Start with 1.2 Steps 2-3 in your Question Pool
   1. Insert one Gap with the “Insert Gap” Button below the “Cloze Text” field
   2. Click in the “[gap 1][/gap]” field in the “Cloze Text” box field in the overlay
      1. Change the “Type” of the Gap to “Numeric Gap”
      2. Enter your answer in the “Value” field
      3. Enter your “Lower Bound” and “Upper Bound”
      4. Enter your Points
2. Save the Question

* The Numeric Question supports decimal and and fraction values! No formulas. 1.6 Simple Multiple Text Gap

1. Start with 1.2 Steps 2-3 in your Question Pool
   1. Insert two Gaps with the “Insert Gap” Button below the “Cloze Text” field
   2. Click in the “[gap 1][/gap]” field in the “Cloze Text” box
      1. Enter one answer in the first “Values” field
      2. Click on the green plus button
      3. Enter the second answer in the second “Values” field
      4. Enter the value 1 in the points field for the Correct answer value
      5. Click on the “Close” button to get rid of the overlay
   3. Click in the “[gap 2][/gap]” field in the “Cloze Text” box
      1. Repeat Steps (i - v)
   4. Check if your entered answer values are in the gap field in your “Cloze Text” box
   5. Check if your entered answer values in the form below with the title “Gap 1”
2. Save the Question

1.7 Copy & Paste Gaps

1. Copy one of your Cloze Question like mentioned in 1.6
2. Edit the newly copied Question
   1. Mark your Gap fields in the “Cloze Text Field” and copy the content
   2. Paste your copied gaps in the “Cloze Text Field”
   3. Your newly inserted Gap should now have new gap numbers
   4. When you scroll down, your newly pasted gaps should now have a own form, where now points are given
   5. Enter the correct points for your Gaps and
3. Save the question

1.8 Simple insert multi term gap

1. Start with 1.2 Steps 2-3 in your Question Pool
   1. Enter comma separated terms in the Cloze Text field like: Term1,Term2,Term3
   2. Mark the terms
   3. Click on the “Insert Gap” button
   4. Scroll down to check if your new gap is created with all marked terms as values
   5. Insert Points for this new Gap
2. Save the question.

1.9 Copy Questions

1. Mark your Questions in the Question Pool
   1. Execute a Copy
   2. Click on the Paste Button
   3. Check if your now Copied Question are complete

1.10 Export/Import QuestionPool with Cloze Questions

1. Click on the “Create Export File” in your Question Pool
   1. It should be “QTI XML File” selected
   2. Download the Export File
   3. Create a new “Question Pool Test”
   4. Change to Option 2 “Import Question Pool Test”
   5. Select your in step b exported file an click the “Import” button
   6. Make sure all Question are selected and press the “Import” button
   7. Check if your imported Questions are complete

 
2 Gap Combinations
2.0 Essential Informations

1. You must have at least two Gaps to create a valid Gap combination
2. Every Gap can only be part of exact one combination.
3. Selecting the same Gap twice in one combination isn’t possible.
4. Gap Combination can have positive or negative points assigned to them
   1. Gap Combination replace the requirement to assign points to a Gap
   2. Only the points for the gap combination are counted if you entered points for the gap before you added the combination, this points will be ignored
5. If you have a Gap Combination, with only 2 Conditions and you click on the minus button, next to the condition form, the combination will be removed
6. If you have a Gap Combiantion with only one answerset and you remove this last answerset your combination will be removed

2.1 Prerequisites

1. Create a “Question Pool Test”
2. Create a “Cloze Question”
3. Enter a Title and Question

2.2 Simple 2-Gap Combination

1. Create a Simple Multiple Text Gap like in 1.5
   1. Click on the Button “Create Gap Combination” on the bottom of the page
   2. Enter 1 in the Points field in the newly appeared “Gap Combination 1” form
   3. Select your two Gaps in the Condition field and their correct answer
2. Save the Question

2.3 Multi Gap Combination

1. Create a Simple Multiple Text Gap like in 1.5 with at least 4 Gaps, you can also use the method mentioned in 1.7 and copy and paste your gaps
2. Create a new Gap Combination, you will notice that since you have more than 2 Gaps, you have add and remove buttons next to your “Conditions”
3. Click so often on the plus buttons in your “Gap Combination” until the plus buttons disappear
   1. This means you have the exact amount of gaps in your “Gap Combination”, like you have gaps in your Question
4. Remove one of the Condition
   1. The plus buttons will reappear
   2. Add one condition again
5. Fill out your remaining fields in your “Gap Combination”
6. Save your question

2.4 Pure Gap Combination Points

1. Create a new Cloze Question
   1. Add at least two Gaps without adding points for the correct answer in the gap
   2. You will see the Gaps have a red highlighting an the “Points” field
   3. Add a “Gap combination”
      1. Add “Points” for the “Gap Combination”
      2. Select your Gaps and the correct answer
         1. You will see, if you select your previously not correctly entered Gap, remember it hadn’t Points assigned to the correct answer, the red highlighting and the points field for this gap will disappear.
2. Save your question.

2.5 Copy Questions

1. Mark your Questions in the Question Pool
   1. Execute a Copy
   2. Click on the Paste Button
   3. Check if your now Copied Question are complete

2.6 Export/Import QuestionPool with Cloze Questions

1. Click on the “Create Export File” in your Question Pool
   1. It should be “QTI XML File” selected
   2. Download the Export File
   3. Create a new “Question Pool Test”
   4. Change to Option 2 “Import Question Pool Test”
   5. Select your in step b exported file an click the “Import” button
   6. Make sure all Question are selected and press the “Import” button
   7. Check if your imported Questions are complete