Feature Wiki

Information about planned and released features

Tabs

Optional History for saved Testinput

Page Overview

[Hide]

1 Initial Problem

Bad things can happen during a test: Deleting (and saving) test input by mistake, an error during transmission or malicious participants who lie about their input.

The only way to recover input or to prove whether an answer was given (or not given) is the detailed Database log so far - if it exists:

  • ILIAS only stores the latest input - the old input is overwritten.
  • The current log gives the information whether values have been entered and how many points have been granted. This is insufficient e.g. in case of a cloze-question you are still unable to say which gap has been or has not been answered.
To prevent this, I suggest the creation of a history-log for every input on saving.The feature will require
  • an option for every questiontype in the T&A-section (it results in large logs, so it may not be active by default or for every qst by default).
  • an additional if-statement checking this option in every questiontype before the input is submitted to the assessment-log.
  • changing the logtext-column in the database to longtext.
  • an update to the DevGuide so developers are aware there exists a detailed log for the answers.
We use this enhancement in Halle & Leipzig for essay and cloze at the moment. It saves us some trouble.

2 Conceptual Summary

We suggest to enhance the options inside T&A-section subtab "Log Data -> Settings". An option per questiontype (similar to the list at "Question Settings -> Available Question Types") shall be available to enable/disable logging of entries during testruns.

Since saved input can be authorized and unauthorized, there are three possible ways to implement the log:

  • Authorised input only (manually confirmed saving by the participant)
  • Authorised AND automatically saved input
  • An option to choose between both
Behaviour:
  • If an answer is free text, the text should be logged, if an answer is represended by ID(s), e.g. multiple choice, the ID(s) would be sufficient, as long as a admin or a tester is able to draw conclusions which (humanreadable) answer was givven. (The possibility to mess up the question with the correction mode and therefore being probably unable to reengineer the presented answers to the ID's is ignored here.)
  • The Log shall be extended to add a line per save action, showing the answer(s)/answer-ID(s) (as stated in the table tst_solutionsvalue1 and value2).
  • The automatically triggered save when time for the test is over is handled like an authorised input and shall therefore be logged as such.
  • Optional: To prepare for potential future needs (e.g. restoring during active test), the logged data schould be serializable.
 To identify the data to be saved, we can use the following table:

Saving answers in ILIAS

Questiontype

Data

Stored in...

SingleChoice

Answer-ID

value1

MultipleChoice

Answer-ID

value1, own row (question_fi and answer_fi) per chosen answer

KprimChoice

1 or 0 (pos. or neg. answer)

value2, value1 represents the kprim-row (0-3), own row per answer

ErrorText

Answer-ID

value1, own row per chosen word

ImagemapQuestion

Answer-ID

value1, own row (question_fi and answer_fi) per chosen answer

ClozeTest

Answer-ID, Answer-Text

value2, value1 represents the number of the gap, own row for each gap

Numeric

Answer

value1

Formular

Variables, Answer

value2, value1 stores the variablenames, value2 stores the values, own row for each value

TextSubset

Answer

value1, own row for each answer

OrderingQuestion

Answer-ID

value1 and value2, 1 stores the position (0-x), 2 stores a random ID for the options, one row per option, would not be understandable for normal users when they see the log

OrderingHorizontal

Answer

value1, stores the answer as a single string

MatchingQuestion

Answer-ID

value1 stores the term, value2 the definition-ID, both widh random IDs, own row per definition

TextQuestion

Answer

value1, stores the answer as a single string

FileUpload

-

Only filenames could be logged, won't help to backtrace answers in this approach (maybe not even needed here)

LongMenu

Answer

value2 stores the string of the answer, value1 stores the gap#, own row per gap

FlashQuestion

Depends

-

JavaApplet

Depends

-

qstPlugins

Depends

A optional function to safe to (and load from) the log shall be usable for each qstPlugin in the future.

To get rid of inconsistency additionally save:

  • when a test pass starts
  • when a question was reached
  • when a question was skipped (has not been answered or edited)
  • when a solution was willingly deleted by the participent (already merged with this Pull Request)
  • when a test pass ends willingly
  • when a test pass ends unwillingly due to timeout
  • the ID of the question in addition to the Title

3 User Interface Modifications

3.1 List of Affected Views

Administration->T&A

3.2 User Interface Details

The table and the generated CSV at "Administration -> Test and Assessment -> Log Data -> Log Data Output" will show more entries.

3.3 New User Interface Concepts

None.

4 Technical Information

5 Contact

  • Author of the Request:  Jobst, Christoph [cjobst] Uni Halle/Uni Leipzig
  • Maintainer: (will be set by Jour Fixe / maintainer)
  • Implementation of the feature is done by: {The maintainer must add the name of the implementing developer.}

6 Funding

If you are interest in funding this feature, please add your name and institution to this list.

  • University Leipzig
  • , Hochschule Karlsruhe - Technik und Wirtschaft
  • Universität Bonn
  • Universität Halle
  • Universität Regensburg
  • Universität Marburg

7 Discussion

C. Jobst, 5.12.2016: Approved during the SIG EA Workshop in Leipzig without alterations.

Heyser, Björn [bheyser], 21 Dez 2017: The maintainer highly supports this request in general. A general refactoring to the solution handling of a future question service should be done prior to this strong enhancement.

, 02.05.2018: I would recommend extending this feature:

  • Saving the input when student clicks "Check/Prüfen" (when using instant feedback)
  • The feature shuold be available for the question type STACK.
Also, it is would be important to log:
  • The name/username of the student.
  • Total points received for the answer.
  • Timestamp.
We would use the logs to evaluate the difficulty of the questions, to improve the automatic feedback provided to the students, and to find out possible misunderstandings and pre-concepts, as described in the feature-request Saving students answers in tests with instant feedback.

Jobst, Christoph [cjobst], 2.5.2018: Logging the input on "Check" is covered due to the already implemented saving routine. Clicking "Check" results in a POST-request which triggers saving and therefore putting the reached points, along with name and time into the log. Extended by this feature request it would automatically include the specific answer(s) too.
For the meantime: as long as logging question specific stuff has to be implemented inside the questiontype, it has to be prepared inside the plugin itself (for STACK: exactly HERE).

, 03.05.2018: Thanks, that sounds good! I would support implementing this feature instead of Saving students answers in tests with instant feedback.

SIG EA 18.06.2019 : We would like to have this feature integrated to ILIAS 6.0.

  • The Logging of participants solutions should be activatable per question type
  • We do not want to differentiate between authorized and intermediate solutions, bot should be logged
  • The logging for selection question types should involve the technical identifier for the submitted solution as well as the human readable representation
  • The assessment log should be part of the archive export

8 Implementation

{please give a description of the final implementation and add screenshots if possible}

Test Cases

Test cases completed at {date} by {user}

  • {Test case number linked to Testrail} : {test case title}

Approval

Approved at {date} by {user}.

Last edited: 18. Jun 2019, 16:14, Heyser, Björn [bheyser]