Feature Wiki

Information about planned and released features

Tabs

Clarification of Log Data for Tests

1 Initial Problem

This FR aims at clarifying some log entries in ILIAS log data files. The goal is to make log data easier to understand for humans, i.e. examiners.

This FR only concerns data that can be exported by ILIAS admins via: Administration -> Repository and Objects -> Test and Assessment -> Log Data!
However, it would not be surprising if by changing text in this dataset will also change data in other files elsewhere. I.e. "test.log" created with the core archive export file for ILIAS tests.

Problems:

  • log entries cannot be mapped to users, because no unique user account is referenced; "Jon Doe"...could be anyone...
  • log entry for test finish is too technical/cryptic: "updateLearningProgressAfterPassFinishedIsWritten has been called from ilTestPlayerAbstractGUI (finPass=0 / Lp=)"
    • it also does not differ between an automatic test finish due to a reached time limit or the user manually finishing the test pass
  • log entry for only viewing/reading a question are unclear, right now this action can be derived from the log entry "User did not enter values"
  • the log contains UNIX timestamps that are not readable for humans: "The test was modified [complete: 1 => , tstamp: 1613562328 => 1613562898]"
  • log entries contain unresolved variable names that are not comprehensible to the average ILIAS user (i.e. examiner): "The test was modified [tstamp: 1613562269 => 1613562283, question_set_type: FIXED_QUEST_SET => RANDOM_QUEST_SET]"
  • log entries for manual feedback make no use of the question column and make it hard to find the actual feedback/comment by the examiner: "Fabian Sesterhenn (admin-fsesterh) added manual feedback for test person Doe1, Jon and question Question 13 - Essay: <p>test 1 2 3... dsdsdsd</p>"

2 Conceptual Summary

2.1 RFC 2119

In order to provide a clear understanding of the importance/relevance of a requirement we use the following keywords as described in RFC 2119:

  • MUST, MUST NOT
  • REQUIRED
  • SHALL, SHALL NOT
  • SHOULD, SHOULD NOT
  • RECOMMENDED
  • MAY
  • OPTIONAL

2.2 Requirements

  • The user name is extended with the account loginname in parentheses, i.e. "Fabian Sesterhenn (sesterhenn)"
  • "updateLearningProgressAfterPassFinishedIsWritten has been called from ilTestPlayerAbstractGUI" is replaced with "Test finished"
  • "User did not enter values" is replaced with "User viewed question"
  • manual feedback for a question will reference the question in the intended column, the log entry itself can be shortened to "Manual Feedback for participant [TESTPARTICIPANT]: [FEEDBACK]."

3 User Interface Modifications

3.1 List of Affected Views

None. Only the export data for test protocols is affected.

3.2 User Interface Details

None.

3.3 New User Interface Concepts

None.

3.4 Accessibility Implications

None.

4 Technical Information

{ The maintainer has to provide necessary technical information, e.g. dependencies on other ILIAS components, necessary modifications in general services/architecture, potential security or performance issues. }

5 Privacy

  • user name (login)
  • name and surname of user account
We should be allowed to process this data for e-exams.

6 Security

No security implications.

7 Contact

8 Funding

If you are interest in funding this feature, please add your name and institution to this list.

9 Discussion

Tödt, Alexandra [atoedt]: 29 AUG 2022:

  • I cannot reproduce the John Doe issue. I get clear names in ILIAS 8.
  • Please elaborate a bit for whom / for what purpose the log is. Please outline legal requirements for the log very broadly.
  • Have you considered what was the reason for using the admittedly not very human-friendly time stamp? SInce the export does contain human-friendly dates next to time stamps, I wonder whether this is done on purpose and for standing up in court. 

Sesterhenn, Fabian [sesterhenn], 14. Sep 2022:

The feature request was discussed in a first workshop on September, 13th 2022. See the full protocol for details. Key points:

  • In order to achieve some longterm goals regarding e-exams and to avoid working on features that will not last we need a full report on the implementation of logging for ILIAS tests.  Judging by the current workload for ILIAS developers this must be postponed until 2023.
    • The specifics for this analysis need to be worked out. In short it should answer: What, Where, When does ILIAS log regarding tests?
    • Based on this information we can create a concept for log data for ILIAS tests that better fulfill the requirements for e-exams.
  • In the meantime participants agree that small changes ("baby steps") like this FR make sense as long as the scope of the FR is limited to changes in text. I.e. we can change the text of a log entry in the log data, but we should not, without the aforementioned analysis, start adding new actions to be logged.

10 Implementation

{ The maintainer has to give a description of the final implementation and add screenshots if possible. }

Test Cases

Test cases completed at {date} by {user}

  • {Test case number linked to Testrail} : {test case title}

Approval

Approved at {date} by {user}.

Last edited: 14. Sep 2022, 14:36, Sesterhenn, Fabian [sesterhenn]