Feature Wiki

Information about planned and released features

Tabs

Learning Progress Trigger after Finish

Page Overview

[Hide]

1 Requirements

The current implementation of the learning progress in tests is a constant source for misunderstandings about the progress and performance of users in tests. A revision is therefore highly appreciated.

Current Situation

Several problems arise due to the fact that the learning progress is calculated everytime a learner submits an answer in a test.

  • When a participant starts the test, the learning progress status is "failed" because not sufficient points have been gathered yet to pass the test.
  • The learning progress stays at "failed" while the participant works through the test - as long as the minimum amount of points has not been reached to pass it.
  • Even if the participant has still to answer several questions and not finished the test yet, the learning progress of the test can already change to "passed".
  • A test is never "in progress" - even if some questions still have to be answered. Therefore, it is not easy to identify for a user which tests still has to be finished and which not.

Desired Behaviour

In a future ILIAS we would like to redefine the statuses of learning progress in tests as follows:

  • LP status ‘Not started yet’: User has not started yet a first test run. The visit of the ‘Info’ page is not considered as a start of the test.
  • LP status ‘In progress’: User has started first test run
    • and the user has not finished the test by his own (clicking on button ‘Finish Test’)
    • and the test run has not been finished by a cron job or tutor ( ➔ new feature! )
  • LP status ‘Passed’: Participant's test run has been finished by participant, tutor or cron job and the defined number of points for passed status has been reached.
  • LP status ‘Failed’: Participant's test run has been finished by participant, tutor or cron job and the defined number of points for passed status has not been reached.

Consequences

  1. A test participant should be able to finish the test on every test page and the ‘Info’ (or ‘Start’ page) by clicking on a button Finish Test. Important: this has to be possible even after the defined ‘finishing time’ of a test. Clicking on a button "Finish" cannot be treated like answering questions.
  2. When the ‘finishing time’ of a test has been reached, also a tutor should be able to finish the tests of those learners that have not finished it by their own.
  3. Alternatively, a cron job may finish test runs after the test's ending time (= Finishing Time) or after the ‘Limit Duration of Test' has been reached by the participant.
  4. The view ‘Participating Users’ (tab ‘Participants’) has to be extended to allow tutors to finish the test for all participants (dedicated button) or for selected users (extension of multi-select).

Points to be discussed

  1. If a test has neither a defined finishing time nor a limit duration of test, should there be a third trigger to finish the test automatically? IMHO it could only be something like a period of inactivity.
  2. What happens with the LP status in a second test run? Is there an alternative to keeping the reached status (passed or failed) during the following test runs?

2 Status

3 Discussion

JF 23 June 2014: We schedule this as a usability fix for for 4.5.

Kiegel, Colin [kiegel] Sept 29 2015: This would be a great improvement to ILIAS. But I think it should be discussed what happens to "heritage data". What about users that were in the middle of some test before an update to the new behaviour?

  • If these heritage testruns keep their previous learning progress it would lead to a mix of (a) users with new test runs treated according to the new system and (b) users with heritage test runs treated according to the old system. This might *increase* the confusion compared to the current status. It could also be seen as unfair.
  • If these heritage testruns are treated according to the new system it could also lead to confusion. Some people might already have forgotten that they didn't "finish" the test and wonder why their learning progress is being reset to "in progress" or something like that.
I think this migration is a serious issue either way. A third option would be to switch old vs. new behaviour on a per-test basis. This would lead to legacy tests with different behaviour than new tests, which could only be a temporary solution until support for thoses legacy tests should be dropped someday in the future. I think the pros and cons should be discussed and weighed carefully.

Any other suggestions?

Jour Fixe, October 12, 2015: We highly appreciate this change of behaviour in tests and schedule it for 5.2.

  • When LP mode "test passed" is activated and "last run" is relevant for passed status, all test runs are handled in the same way: test run started = in progress, test finished => passed or failed.
  • When LP mode "test passed" and "best run" is activated, the achieved status of the first test run is kept until user reached passed (or failed).
  • For LP mode "test completed" the status is "in progress" as long as the first test run has not been finished. If finished, the status is set to "completed".

4 Implementation

The LP States are now as followed:

  • LP Mode "Test Passed" & Scoring Mode "Last Pass"
    • Prior to Any Action => Not Attempted
    • First Pass Started => In Progress
    • First Pass Finished with
      • FAILING Result => Failed
      • PASSING Result => Completed
    • Additional Test Passes will be handled the same way
  • LP Mode "Test Passed" & Scoring Mode "Best Pass"
    • Prior to Any Action => Not Attempted
    • First Pass Started => In Progress
    • First Pass Finished ..
      • .. with PASSING Result => Completed (won't change any longer)
      • .. with FAILING Result => In Progress (is kept until a further pass is finished with a PASSING result)
    • Futher Passes with a PASSING result can switch from In Progress to Completed
    • Last Pass Finished ..
      • .. with PASSING Result => Completed
      • .. with FAILING Result => Failed
  • LP Mode "Test Completed" (scoring mode doesn't matter)
    • Prior to Any Action => Not Attempted
    • First Pass Started => In Progress
    • First Pass Finished => Completed
    • This LP won't change any longer
Also a button was added to the participants tab to finish the passes manually for users and a cronjob was implemented to finish unfinished passes but only for test which have set an ending time or a processing time. 

Test Cases

Test cases completed at 16 Aug 2016 by Heyser, Björn [bheyser]

Approval

Approved at August 16, 2016 by Kunkel, Matthias [mkunkel].

Last edited: 19. Apr 2023, 12:27, Kunkel, Matthias [mkunkel]