Show Advanced KnowledgeHide Advanced Knowledge

Learning Objectives-Driven Course

1 Requirements

1.1 Settings within Learning Objectives tab

The original 3 'Course Design' settings

• ‘Course Starts with Placement Test’
• ‘Credit-by-examination design’
• ‘Course Starts with Learning and Practise’

should be split up into 2 settings with 7options and 5 sub-options:

• Initial Test
  • Initial diagnostic Test on all learning objectives (1)
    • Initial Diagnostic Test on all learning objectives is start object (1a)
  • Initial diagnostic Test per learning objective (2) 
  • Initial  achievment test on all learning objectives (3)
    • Initial  achievment test on all learning objectives is start object (3a)
  • Initial  achievment test per learning objective (4)
  • no initial test (5) 
• Final Test 
  • Final achievement test on all learning objectives (6) 
    • Final achievement test on all learning objectives is start object (6a)
    • test question on successfully mastered test questions are hidden (6b)
    • test question on successfully mastered test are flagged (6c) 
  • Final achievement test per learning objective (7) 
• reset all test results 

Remark on the options above

• if for option 1 or 3 the initial test is NOT the start object, participants are presented with all learning objectives and assigned content on opening the course. Participants are asked to complete the inital tests. 
• for options 2 or 4 all learning objectives are displayed but assigned content, final test and content of the page editor is hidden until the respective initial test was taken. content that is not assigned to learning objectives will be displayed. 
• behavior of option 6b and 6c has to be decided upon by the test maintainer: the learning objective course is handing over data to the test. 
• option 6c questions already answered are shown but can no longer be changed. This behavior is not available for random tests. 

Labels of sub-tabs

• settings
• initial test (not displayed if there is no intial test) 
• final test
• learning objectives
• start objects

1.2 Tests per learning objective 

The implementation of tests for individual learning objectives is fundamentally changed. 

Until 5.0 one test was used for testing mastery of all objectives and questions from this test were assigned to the learning objectives allowing mastering them while still being comprised in the same test object.
Now the suggestion is to use separate test objects for each indvidual learning objective. Each test can thus be assigned only one learning objective.  

Assigning Tests to Learning Objectives
The workflow of assigning tests to learning objectives starts in course > learning objectives > learning objectives

• For tests on all learning obejctives (options 1, 3, 6)  questions are managed by the action menu entry "assign questions for initial test" repsectivele "assign questions for final test"
  • for fixed tests the questions for each individual learning objectives can be selected 
  • for random tests pools or taxonomy paths can be selected
• for tests per learning objective (options 2, 4, 7) there is no selection of questions: all questions of a test will be presented. Thus whole tests are assigned by the action menu entry "assign tests for initial test" repsectivele "assign tests for final test"
  • the dialogue triggered by the "assign test" action only offeres test that have not yet been assigned to other learning objectives
  • users have to monitor the consistent configuration of individual test objects by themselves / burden their participants with tests beaving differently
• the thresholds for passing a test are set at percentages. whether the test is actually passed according to it's own settings is ignored, for mastering the learning objective the percentage figure is the only benchmark.

Sub-tabs "initial test" and "final test"
The following changes are suggested compared with 5.0: 

• if tests are assigned per learning objective (options 2, 4, 7) all test are listed in the sub-tabs "initial test" respectively "final test"
• in these subtest a message is displayed to inform users about the setting of these tests as
  • diagnostic vs. achievement 
  • on all learning objectives vs per learning objectives
• if tests are assigned per learning objective a new column "learning objectives" is added

Behavior of tests

• the number of test passes will be determined only by the test object itsel no longer streamlined for all tests by the course. 
• how random tests / fixed test should behave in case of more than one test pass is to be decided by the test maintainer

1.3 results in more than one test pass 

Tests per learning objective (options 2, 4, 7)

• depending on the settings of the test more than one test pass can be made
• generally the last test pass will be considered as the result of the test and displayed accordingly. 
• if the learning objective was mastered and there are still an additional test pass would be possible then users would be asked: You have mastered all objectives. Restarting the test nulls your results: do you want to restart the test?

Tests on all learning objectives (options 1, 3, 6) 

• Depending on the seetings of the individual test there may be mor than one test pass. 
• Invariantly the results last pass will be scored and displayed 
• Option 6b: 
  • Only questions on learning objectives not yet mastered will be shown. 
  • If all objectives were mastered and and additional test pass would be possible then users would be asked: You have mastered all objectives. Restarting the test nulls your results: do you want to restart the test?
• Option 6c: 
  • questions on learning objectives already mastered are highlighted in case the test is of fixed questions - for random test this is not possible. 
  • If all objectives were mastered and an additional test pass would be possible then users would be asked: You have mastered all objectives. Restarting the test nulls your results: do you want to restart the test?

1.4 Preconditions

• Preconditions are set per Test using the normal IIAS-functionality. 
• Preconditions should be supported for Plug-ins both as a trigger and as a result of preconditions. 
• The link "back to course" cannot be consistently offered, since some objects are maintained by other service providers

1.5 Display of Test Results in Learning Objective Accordions

The following mock-ups show the presentation of test results in the learning objective accordions. 

The result presentation screens are linked as follows: 

• if a results is displayed in the learning objective accordion, then the display links to the screen with the resultpresentation of the test. 
• if a test has a result, then participants get an entry in the action-menu linking to it's results screen. 
• in the Member tab tutors get a new entry in the action-menu of every participant: "Show test results". This links invokes a screen showing all results of all tests of this participant in the learning objective -driven course. Per test results there is a link to the detailed test results screen. (This whole section is to be handeled by the test component)

1.6 Tabs with Learning Objectives

• More than one tab can be open at the same time
• 'Collapse all / open all' button avaliable
• Latex code can be used in tabs, unless suprressed by an external MathJax component

2 Additional Information

• Idea / concept: Matthias Kunkel et. al.
• Funding: Helmut Schmidt Universität
• Maintainer: Meyer, Stefan [smeyer]
• Implementation of the feature is done by Meyer, Stefan [smeyer] (course component), Heyser, Björn [bheyser] (test component)
• Test cases by / status: Tödt, Alexandra [atoedt]

3 Discussion

Fabian Kruse, 11.05.2015: I appreciate the work put into this well-prepared concept. That said, I would also like to raise awareness for the increased complexity that would result from an implementation as suggested here. While I can understand the desire for many of the suggested finer settings, they could also put off new users of the LOC. That’s not to say it shouldn’t be done, but it is certainly a matter worth considering.

Specifically, the deeper entanglement between test and LOC look problematic to me: "users have to monitor the consistent configuration of individual test objects by themselves / burden their participants with tests beaving differently" - This is likely to cause many issues, as the test tool itself is quite complex on its own already, even for seasoned ILIAS users.

JourFixe, ILIAS [jourfixe], 2015-08-18: We appreciate the feature and schedule it for 5.1 if the following changes are made:

• Please add bylines per option in Learning Objectives » Settings.
• Existing settings for course design in 5.0 courses have to be migrated as follows:
  • 'Course starts with placement test' => 'Initial diagnostic test on all learning objectives' + start object enabled + 'Final achievement test on all learning objectives'
  • 'Credit-by-examination design' => 'No initial test' + 'Final achievement test on all learning objectives is start object' + 'test question on successfully mastered test questions are hidden'
  • Course starts with learning and practise' => 'No initial test' + 'Final achievement test on all learning objectives' + 'test question on successfully mastered test questions are hidden' + '
• To reduce complexity and improve usability, three specific options are disabled per default by an additional setting in Administration » Course.
  • New options 'Initial achievment test on all ...' and '... per learning objective' are disabled . The administration settings gets a byline that recommends to use tests of the same level for initial and final test.
  • Option 'Mastered Learning Objectives' is disabled. (correction of AT and MK, 2016-12-20)
• Default option when configuring the settings of the initial test should be "Use Existing Test" _IF_ the course already contains at least one test.
• Display of test results:
  • Please change the label "Ergebnis Einführungstest" into "Diagnose"
  • If "Ergebnis" is a link that directs to the test results, the "link" should be displayed as button to make it easily distinguishable from a non-linked "Ergebnis" text (which is also displayed in blue due to the accordion style).
• Learning Progress status: if a test that is assigned to a learning objective has been started by user, the learning progress status is set to "in progress". If one test is used for several LO, all LO get status "in progress". LO status "green" is reserved vor achieved LO, "red" for those LO that have no more runs to achieve them (finally failed).

Glaubitz, Marko [mglaubitz], 01.09.2015: Thanks for improving this feature so massively. I have a conecptional question though: is tehre a reason, why only "Test" objects  serve the function of the placement / final test. I could imagine lots of scenarios where an excercise (or an ILIAS leraning module, ...) makes a slot of sense, too. An example for having an excercise would be having a peer feedback element as the final test. The same argument holds for the placements tests.
Myabe one way of solving this, would be to allow the option: "no final test" and use the learning progress as a marker for having reached the learning objectives. I know taht this diludes the state of "having reached a learning objective" somewhat, but if a learner reaches 70% of a test an passes, it does not mean that they reached the objective precisely / fully.

4 Implementation

Settings for learning objective driven courses are implemented as described in 1.1

The new features

• Tests per learning objective (1.2)
• Results in more than one pass (1.3)
• Preconditions
• Display of Learning Objective Results Accordeons
• Tabs with Learning Objectives

are implemented as described in 1)

Test Cases

Test cases completed at February 2015 by Alexandra Tödt

• http://testrail.ilias.de/index.php?/suites/view/53: It is a total of 108 test cases thus I did not link them individually

Approval

Karola Koch at 2015-08-05: I, Karola Koch as a representative of the HSU/ UniBw H, accept the implementation of the LoK 2.0. Nevertheless more testing has to be done by us to eliminate still remaining bugs and weaknesses...