Criteria for assignment reviews

1 Requirements

Basics

Exercises should be evaluate able by a criteria catalog. This function is especially useful for fairer peer feedback results.
The Sum of criteria can be used for automatic “learning progress“, in future.

Function

• Exercise editors can add criteria catalogs to excercises.
• Exercise editors can add any amount of criteria to each catalog.
• For each assignment editors can choose one of thes catalog
• Each criterion has a description, at least one of the following options and is "required" or not:
  • Rating (5 Stars)
  • Text
  • File
  • Fulfilled criteria (2 Radios (Yes/No))
• A review is complete if all required criteria are rated.

Future use

• Each criterion has a weight. Weight is allocated equal for all criteria, by default.
• For end result the overview shows:
  • Average ratings (weight)
  • percentage value for fulfilled criteria (weight)
  • list of files
  • summarized text (criterion as headlines, entered texts as paragraphs)

• Learning progress can be set automatically by rating or fulfilled criteria.

Mockups

2 Additional Information

• Idea / concept: Jochen Erkens (j.erkens@fh-aachen.de)
• Interest in funding: FH Aachen
• Maintainer: (will be set by Jour Fixe / maintainer)
• Implementation of the feature is done by (will be set by Jour Fixe / maintainer)
• Testcases by: FH Aachen

3 Discussion

Killing, Alexander [alex], 16 July 2015: We got funding for this from FH Aachen, but will most likely not be able to deliver this with 5.1. So this feature comes (if appreciated by JF) with 5.2.

JourFixe, ILIAS [jourfixe], August 03, 2015: We highly appreciate this feature and schedule it for 5.1 (resp. 5.2 if implementation is not possible until August 28). Jörg and Matthias will discuss and decide upon presentation details, esp. where and how to present feedback for uploaded assignments. Open question: will a re-use of a criterial catalogue be supported?

4 Implementation

, 13 Aug 2015: The criteria catalogues have been added to the exercise. We already did this for 5.1 because several other major features were changed or added to the exercise module including peer feedback settings. As we should not change the general workflow in every major release, 5.1 is now a bigger step - especially for peer feedback.
The existing feedback settings are still available if no catalogue is selected, a migration is not necessary in any way. Using the catalogues enables any combination of criterias for peer feedback. Because types of criteria (and critiera properties) might be added in future releases, we opted to have the same basic editing workflow as custom/advanced metadata.

Test Cases

Test cases completed at 2015-08-24 by Alexandra Tödt 

• http://testrail.ilias.de/index.php?/cases/view/6356: C6356 Übung anlegen
• http://testrail.ilias.de/index.php?/cases/view/6358: C6358 Übungseinheit mit dem Abgabetyp Datei hinzufügen    
• http://testrail.ilias.de/index.php?/cases/view/6357: C6357 Übungseinheit mit dem Abgabetyp Text hinzufügen      
• http://testrail.ilias.de/index.php?/cases/view/6359: C6359 Übungseinheit mit dem Abgabetyp Blog hinzufügen       
• http://testrail.ilias.de/index.php?/cases/view/6360: C6360 Übungseinheit vom Typ Portfolio erstellen                
• http://testrail.ilias.de/index.php?/cases/view/6349: C6349 Kriterienkatalog im Reiter Einstellungen der Übung anlegen                           
• http://testrail.ilias.de/index.php?/cases/view/6347: C6347 Kriterien bearbeiten/erstellen                  
• http://testrail.ilias.de/index.php?/cases/view/6348: C6348 Kriterienkatalog einer Übungseinheit zuordnen
• http://testrail.ilias.de/index.php?/cases/view/6378: C6378 Übungseinheit vom Typ Datei abgeben                      
• http://testrail.ilias.de/index.php?/cases/view/6377: C6377 Übungseinheit vom Typ Blog abgeben: Bestehendes Blog verknüpfen                        
• http://testrail.ilias.de/index.php?/cases/view/6379: C6379 Übungseinheit vom Typ Text abgeben                        
• http://testrail.ilias.de/index.php?/cases/view/6380: C6380 Übungseinheit vom Typ Portfolio abgeben: Bestehendes Portfolio nutzen                
• http://testrail.ilias.de/index.php?/cases/view/6381: C6381 Feedback zu Lösungen anhand von Kriterienkatalogen erstelle und einsehen

Approval

Tested successfully and approved at 2015-08-24 by Jochen Erkens.