Show Advanced KnowledgeHide Advanced Knowledge

Centralized Survey

1 Requirements

1.1 Context

More and more universities are conducting surveys to improve their services. Among other issues, this includes:

• Teaching evaluation
• Supplier evaluation (= faculty survey)
• Freshmen surveys
• Alumni surveys

ILIAS has a survey tool which already covers many of the requirements.
This article outlines requirements that are not yet covered so far by this survey tool.

1.2 Objectives

1.2.1 General

The survey tool should meet the following general objectives:

• Good performance in administration and implementation as well as good usability (minimum of clicks, good display, intuitive operation)
• Privacy policy (in different shades)
• Anonymity of the students must be ensured.
• Results are presented to all stakeholders according to their permissions.

In a later incremental step, it would be useful to be able to filter surveys according to their activation period.

1.2.2 Automated Copying of a Survey Template into Courses

• Prerequisite for getting aggregated evaluations across courses is a unified survey in all relevant courses. Accordingly, a survey in ILIAS must be created that will be used as a centralized template.

  Due to the abovementioned different target groups, it will be necessary that different surveys templates can be offered for different purposes.

• Since it may be that several trainers are involved in a course, it is required that an appropriate survey is created for each relevant trainer.

  When creating a course, it can be selected whether a survey is to be created for each trainer (e.g., everyone with the role "Course tutor").

• The should be a semantic markup in the titles of automatically created surveys, so they can be distinguished in overviews.
  The markup could look like this: <<Current study term>><<Subject>><<Trainer>><<Course of studies>><<Study term of the subject>><<Group (in course)>>
  Example: "Summer term 2013, Basics of Economy, Dr. Doolittle, M.B.A 1st term, Group 1".
  The following have to be retrieved for that purpose:

  Piece of information	Origin of data	Comment
  Study term	Course	Can be realized via extended metadata for courses
  Subject	Course	s. above
  Trainer	Course	Can be retrieved from members of local roles Maybe it is necessary to separately retrieve username and realname
  Course of studies	User (learner)	Can be realized via a user-defined field in user accounts
  ...

• An activation period for surveys should be set automatically.
  This could be calculated depending on the activation period of the course or be controlled by a central administation setting (end of current evaluation period).

1.2.3 Permissions of Target Groups

Target Group	Permissions
Students	Anonymous participation View own evaluations in an overview
Trainers	No participation possible No edit permisison in surveys (even if user is course administrator or tutor) View evaluations about them in an overview (however, only after the survey has exceeded its activation period)
Deans (or other types of supervisors)	View evaluation results on all courses and trainers of their department

1.2.4 Promoting Evaluation Surveys

One crucial issue in academic evaluation is a high participation rate.
So, ILIAS should offer comfortable access to those surveys and remind the user insistently:

• All evaluation surveys relevant for a user could be displayed in a separate view, e.g., an additional dialogue of the personal desktop.
  The surveys could appear directly in this view: On the left a navigation frame with surveys to be taken while in the right main frame the questions appear.
• A flashy reminder could be displayed on top right of the screen when evaluation surveys are to be taken (similar to the display of chat invitations).

1.2.5 History Function

All evaluation results have to be documented "forever".
So, the must be a historization of survey data independent of whether the survey, the user, the trainer, the course still exists.

1.2.6 Calculation of Aggregated Results

• The evaluation should offer aggregation measures like arithmetic means.
  Up to now, however, ILIAS doesn't offer arithmetic means for single-choice questions (because they are assumed not to be likert-scaled).
  Most evaluations, however, will use single-choice questions, so what's needed here is an option to declare single choioce questions likert-scaled.
  This might be done in the individual question or in the survey as a whole.

• Minimum number of participants:
  Data validity often requires a certain volume of cases. So, surveys should automatically excluded from all calculations and overviews if a specific number of survey participants hasn't been reached by a specific date.

1.2.7 Results Overview

(Aggregated) results of the evaluation should be visible to a user depending on his/her permissions.
 

• Overview of individual and aggregated evaluation data is desirable.
  Aggregation could be made for:

  • one course
  • one trainer
  • student properties like "all students in their 2nd term"
  • other metadata like "course of studies", location, department (course might be assigned to several locations, departments ect.!)

• The results overview should make it possible to compare results of different units like:

  • How is the judgment of trainers in subject A compared to subject B?
  • How is the judgment of trainers in subject A compared to each other?
  • How is the judgment of trainer A in subject B compared to his judgment in subject C?
  • How is the judgment of trainer A in subject B (that he teaches for different courses of study) in comparison of those courses of study?

• There should be an export and print function for all views of the results overview.

2 Status

• Scheduled for: Not scheduled yet
• Funding: Required
• Development: Feature is to be developed by tbd

3 Additional Information

• If you want to know more about this feature, its implementation or funding, please contact: bromberger (at) qualitus.de

4 Discussion

Zenzen, Enrico [ezenzen], 16 SEP 2022: This request no longer fulfills the requirements of the Feature Wiki. In consultation with the maintainer I change the status of the feature request to "Redundant / outdated". If the request is still relevant, please update template and mockups.

5 Implementation

...