Show Advanced KnowledgeHide Advanced Knowledge

Introduction of Audience Response System

The concept can be downloaded as PDF:

1 Introduction

Educational institutions often use hardware-based Clicker Systems in class to get immediate feedback from an audience. These systems are very expensive and require installations in class rooms. Every new room causes again more costs.
 
Smaller institutions can’t afford such systems. But nowadays most students bring a lot of mobile devices such as laptops, tables and cell phones in class. These devices can be used to communicate with a Clicker System built in in ILIAS.
 
Clicker Systems are just one way to get feedback or inputs from the audience. Today Audience Response Systems enabled lecturers to get feedback, conceptual input and differentiated information about the level of knowledge or student opinions. These systems are often web-based in combination with a mobile user interface and/or dedicated apps.
 
To build the bridge between content and usage of an LMS and the lecturer in front of the audience, being ILIAS one of the most important Open Source LMS, should provide a multi-functional audience response system. Other Open Source LMS do not yet have such systems but there are already discussions concerning the topic.
 

1.1 General usage of a ARS

Although ILIAS offers a wide range of interactive learning tools, in connection with large scale lectures it is still often kept in the background during the actual lecture. In in-class situations, ILIAS is mostly used to provide additional information to a topic, and offers possibilities to learn, to practise and to check the state of knowledge of the learners afterwards.
The dimension of the audience can vary between a handful of students and hundreds of people. At the moment, the LMS ILIAS does not show up in lecture theatres and live teaching situations but rather appears on the students’ computer screens when they are studying in the library or at home. We want to change that.
 
With an Audience Response System ILIAS could set foot into the auditoriums themselves. The presenter would be able to  get instant feedback from the audience  regarding opinions, remaining questions, upcoming ideas …

• immediately and right at the spot
• easily
• straightforward

Such a system should even provide a way to get feedback from an audience that comprises both students and external people who might not have an account on the system. Even in a dynamic, live lecture situation having 300 students log on before they can answer a spontaneously posted question would be a hassle and would impede the didactic effect of the action.
 
However, even in smaller settings with only a few students or workshop participants, an ARS tool would be a valuable e-learning supplement which would open an entire range of new LMS applications for in-class use.
 

1.2 Didactic scenarios

„Please take a minute and submit your questions to the current topic. We will discuss some of them afterwards.”

During the lecture, the professor wants to collect questions concerning the current topic. Students can submit their questions using an SMS, all the questions pop up on the screen of the professor. The professor can then pick some of the questions and discuss them in class.

"Does ILIAS have the better Audience Response System than Moodle? Please vote yes or no.“

During the ILIAS Conference the Product Manager asks the audience a provocative question and wants immediate feedback from hundreds of listeners.

„Submit the topics you’d like to put to the agenda…“

Collect topics for meeting, even from the outside.

„How many of you will attend the next lecture of our University of the Third Age?”

A “University of the Third Age” or lectures of a “Studium Generale” programme usually do not issue campus-accounts to the their listeners. To use an ARS with this audience, the system must provide an easy access for anonymous users.

2 Requirements

The ILIAS Audience Response System functionality should…

• be easy to use
• support multiple devices and ways to communicate with them
• have an instant access functionality without requiring a login
• have a presentation view for the (live) results
• have a mobile and desktop version of the voting-view
• allow different input types

 

2.1 Types of User-Inputs

2.1.1 Cipher-based

Clicker Systems often have a lot of keys (A-Z, 0-9, Yes/No, …) to allow users to choose their input depending on the question. To keep it simple, most input types can be reduced to a cipher which stands for a choice. e.g.:

Question: When was Albert Einstein born?
Possible Answers (SC):         A: 1886       B: 1897       C: 1875       D: 1879       E: 1881

Question: In which cities lived Albert Einstein?
Possible Answers (MC):         A: Bern       B: Leipzig    C: Köln       D: Princeton

Question: Has A. Einstein ever received the Nobel Price?
Possible Answers (SC):         A: Yes        B: No

2.1.2 Value in range

Sometimes you have to get a value from a range, e.g.:

Question: How often did you read the Word “Einstein” in http://de.wikipedia.org/wiki/Albert_Einstein
Possible Range:                300 --|--|--|--|--|--|--|--|--|-- 400

2.1.3 Free input

Or you want to collect inputs like questions to discuss them afterwards, e.g.:

Question: Please post you question here:
Possible Input: [____________________]

2.1.4 Order

In this specific case you’d like to get an order of things. There can be a correct solution or there might be none (a.k.a. prioritization):

Question: Order these events in life of A. Einstein chronological:
Events:                        Birth         Nobel Prize   Citizen of Switzerland       Berlin Studies

Question: Set your priorities from low to high for topics in the next lecture:
Possible Topics:               Switzerland

2.2 Access

2.2.1 ILIAS RBAC

Logged in  ILIAS users should be able to access an ARS item in the way they are used to. As an object in the ILIAS repository it makes use of the ILIAS RBAC for protection against unauthorized access.

2.2.2 Token-based

Often an audience response action  has to be available very spontaneously (lecturer asks question and needs an instant feedback).  Forcing the audience to log in first and to navigate to the respective ARS item within a certain context in ILIAS would take far too long in a live setting and would break the didactic flow of actions.
 
A token-based access (e.g. using a short PIN) is much faster. The token-access can be disabled for each object. If enabled, the ARS item can be accessed without a login. All users who know the token can vote. We need:

• A simple triage-page for token-access-enabled ARS items[mg1]
• The token-based voting-page for a token-access-enabled ARS item.

The second page could also be accessed directly by adding the token to the URL (…/vote/{TOKEN}).
 
Both pages must be highly responsive and can be accessed with mobile devices.

2.3 Multiple pages for one ARS item

It should be possible to add several pages or questions (ARS items) from different input-types to one ARS object. These can be presented in a pre-defined order.

2.4 Presentation

2.4.1 Voting-Pages (Read Access)

Token-Access

The simple triage-page needs some branding (ILIAS logo and installation name) but not the whole standard-template. Same goes for the voting-page.
The voting page only shows the currently active question and is “alive”. Whenever the presenter changes the question, the voting page automatically changes, too. If the presenter stops the current question, the voting page also changes to a waiting page.
 

ILIAS Context

When log-in users at least have read-access in the ILIAS context of the ARS object, the object renders the voting page within the content area of ILIAS. There is always just the currently active voting available.

2.4.2 Presentation-Pages (Write Access) and workflow of a presentation

The presentation of a clicker session is divided in two steps per question.

Step 1: The question

Step 2: Presentation

Each action of the presenter has an effect on the screen of the mobile devices or the view of the logged-in ILIAS users which makes the presentation page seems “alive and breathing”. When the presenter starts with a row of questions, the users can access the voting page using the token or the QR-Code (other access-methods are possible, such as SMS):

When clicking [>] right to the next question, all logged in devices change their screen to the corresponding voting mask for the question.
When the presenter closes a question, the screens on the logged in devices change their screen to a waiting mask (or a finishing screen, when the last question of an object has passed).
 

2.4.3 General

Bootstrap
All GUI elements should use Bootstrap as far as possible. Here are some Examples:

Mobile presentation
Every voting page and token access page should be highly responsive to be fit for mobile usage.
The Presentations-Pages need less responsiveness since they are usually presented on larger screens.

2.5 Future Prospects

There are many possibilities to build more question types and workflows in the ILIAS Audience Response System, some of them are listed here:

• marking correct/false answers to give students feedback on the quality of their choice
• give additional (textual) feedback depending on the answer
• submit images
• upload images as answers (e.g. select the 42nd President of the USA from these four pictures)
• ARS as a Page Editor element
• Matrix-questions and virtual sticky dots (this would be a great tool for workshops):

3 Additional Information

• Idea / concept: Fabian Schmid, sr.solutions, fs@studer-raimann.ch
• Funding: Universität Bern, Universität Freiburg, sr.solutions, FH Dortmund
• Maintainer: Fabian Schmid
• Implementation of the feature is done by sr.solutions
• Tested by / status: (name, e-mail), (status information set after implementation)

4 Discussion

27.03.15, Pascal: we highly appreciate this feature but ask you to closely work together with the kitchen sink project for the layout.

30.03.15, K.V.: We highly appreciate this feature. It is a chance for ILIAS to get another didactical powerful feature to the core.

JF 13 Apr 2015: We highly appreciate this feature and schedule it for 5.1.

• We would like to have a good name for the object that is added to the repository (ARS is considered as category of type - but you do not add an ARS to a course or category). Any suggestions are welcome.
• Please write a short technical concept on the authentication and necessary changes in the ILIAS init procedure (send this to Stefan Meyer).
• Question editing should be reused from the T&A component (similar to learning modules). Please also check with Björn Heyser, if result storage could be also done in the T&A. If necessary, funding for a proper interface in the T&A may be necessary.

JE 2015-04-29: Will existing LiveVotings be updatet to the new ARS object?

HJL, 18. April 2015:

JF said: Question editing should be reused from the T&A component (similar to learning modules). Please also check with Björn Heyser, if result storage could be also done in the T&A. If necessary, funding for a proper interface in the T&A may be necessary.

The actuel Live-Voting-Plugin and also the suggested AR-System have relations with the survey and the polling tool in ILIAS. The difference is, that ARS could replace the existing hardware voting systems. But it is still a system where the teacher ask a question, and students answer to this question. See also Polleverywhere: "Poll Everywhere lets you engage your audience or class in real time. Type your question into our poll creator, select the kind of poll you want to make, and press “create.” That’s it. Your poll is ready to share with participants!"

In general, it is a great idea to streamline the functionalities and reuse, (it is a really good idea to have the possibility to reuse TA-questions in a ILIAS-Learning module). But it adds additional complexity to the development process of this new feature. Questions would be:

• Why the AR-System should use the T&A-component, why not the Survey-component?
• Shouldn't the Poll-object resuse the TA or Survey-component, too?
• As I heard from Fabian S., he and Björn H. think new service for Editing, Saving and "Voting" would be necessary, this service should be available for TA and ARS. Who is willing to pay the additional costs for such a general service?
• ARS should be a simple tool. Reusing existing code should not add additional complexity for teachers, ARS should be as simple as the concept describes.
• This process is time-consuming, we prefer a implementation for 5.1.

Killing, Alexander [alex], 18 May 2015: @HJL: The examples for the different question types given on this page (under 2.1) all have an "assessment" character. This may have confused the JF participants. Personally I think one of the most important (at least technical) differences between T&A and survey questions is: Is there a "correct" answer (yes, in T&A). This is the case for the examples on this page (under 2.1).

Survey questions do not have a correct answer - a fundamental difference. Should the ARS show the correct answer (when was Einstein born)? Should it show how many users have been right/wrong?

A) Quote: "In general, it is a great idea to streamline the functionalities and reuse, (it is a really good idea to have the possibility to reuse TA-questions in a ILIAS-Learning module). But it adds additional complexity to the development process of this new feature."

Why do we add complexity here? The opposite is true. We would have a lot of more code, if I would have started to write my own question editing in the learning modules and not re-use the existing implementation. This is especially true for future maintenance costs and also for user experience (question editing looks the same in both components).

B) Quote: "ARS should be a simple tool. Reusing existing code should not add additional complexity for teachers."

This tool will not be simple. This proposal already includes four question types, a token-based authentication and a presentation and request handling that is quite unique in ILIAS. Quite a different class of complexity than the poll component. Why should reuse of of the question editing make things additionally complex for teachers? You are not forced to use question pools in learning modules, for example.

C) Quote: "This process is time-consuming".

Why is that? The re-use of question editing in learning modules shows that the general code base is already there. Having not to implement question editing in the ARS component should save time. The re-use of result storage from T&A has been an "optional" demand by the JF. But even if a redundant implementation is already there - Should we really rush into this on the costs of no code re-use, increased future maintenance, yet another user experience for question editing and no re-use of existing functionalities (question pools)?

HJL, @AK:

In Polleverywhere and in the Live-Voting-Plugin, the following question-types are questions without a solution
- 2.1.1 Cipher based (Single Choice, Multiple choice)
- 2.1.2 Value in Range
- 2.1.3 Free Input

The question type: "2.1.4 Order" could be a question type without solution (Polleverywhere), but could also be a question type with a defined solution. And ILIAS should shows only two "elements": how much students voted for the the right solution, how much for the wrong solution.

The examples with assessment characters are not good examples for a AR-System, in my opinion. Better examples:

http://www.polleverywhere.com/blog/six-ways-to-use-free-text-polls/
http://iheartedtech.blogspot.ch/2013/11/using-polleverywherecom-for-formative.html

Of course, Cipher based questions in educational context have assessment-character sometimes, eg.:

Bei Gedeihstörungen bei Kleinkindern, an was denken Sie?
- A: Colitis ulcerosa oder Morbus Crohn
- B: Infektiöse Diarrhoe
- C: Laktoseintoleranz, Zöliakie

So the Cipher based question can be used as a formal assessment system, too. But in general, teacher want to use a AR-System to get an overview of all opinions, or to see what`s the actual knowledge of the audiance.

>Is there a "correct" answer (yes, in T&A). This is the case for the examples on this page (under 2.1).
So the answer is NO AND YES. In general, there is not a correct solution, except for Order, and implicit in Single-choice question.

>Should the ARS show the correct answer (when was Einstein born)?
Actually, Live-Voting and Polleverywhere don`t. ILIAS should show the correct answer for Order-question. Maybe there are other needs.

>question editing looks the same in both components
>Why should reuse of of the question editing make things additionally complex for teachers?
I agree, in General.

But we should distinguish between the form od single-choice question in TA or Survey and the form of SC-questions in ARS (or Live-voting). For example: Description, Tiny-Editor, Single-Lines-Answers/Multiple-Line-Anwers, Working time (per question) are not necessary when a person edits a single-choice question in ARS, because these fields would add additional unnecessary fields to the User-Interface.

FS should comment this: Why the JF-comment adds additional complexity for the development of the ARS?

>C) Quote: "This process is time-consuming".
>Why is that? The re-use of question editing in learning modules shows that the general code base is already there. Having not to implement question editing in the ARS component should save time. The re-use of result storage from T&A has been an "optional" demand by the JF

OK. Thanks for clarification:
- that the reuse of TA-questions should save time (and budget)
- that the re-use of result storage from T&A is an optional demand.

5 Implementation

...