Feature Wiki

Information about planned and released features

Tabs

One Competence per Question & Raw Points

Page Overview

[Hide]

1 Summary

The Complexity of the assignment of competences to questions should be reduced. This means:

  1. Abandon the Assignment of one question to multiple competencies: Each question should be assigned to exactly one competence 
  2. Instead of having a separate "Competence points by question result/solution compare" competence point assessment, the raw points of each question should be used directly as points contributing to the assigned competence
  3. Several Questions can still contribute to the same competence
  4. Competence Thresholds are defined in absolute points instead of percentages ("from n points" per threshold)
  5. The required amount of questions for competence trigger is being abandoned: Abandon «Required Amount of Answers for Competence Triggers»
  6. The max points that define a competence are clearly visible and shown in a data-table. This data-table will move one level up since the panels showing the competences a question writes into are not needed anymore.

2 Problem Statement

  1. In ILIAS tests an individual question (items) can be assigned to “pay into” more than one competence. Without evidence for multidimensional measurement, competence-specific scores derived from shared items are very difficult to justify from a validity point of of view. Additionally this comes with some complexity to the code.

    screenshot
    Test > Questions > Questions/Competence Assignment
    Questions

  2. Assignment Properties "Evaluation of Competence Points by Question Result" and "Evaluation of Competence Points by Solution Compare" is to be abandonned since it is deemed untestable and very difficult to understand. 

    Screenshot
    Different Assignment Properties and Competence Points in Table
    Screenshot
    Assignment Properties "Evaluation of Competence Poitnts by Question Results" with Competence Points
    Screenshot
    Assignment Properties " Evaluation of Competence Points by Solution Compare"

    3. The percentage thresholds are supposed to be abandoned since they make it very difficult to understand how a competence level is written: When defining the percentage thresholds there is no clear indication at all on what is required to reach a certain competence level. Competence administrators need to calculcate by hand how these values can be mapped to points. This makes it very hard to test, too. Instead an "up to x points" threshold-management is supposed to be established (streamlined to the survey). 

3 Concept

This concept was discussed and agreed with 12 community members attending a conceptual workshop of the SIG Portfolio & Competence Management. 

  • Instead of assigning multiple competences to a single question ILIAS must allow exactly one question to be assigned to one competence. This is also needed because the migration to raw points will not allow anymore to split different answer options to different competences.
  • Instead of Competence Points the raw points of a question are used directly to pay into the competence. Weighting options for questions need to be done in the question itself (via the raw question points).
  • Still several questions can be assigned to the same competence. In the Competence Threshold subtab, the accumulated raw points will be presented 
  • Competence thresholds are set via "up to x points" per competence level in tests (please compare to the survey's solution).

Concept from a didactical perspective:

The simplification makes competence-based assessment clearer and more transparent:

  • Each question is assigned to exactly one competence.
  • This creates a clear chain: Question → Competence → Raw Points → Competence Level.
  • Multiple questions can still assess the same competence, providing more reliable evidence.
  • Separating raw performance from its interpretation through thresholds makes the assessment easier to understand.
  • Overall, the model becomes simpler, more transparent, and didactically clearer.

4 Obstacles

There are few difficulties concerning the migration from the old concept to the new one. These are:

  • Old assignments need to be dealt with
    • If a question is assigned to multiple competences, then there are three options: a) remove all competence assignments, b) Remove all multi-assignments, c) keep all single assignments and one (random) multi assignment. The amount we are dealing with here can be queried per installation via sql. This could be done before making a final decision.
  • Do we want to keep the assignment in the qpl
    • The assignments in the qpl should be kept: When adding a question from a qpl to a test, then it is copied. If there shall be no competence assignment in the test then it can be removed there
    • Alternative (and probably the preferred) option as a prerequisite for the new question service: Only allow to assign question to competences from within a test.
  • Handle Percentage Thresholds
    • The percentage thresholds were introduced due to the random test option. We want to remove the possibility to use competences in random tests. This would be a prerequisite to switch to absolute/raw points
    • Migration of percentage thresholds:
      • Option A: Our proposal is to simply remove those. (Fallback if the sql queries or the community does absolutely want to keep these, then we might consider calculating from the old percentages to absolute values. This is probably not clear to handle, thus a removal is the better way). Wish: There needs to be a hint that the threshold values are not implemented (anymore). The tests that are affected can be listed during a db update steps or similar.
      • Option B: Calculate values via rule of three. This works if the competence assignments cannot be used anymore in random tests. A problem is the following: The old implementation allowed to write percentage thresholds during the test, i.e. you could reach the highest competence level when only answering 3/5 questions that write into this competence.

5 User Interface Modifications

5.1 List of Affected Views

  • Test > Questions > Question/Competence Assignment
  • Test > Settings > Competence Thresholds

5.2 User Interface Details

Option 1: Test > Questions > Question/Competence Assignment: A datatable is supposed to be introduced in the sub-tab "Questions/Competence Assignment" with the columns "Question", "Competence", , "max points", "Actions". The Action is "Assign Competence" for questions without assigned competence and "Remove Competence" for questions with an assigned competence. The panel on this screen is not needed anymore since it only shows which question has which competence(s) assigned. 

Simple Datatable for Simple Assignment
Select exactly one competence per question

Information: If a question has no competence assigned then it still is shown in this table but the "competence" column is just empty.

Test > Settings > Competence Thresholds: Raw Points are assigned to competence Levels.

Assigning Raw Points to Competence Levels

5.3 New User Interface Concepts

No new user interface concepts

5.4 Technical Aspects

{ Necessary technical information have to be provided here, e.g. dependencies on other ILIAS components, necessary modifications in general services/architecture, potential security or performance issues. }

6 Non-Functional Implications

6.1 Security

{ Does the feature include any special security relevant changes, e.g. the introducion of new endpoints or other new possible attack vectors. If yes, please explain these implications and include a commitment to deliver a written security concept as part of the feature development. This concept will need an additional approvement by the JourFixe. }

6.2 Privacy

no impact

6.3 Accessibility Implications

just Kitchensink

7 Process Information

7.1 Involved Authorities

  • Authority to Sign off on Conceptual Changes: {Please add related profile link of this person}
  • Authority to Sign off Code Changes: {Please add related profile link of this person}

If this request is related to multiple components, please list both authorities for all related components.

7.2 Contact

Person to be contacted in case of questions about the feature or for funding offers:  {Please add related profile link of this person}

7.3 Funding

Funding status and funding parties are listed in the block 'Status of Feature' in the right column of this page.

If you are interested to give funding for this feature, please get into contact with the person mentioned above as 'Contact'.

8 Discussion

Kergomard, Stephan [skergomard], 16 SEP 2026: I'm completely fine with removing this, just one comment: It is completely and utterly untrue (and you should remove it from the reasoning) that assigning multiple competences to a question poses somehow an issue. But for this you need to be able to weight the impact of the result. Taking an example I'm comfortable with: You have many different systems to measure the competence to act in intercultural situations, but many of them have common loading factors. If you can weigh how much a result weighs on the scale, you can very well reuse a question across multiple such systems, thus simplifying maintenance. Again, I think it is reasonable to do as you suggested, I would just ask you to make a clean argument and clearly formulate that with this change we are removing this completely valid option.

Falkenstein, Rob [rob], 18 SEP 2026: Thank you for this feedback. We think this change makes sense for two different reasons: a) The change to raw points would increase the difficulty to write into more than one competence because all questions that write into a competence would have to consider their reachable and max points in order to do proper weighting. b) The abandonment of this option will make it easier to handle the assignments and should allow to remove the panel in "questions->question/competence-assignment" and instead move the data table from "manage competence assignment one level higher and reduce the amount of clicks needed. We made this change clearer in the Chapter "Summary" and in the Chapter "Concept"

Strassner, Denis [dstrassner], 29 SEP 2026: Dobosz, Nicole [nicoledobosz], Falkenstein, Rob [rob], Joußen, Thomas [tjoussen], Samoila, Oliver [oliver.samoila], and I had an informal meeting about this feature request today. I agree with the general movement of this request and thank all participating persons for their work on this.
There are open points which are need to be discussed with the community or cleared within the next months before we can bring this to the JourFixe.
But Nicole and Rob know about them and will be coordinating the work to bring this request to a stand, where I can change the metadata to "Ready for JF".Strassner, Denis [dstrassner], 29 SEP 2026: 

9 Implementation

Feature has been implemented by {Please add related profile link of this person}

9.1 Description and Screenshots

{ Description of the final implementation and the changed behaviour in the related components. Please add screenshots to visualise the changes if possible. }

9.2 Test Cases

Test cases completed at {date} by {user}

  • {Test case number linked to Testrail} : {test case title}

9.3 Privacy

Information in privacy.md of component: updated at {date} by {user} | no change required

9.4 Approval

Approved at {date} by {user}.

Full Screen

Last edited: 30. Sep 2026, 14:34, Falkenstein, Rob [rob]