TestRail for Authors: Creating and Organizing Test Cases

How to Create Test Cases

One Test Suite per ILIAS Module

All test cases should be created directly in TestRail. For each module, TestRail provides one test suite (for example: Blog, Test&Assessment, Portfolio, and so on). This test suite contains all test cases for this module.

The name of the test suite is in agreement with the official module names that are also being used in our bug tracker, Mantis. For this reason, it is only allowed to use exactly one test suite per module. If you as an author would like to use a more granular organisation for your test cases, you should split your test suite into sections and sub sections. These sections can be created as required for each part of a module.

It is recommended to name sections and sub sections in a way coherent with the software. For example, you could call a section according to a tab ("Settings") or according to a specific workflow in ILIAS. This sections then contains all related test cases.

It is easy to move test cases around. If you feel that you would like to update or improve your organisation within a test suite, feel free to go ahead.

Structure of a Test Case in TestRail

TestRail's test case creation screen does contain a few areas that are not used for testing ILIAS: The relevant areas are the following ones:

• Title: Here you should enter an apt and concise title, in order to identify desired test cases easily in the overview of the test suite or test run. It is recommended to name test cases in accordance with the ILIAS terminology.
• Preconditions: Here you need to list all preconditions that are required in order to successfully execute the test case. Often, these preconditions concern permissions and roles (for example: "Tester must have the ADMIN role"), settings or the execution of preceding test cases.
  • Hint: In order to link to another test case, simply put its ID into square brackets, for example: [C454]
• Steps: All required steps to execute the test case, numbered consecutively.
  • The first step is always the starting point for an action: Where does the tester have to be in ILIAS, in order to execute the subsequent steps?
  • Steps can alternate as required: User action - reaction of ILIAS - user action - reaction of ILIAS, etc.
  • It is possible and recommended to treat several variations of an action within a single test case. Please make sure, though, to always list variations explicitly in your test case.
    • Example: "Variation: Before saving your data, click on the button "Cancel"."
• Expected Results: Here you should precisely describe what should happen in ILIAS after the execution of all steps, in order to know if the test case can be set to "passed".
  • State what ILIAS has done: "User has been created and received an email" or "The forum post has been deleted".
  • Organise your result list using bullet points.
  • Also list expected results for your variations.

The most important thing when writing your test cases is that they should be intelligible also for non-experts!

Formating Test Cases using Markdown

If you would like to format your test cases, you can use Markdown. A few examples:

• *italic*
• **bold**
• * Bullet list (only works in a new paragraph)
• 1. Numbered list (It is also possible to begin lists with higher numbers)
• # Headline
• ---- Dividing line
• [C343] Link to another test case

Language and Sources of Inspiration for Test Cases

Your own experience with problems and the analysis of Mantis bugs are always helpful in order to write good test cases and reach a decent coverage. You can also have a look at the official ILIAS user documentation and the Feature Wiki in order to learn more about a module.

As most ILIAS testers so far come from German-speaking countries, German is used for many test cases. Since the test of ILIAS 5.3 it is also possible to maintain a test suite in English. This allows members from the international community to get involved more easily. It is NOT allowed to maintain test suites in mixed languages, as this would lead to chaos too easily.