Tester Guide

Which Tools are used to Test ILIAS?

Our Tools

Four platforms and tools are most important for testing ILIAS:

  • Test Installation: Here you can test the next version of ILIAS in all its beauty (and with all its bugs) → Direct Link to Test9.
  • TestRail: Used to manage all our test cases. A test case indicates which actions a tester should perform on the test installation - and which results is to be expected. If everything goes smoothly, the tester can tick of the test case. → Direct Link to TestRail.
  • Mantis: And when it’s not going smoothly, Mantis enters the stage. On Mantis we report all issues and bugs that are discovered during testing. → Direct Link to Mantis.
  • Feature Wiki: Here you can learn about all new features and find links to the respective test cases → Direct Link to Release 9 on the Feature Wiki.

Test Installation

The ILIAS Society provides a new test installation for each major release of ILIAS. This installation gets automatically updated at night, so that it always reflects the current state of development. All basic settings and configurations are made by the ILIAS Society in order to assure flawless functionality. Apart from the testers, also all maintainers and deveopers have access to the test installation.

  • You can find the test installation for ILIAS 9 under the following URL: https://test9.ilias.de/.
  • Account for test installation: In order to access the test installation, you need an account.
    • New testers: If you have never registered yourself on earlier test installations, you can create an account here.
    • Testers who tested ILIAS in earlier years: If you had an account for one of the earlier test installations (test8.ilias.de, test7.ilias.de, and so on), you automatically have an account for the current installation. The database of the last installation with all user accounts is always transferred from one installation to the next.
  • Role TESTER: All official testers need the global role TESTER in order to access all information and to get all required permissions in the Test Area. Please check your assigned roles in your Personal Profile and contact the test manager in case you need further permissions.
  • Test Area: On the test installation there exists a proper category where you can test the different modules. It is called "Test Area" and contains one sub category per module which can be used to create as many objects as required. Official testers should use this category for their tests.

In order to assure a safe and controlled testing process, it is required for all testers to use the official test installation. Usage of a different installation for the community testing is not allowed.

TestRail

Since the development of ILIAS 5.0 we have been using a test case management software. This proprietary software is called TestRail and a license is provided to ILIAS Society free of charge by the developer. TestRail runs on the Society server under http://testrail.ilias.de. Official testers can get an account by contacting the test manager.

A test case presented in TestRail

Using TestRail has several advantages for the testing process of ILIAS:

  1. The structure of a test case is exactly what we need. Each test case has three main parts:
    • Preconditions: Any preconditions that have to be fulfilled in order to execute a test case, i.e. specific permissions, a certain configuration or the execution of a previous test case.
    • Steps: The exacts steps and clicks necessary in order to execute the test case.
    • Expected Results: What should ILIAS do or show after the last step? If this result is reached without any issues, the test case can be ticked off as a success. In case any problems or errors are discovered, a bug report must be filed.
  2. TestRail is connected with our bug tracker Mantis. Testers can create a Failed Status in TestRail and push the status as a bug report to Mantis. The report automatically includes a link to the respective test case. The status of the bug report can be seen from within TestRail.
  3. Several testers can test one module at the same time without any hassle or limitations. Each tester can see which test cases already have been executed and which results have been recorded.
  4. The current test status of each module gets visualized automatically and provides a transparent overview of the state of things.

Mantis Issue Tracker

All bugs that are discovered during testing must be reported in our central issue tracker Mantis under: http://mantis.ilias.de/ - In order to report a bug, a Mantis account is required. You can set it up easily by clicking on the link "Signup for a new account".

Filing a report is possible from within TestRail. This is the fastest and easiest way to file a bug report during the testing phase. Please note that simply documenting a bug in the TestRail result is NOT sufficient!

All other kinds of reporting a bug - be it by email, phone or other means - are NOT allowed. This is due to our intention of maintaining full transparency on the bugfixing process. Further information on how to report a bug can be found in the chapter Reporting Bugs and Follow-Up.

Feature Wiki

Before a new feature gets developed, it gets drafted and discussed in our Feature Wiki. As a consequence, the feature wiki entries are always a good source of information on new features.

Since ILIAS 5.1, all new features need to come with their respective test cases in order to make it to the ILIAS core. In each Feature Wiki entry, you can find links to the new test cases in TestRail.

You can find the Feature Wiki here: http://feature.ilias.de



Information Message
No comment has been posted yet.