Show Advanced KnowledgeHide Advanced Knowledge

(Project) Component Revision

1 Aim of Project

ILIAS is composed of various components that need to integrate with each other to make the system work. Besides the components that are delivered with the ILIAS core, there are components, the plugins, that can be added on installations on demand. The modular structure of ILIAS is of high importance for the system for several reasons:

• a modular structure allows more people to work on the system without interfering
•  a modular structure requires to focus on interfaces over implementations, which is good for the maintainability of the system
•  a modular structure allows to add and remove components easily

The topic "Adaptable Learning Environment" from the Strategy 2019 of the TB mostly speaks to the same goals.
The current state of the art in the implementation lacks behind these goals and the possibility:

• there is a Service `Components` but it is only one mechanism among many to integrate components
• core components and plugins, though similar in theory, are integrated rather differently
• integrations between components are rather explicit in many locations

One can surely point to other problems that can be attributed to lacking integration mechanisms for components.

The Component Revision 2022 aims to improve that state of the art by achieving the following objectives.

1.1 Modernize Services/Component

Based on its naming and content, the folder `Services/Component` seems to have the goal to provide the rest of the system with mechanisms to integrate various components with each other. In the current state, this goal is not fulfilled. Besides the lack of functionality, where the aims are outlined below, the implementation of the functionality in `Services\Component` could use some modernization

• it (partially) lacks tests, even after the latest refactoring
• its relations to other components, such as language and learning progress, feel adhoc
• the parsing of information from components (core and plugins) is different
• the parsing of information from core components relies on ad hoc xml
• it seems to be missing useful base classes and interfaces for core components
• the integration mechanism currently provided seem to be lacking crucial functionality, since so many alternative mechanism are in use

The aim of this project, hence, is to overhaul the implementation of Services/Component completely. This implies that components and plugins alike will need to move in the upcoming releases.

1.2 Clarify Role and Implementation of Dependency Injection and Initialisation

Access to functionality from other components is a core requirement that a component framework needs to deliver. This strongly relates to the question how components depend on each other and how the system is initialised. To represent dependencies we have steadily increased the usage of the dependency injection (DI) in the previous years by moving from `global` to a dependency injection container (DIC). This preliminary work can now be leveraged to connect components with each other in a principled way and restructure the initialisation of the system and the internal initialisation of the components.

1.3 Clarify Namespacing and Directory Structure of ILIAS

Naming and (directory) structure is a point of huge importance when integrating a system. Various problems from this space have been shown up in the past:

• Do we want to implement a PSR-4-like namespacing? How?
• Which directories need to be exposed to the web? Where do we put our endpoints?
• Why do we put plugins in the Customizing folder, even if the implement repository objects, like modules do?
• How exactly do we distinguish between "Modules", "Services" and "src" and is this discrimination really useful?
• Where do we put tests?
• How do we disambiguate ids, for plugins and components?
• Can we clean up our directory structure to clarify the structure of the application?

This project aims to give conclusive answers to these questions and propose a strategy to move the code base to the implied state.

1.4 Align Mechanisms Used by Core Components and Plugins

Currently, core components and plugins use a different approach and implemantion to integrate with each other. This is unfortunate for various reasons:

• we need to maintain two different sets of code that both solve a similar problem
• plugins cannot be added easily to core once they are implemented as plugin
• core components cannot be easily be turned into plugins
• plugins are (mostly) one trick ponies that can only fullfil a certain role assigned by one component
• components cannot use mechanism designated for plugins to integrate with other components

The aim of this project, hence, is to reduce the integration mechanism that ILIAS uses to one mechanism that fits core components and plugins equally well. A side effect of this will be, that we will get rid of the one trick pony plugin phenomenon and gain interesting possibilities for the product management of ILIAS.

1.5 Use New Framework to Implement Existing Integrations

The aim of this projects is to replace all existing integrations between components. This, however, will not be possible quickly and will require effort for each component. Thus, this project is prone to make the situation worse by just adding another mechanism, on top of the ones that already exist. To not fail in this way, the project aims to at least replace the following integration mechanisms:

• the `InterfaceFinder` currently used by the Setup, Global Screen and UICore
• service.xml, module.xml, plugin.php, plugin.xml

Thus, we will be at least left with less integration mechanism than before. The modernized integrations for the functionality in question aims to provide a compelling example to also move integrations based on other mechanisms to the new framework.

2 Involved Maintainers and Stakeholders

• All maintainers are potentially involved.
• Everyone interested in a modular software should have stakes in this.
• Everyone using the current integration mechanisms for plugins has a stake in this.

3 Timeline

ILIAS 9	start with implementation of 1.1 start with implementation of 1.4 start with implementation of 1.5 start with concept for 1.2 start with concept for 1.3
ILIAS 10	finish implementation of 1.1 finish implementation of 1.4 proceed with implementation of 1.5 start with implementation of 1.2 start with implementation of 1.3
ILIAS 11+	proceed with various implementations

4 Related Feature Requests and Status

Feature Request	Suggested by	Funding	Planned Release	Status
Component Framework/Setup: Make Setup handle Legacy Plugins	Richard Klees	yes	10	in development
…
…
…

5 Further Results

5.1 Progress until Nov-23

• regarding 1.1: Services/Component (now components/ILIAS/Component) has been modernized in various regards (https://github.com/ILIAS-eLearning/ILIAS/pull/3901, https://github.com/ILIAS-eLearning/ILIAS/pull/4008). Data handling and storage has been modernized, tests have been created. Still there are things left to do, as infrastructure for Plugins still exists and can be removed at some point.
• regarding 1.2 and 1.3: The paper "Components and Directories" has been discussed in the community on various occassions and is approved by the JF. The directories already have been restructured according to plan, an implementation for the integration mechanism its on its way.. Much of the remaining work will need to be done inside the single components and can happen incremental and in parallel.
• regarding 1.4: The plan to do so is layed out in the paper "Components and Directories". Plugins can be provided as unified components with ILIAS 10, if everything goes according to plan, but the old infrastructure will need to be removed at some point.
• regarding 1.5: During the implementation for the integration mechanism some existing integrations will be switched to the new mechanism, but there are numerous integrations between components that need to be moved. This will be another huge chunk of work, but this also can happen incremental and in parallel.

The projected timeline hence is valid, we are en route as planned.

5.2 Progress until Nov-24

• regarding 1.1: The component has been ported to the new directory structure, as other components. The subdirectories `classes` has not been merged to `src` and also still contains untested code and code for plugins. We are also still missing an updated version of the xmls. I.e.: There is some cleanup left to do here.
• regarding 1.2: The mechanism to move ILIAS to full DI do exist and can be used. Strategies to move the system in the according directory have been investigated and are proposed with the UI framwork as example. One can already see how this will replace the current `ilInitialisation` in the future.
• regarding 1.3: The overall directory structure has been modified according to plan. There is work to do inside each component. This mostly seem to be two kinds of tasks:
  • Some components exist as duplicates, e.g. `Cache` and `Cache_`. This is not unexpected, as this is a residue from the previous Modules/Service/src-structure. Still, these components need to be consolidated.
  • Most components need to improve on their internal structure to become compliant to the agreed upon directory structure. Most notably, almost all components use a `class` folder instead of `src`. This is also expected, as this was the previous structure in `Modules` and `Services`
• regarding 1.4: Core components and plugins now can indeed use the same infrastructure in general, as the new directory structure and the mechanisms implemented for the component revision do not differentiated between different vendors in the directory structure. Still, interfaces for plugins are still separated from their counterparts for core components. Also, many interfaces for core components are not exposed via the mechanism of the component revision.
• regarding 1.5: The most notable addition to components that are moved to the new integration mechanism is the UI framework.

The project has made some progress in the last year. The most exiting piece of work is the PR to move the UI framework (already mentioned 2 times before). Thanks to Thibeau Fuhrer from sr.solutions to work on this. The PR should show techniques to move individual components and also has been shown to and discussed in the wider community. We hope it will provide enough exemplification to allow to move other components.

Most relevant work for ILIAS 11 will be to move the existing plugin integrations to the new mechanism to allow to abandoned the legacy plugin integrations. All components can move incrementally with the tactics deviced by Thibeau, be it regarding the internal structure or the integration via the new mechanisms with the rest of the system.

We would appreciate some funding to push this process, be it as developer time or money.

6 Additional Information

…

7 General Discussion

Please discuss specific questions of feature requests on the related feature wiki pages. This discussion section is only for a general discussion of the project and its realisation.

[amichels] Andre Michels 2022/02/10:

First of all. Thank you for the work you putting into such a great (in size and good work) proposal to enhance the whole ilias architecture.
I am really pleased to see such changes coming to ilias and think they will be great for the further development of the ilias core and the whole ecosystem around it.

Eventually it is a good idea not only to refactor Services/Components into the same structure but also Modules/ and src/ (soap/ and all other directories of code) so that all code is aligned on one level.
Sorry if i go tooo far with my comment and expand the whole proposal but this is the chance to change a good part of the legacy architecture.

Here are some of my ideas i got for this proposal:
I am a huge fan of refactoring all components into same weighted modules. Coming from years of typo3 development i
really liked the concept of core and plugin modules, so much i use the architecture in almost every project i have.

Here is a link to the basic idea behind the typo3 model of "sysext" (Core Modules) and "ext" (Plugins):
https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ExtensionArchitecture/SystemAndLocalExtensions/Index.html
I think this documentation has some good information about how to setup a good practical easy system, a lot of developers tested and used it in production.
Like how every module could have its own composer.json, but in the end they ship a single build of their system.

The idea behind the "ext" (Plugins) is to have all possibilities of a core module while not having to define plugin interfaces.
The pluginslots of ilias should therefore completely removed. (If that is still a thing)
To archive this all functionality like defining the current "UIHooks" or creating a "Cronjob" should be moved into a
configuration file which then register s the new functionality. Basic example:

Configuration.php:

```php
$this->registerGUIClass(ComponentTableGUI::class); // would do some ilCtrl stuff
$this->addUiHook(ComponentUiHook::class); // adds the old UIHook concept to a list which then is parsed like now
$this->addCronjob(ComponentCronjob::class); // adds the class as a cronjob like the cron pluginslot
$this->addRepositoryObject(CompontentObject::class) // adds a repository object to the list which currently holds all objects
```

The Configuration.php will hold some variables and configuration that is needed for a module. A Configuration.php should be mandatory.
There could be different files for different tasks. Another file could be a "di.php" which holds the dependency injection configuration and can manipulate this as well.
This will properly match up the functionality provided in the legacy fields you mentioned:
service.xml, module.xml, plugin.php, plugin.xml
(I dont know which hidden features each file has, but i think they are all needed for a module and could be merged into a single configuration file ala "code as configuration")

An example to replace the interfaceFinder here is the same as proposed some lines before:

```php
$this->promoteScreenProvider(ComponentScreenProvider::class);
$this->addSetUpAgent(ComponentSetupAgent::class);
```
This will result in static analyse able code overall modules living inside the project. Furthermore, there could be functionality
that modifies core functionality and overwrites already defined e.g. ScreenProviders.

To make the migration more easy typo3 has some old modules containing the legacy code. In these there are some config files which maps all functionality
to the new system. Over the time (or not) some features are moved into their own modules. This will make this huge task at least viable.

Another point (i dont know if this goes to far for your original proposal) i see here is that ilias will need a new hook system.
Currently, the pluginslots, "Hooks" and EventListeners (at least the registration of these) are quite mixed up.
Looking on UIHooks its used for 3 things. Manipulating a html and manipulating a redirect in the getHtml method.
There is also a event system which is used for e.g. `update_learning_progress`.
A new Hook System will quite good if the overall structure is changed, but this could be postponed. Hooks (any kind) could be registered like above.
-> See any Event Hook System from e.g. Symfony/Typo3

This reads like a lot of work... but it isnt. Its quite doable if done in some steps to give all developers enough time to migrate.

Implementing a new system besides another:
To give developers some time to migrate throw deprecation warnings and to make the migration easier the new system should be implemented in a way that
is described in the error message itself. This goes beyond a @deprecated tag.
See this link:
https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Deprecation/Index.html#deprecate-functions-in-extensions
So another system would be ok, if developers have atleast one major version time to migrate. Also errors in the core could be logged and fixed if they arent a regular request.


Again, i hope this isnt too far away from your original proposal. If you want to talk about that or some ideas of this i am very open to discuss it with you. Just ping me.

Klees, Richard [rklees], 2022-02-11:

Hi Andre,

thanks a lot for the input and material (and the cheering).

Your example code looks a lot like what I had in mind, so: Yes, this is what I think we should head for.

I currently cannot see how exactly a directory structure should look like, but I surely agree that 1.3 is not only about `Services/Component` but about the directory structure of the complete system. I will surely have a look into how Typo3 is structured.

Your idea regarding legacy code (put it into a special module) also came to my mind, but I'm not sure at this point how feasible this will be. We will definitely need some strategy for how we want to move the system into a new structure and I definitly agree that we will need to give some time for the migration.

The idea for the deprecation sounds plausible. From my POV this has little to do with this concrete project but is something that we would want and need anyway, already now. It would be great if you could provide a PR that devises some guidelines how to deprecate functionality in the future. Maybe contact Michael Jansen for further directions or just propose something.

Best regards!

Statement Technical Board, 2022 Apr 6: The Technical Board believes that this project has a big technical relevance and will have deep architectural impacts to allow ILIAS to be a more "Adaptable Learning Environment". The Technical Board assigns a high priority to this development, and we’re looking forward to further collaborate on this matter. The project also requires the focus of the component maintainers.

11 DEC 2023: The Product Manager considers this as a very valuable and important project to reorganise and improve the code of ILIAS - and hopefully to make it easier in the future to quickly understand the ILIAS software architecture. As this project is not only limited on 2022 I have modified the page title to make it 'timeless' (Component Revision 2022 → Component Revision).