Show Advanced KnowledgeHide Advanced Knowledge

(Project) CLI-Setup

This is a project page that bundles several feature wiki pages which belong to a larger development activity for revising the ILIAS component Setup and creating a new CLI-based setup for ILIAS.

Klees, Richard [rklees] In JourFixe-2017-09-25 Alex Killing declared to step down from the maintainership of the Setup and Matthias Kunkel called for applicants for that vacancy. I am indeed interested to take over the maintainership and the CaT Concepts and Training GmbH is willing to make investments in the setup. Since I have some opinions about the general direction the setup should take and I'm not willing to just continue with the current model or blindly accept payment for any feature in the setup, I want to check these ideas with the community first before finally applying for maintainership. Please leave comments and questions in the discussion section of this page as always.

1 Initial Problem

The current ILIAS setup has some problems, most prominently, but not exclusively:

• the setup is not automatable
• configuration can not be shared or saved easily
• the shared list of DB-steps introduces friction when updating steps, especially in hot phases of development cycle
• the general distribution of settings between the setup and the administration menu is unclear
• plugins cannot take part in the update process of the setup, administrators need to update every plugin individually
• the installation process is intricate code-wise and seems to have grown mostly organically

At CaT Concepts and Training GmbH we tried to tackle some of the problems with our ilias setup script ilse, a general revision of the setup could steer the general ILIAS setup in a similar direction.

2 Conceptual Summary

2.1 Complete setup and update process must be automatable. CLI for setup is paramount to web based GUI

There is a lot of value hidden in the setup that would be uncaged by having a setup that could be completely automated. A completely automated setup will have benefits for testing, whether manual or automated, will simplify the work of developers, old and new, and will also simplify the work of hosting providers when setting up and maintaining their ILIAS instances.

Thus an automated setup with a command line interface (CLI) will be paramount to a web based GUI setup. A web based GUI setup will only be implemented as an afterthought to the CLI-setup if there is a real requirement for such a setup. It will most likely not be as interactive as the current setup is, possibly just being a configuration generator for the CLI setup.

2.2 Database updates of components needs to be decoupled

Currently all database update steps go to one central file in ILIAS. Especially in hot phases of the development cycle (think: 30th August) this creates a lot of friction between developers. There are indeed some steps that need to be executed in a defined order, but a whole lot of update steps only depends on updates in the same component. Thus every component should get their own location for update steps while providing a mechanism to express inter-component dependencies.

2.3 Database updates need to be modernized

The current mechanism for database updates relies on a custom file format where discrete parts of the file are eval'd by the db-update mechanism. This leads to odd behaviour when using some special and undocumented variable names in the setup steps. Exceptions cannot be traced to a location in the code properly, since eval hides the stack trace. It furthermore is not clear which resources could be used in the steps as there is no clear interface to them. The current update mechanism should be modernized to tackle these problems.

2.4 Plugins should become first class cititzens (wrt setup at least)

Currently one has to got to different places in the system to update the database of the core system and the plugins. This approach is cumbersome and would decrease the benefits of an automated setup if maintained in the future. Thus plugins should be able to take part in the setup mechanisms just like any other component in the system. This will not supersede the "Plugins"-node in the administration and the configuration of plugins in ILIAS, but there certainly will be configuration of plugins that could be made in the setup.

2.5 Distribution of configurations between admin-menu and setup needs to be reevaluated

There currently is no clear guideline which configuration should go to the setup and which configuration should go into the admin-menu in the system. There is some configuration in the administration menu (e.g. Java-Server) that can only be made by a technical person with knowledge of the server. On the other hand, there is configuration in the setup (e.g. Contact Information) that an administrator in the system might want to edit. The distribution of configuration options between the setup and the administrator menu thus should be reevaluated.

2.6 Storage of configuration needs to be reevaluated

Currently the configuration made in the setup is stored in different locations: ilias.ini, client.ini, the database. On the one hand it is quite unclear how the storage location is defined exactly, on the other hand, every call to ILIAS means at least two parsing passes (ilias.ini, client.ini) and several roundtrips to the database. This model should be reevaluated to have a unified mechanism for storing configuration and obtain the potential performance benefits.

2.7 Multi-Client capability can go away

The current multi client capability of ILIAS seems to be a way to easily share code between different ILIAS installations. The benefits of this (lesser storage requirements, easier checkout of the code) seem to have been greatly reduced with dropping storage prices and git checkouts. The benefits of multi client capabilities will diminish even more once an automatable setup exist. The additional complexity in the system introduced by the multi client capabilities seem not to be backed by the benefits anymore, thus the multi client support will be discontinued.

3 Related Feature Requests and Status of Implementation

3.1 ILIAS 6

Feature Request	Suggested by	Funding	Status
Setup - Abandon i18n in Setup	Klees, Richard [rklees]	CaT Concepts and Training GmbH	implemented
Setup - Introduce CLI-Setup	Klees, Richard [rklees]	CaT Concepts and Training GmbH	implemented

3.2 ILIAS 7

Feature Request	Suggested by	Funding	Status
Guideline for Configuration in the Setup	Klees, Richard [rklees]	CaT Concepts and Training GmbH	effective from 7
Include Plugins into composer autoloader	Klees, Richard [rklees]	CaT Concepts and Training GmbH	implemented
Setup - Abandon Maintenance Mode	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Removed from code
Setup - Abandon Multi Client	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Removed from code
Setup - Abandon web based GUI	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Removed from code
Setup - Add migrate Command	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Implemented
Setup - Add status Command	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Implemented
Setup - Include Plugin Configuration	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Implemented
Setup - Let update-Command change configs	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Implemented
Setup - Move (some) Configuration from Administration to Setup	Klees, Richard [rklees]	CaT Concepts and Training GmbH	Implemented
Setup - Update Plugins via Setup	Klees, Richard [rklees]	CaT Concepts and Training GmbH, Universität Hohenheim	Implemented

3.3 ILIAS 8

Feature Request	Suggested by	Funding	Status
Setup - Introduce Strict Update Paths	Klees, Richard [rklees]	Needed	Ready for JF
Setup - Move (more) Configuration from Administration to Setup	Klees, Richard [rklees]	Needed	In Progress
Setup - Move DB-Updatesteps to Components	Klees, Richard [rklees]	Needed	Ready for JF
…

3.4 Rejected / Outdated

Feature Request	Suggested by	Status
Setup - Refurbish web based GUI	Klees, Richard [rklees]	Rejected due to Setup - Abandon web based GUI
…

4 Further Results

• …

5 Contact

• Author of the Request: Klees, Richard [rklees]
• Maintainer: Klees, Richard [rklees]
• Implementation of the feature is done by: {The maintainer must add the name of the implementing developer.}

6 General Discussion

Schmid, Fabian [fschmid] 2017-10-11: All of the outlined points above have our full support! Thanks a lot for this step in the right direction.

Baumgartner, Robin [rbaumgartner] 2017-10-11: From a server administration point of view, I support these proposals too (especially 2.1, 2.5, 2.6 and 2.7). We never use more than one client anyway, so we won't be sad if this feature is discontinued. I would like to suggest a focus on extensibility during the redesign of the configuration file, with the goal to ultimately have just one configuration file where all dependencies to external services are declared (DB, ilServer, Chatserver, etc.). Maybe the current ini-style configuration file could be extended with new sections for these services.

Hübsch, Wolfgang [wolfganghuebsch] 2017-10-12: I also think that multiclient-support is used rarely. However, there are some more benefits:

1. Plugins only needs to to be installed and maintainced once
2. There is only need for one Lucene- and chatserver

However, I just wanted to say this for to balance the reasons. Is there any institution who uses multiclient-support extensively? I do not know one. In my dreams a lot of schools all use their own client on one installation. But until now, it has never come so far. For example, a Bundesland decides to implement ILIAS for all schools  and every school got its own client - but I am not shure if this will ever happen.

Amstutz, Timon [amstutz] 2017-10-12: All of the outlined points above have the full support of the University of Bern! We strongly believe, that this is the correct vision for the ILIAS setup and might improve as well as modernize the ILIAS development.

We manage different installations running identical versions without using clients, since we do not like to only partly integrated solution provided by ILIAS. We believe that this feature should be cut off in the future. With the advent of GIT and the cheap webspace available there are not real advantages. With lucene and chat we just provide several instances of those on one server each serving a different installation. This is a rather small overhaed configuration wise and almost none in terms of hardware.

Killing, Alexander [alex], 12 Oct 2017: Lucene and chat server both should be able to handle multiple installations. No need for the use of clients here as far as I know. Client ID handling adds a certain amount of complexity not only in the setup but also in session/authentication and url routing / perma link handling. Something like the client list/menu could be easily replaced by a static website in most cases in my opinion.

Not sure about the plugins. Would they autmatically be downloaded using git?

Kunkel, Matthias [mkunkel], October 12.10.2017: Thanks to Richard for this important initiative that I would like to support with all my possibilities. Nevertheless, I just want to drop some comments on this concept:

• Getting an automated setup and update process would be great. And I guess we need this when we want to continue with unit testing. But like Mirko I have some concerns if we drop the setup by UI interface totally. Mirko's suggestion to have a kind of wizard with GUI that creates such an update/setup file sounds very good to me. This could be an additional 'package' to implement. And I am sure we will find funding for this.
• Re-evaluation of configurations in setup and administration is a good job to do. We should take into consideration that there are installations where persons have access to the administration of an ILIAS installation but not to the setup - to avoid that they make unintended configuration changes or something else. It would be great to know a bit more about such scenarios and to know how they could be realised in the future.
• I was a bit shocked when I read about giving up multi-client capability. I always thought it is a good feature of ILIAS. And I remember several people mentioning that this is a big advantage of ILIAS. But as Alex mentioned above, there are also some really ugly effects due to session conflicts when using more than one client. It would be great to know some more problems that result from the current multi-client capability to decide if we get rid of it or not.

Schenk, Ralf [rschenk], October 17: I agree to nearly all points above but DON'T DROP MULTICLIENT !

• We have several important customer Instances using up to 30 Clients. i.e. https://www.lernplattform-bakoev.bund.de
• If we drop multiclient we will have to reorganise the webspace and also the virtual hosts since we have 1 document-root (the ONE ILIAS code directory) now and then we need different document-roots (each client needs its own ILIAS) or have URL changes like http://myilias.domain.de/CLIENT1.
• Involved in real hosting this also needs different DNS Fully-Qualified-Hostnames for each virtual host
• Following we need new SSL Certs for each virtual host
• So no fun for at least me and my coworkers if any of these customers needs to update.
• What will I tell them ? Our next Update will need a few days and you can start organizing DNS and SSL before...
• And for your next bugfix "git pull" - I'll do that 30 times or start writing our own multiclient helper software...

, October 20: Please do not drop the Multi Client Support. Uni Hannover is heavily relying on it : https://ilias.uni-hannover.de 

Killing, Alexander [alex], 20 Oct 2017: Wouldn't it be an option to automatically transform multi-client installations into separate installation in subdirectories under the same document root?
https://www.lernplattform-bakoev.bund.de/login.php?client_id=BRH_ILIAS
becomes ->
https://www.lernplattform-bakoev.bund.de/BRH_ILIAS/login.php

No need for different document roots or new SSL Certificates. Just some rewrite rules that resolve the old perma links.

Neumann, Fred [fneumann], October23, 2017
I agree with most points, but not with 2.1 and 2.7.

I think it is very important that ILIAS provies a self-explaining setup GUI that guides interested people through a basic installation who want to evaluate ILIAS against other systems (e.g. moodle). This GUI does not need to cover everything but it should allow to check the basic requirements and and give hints on what is needed for which functionality. Also the language selection and a guided database creation must be possible. Session Management, Proxy and Global Cache are less important at the very first step. Focus should be on the creation of a consistent installation with the minimum needed requirements. The wish for a CLI installation is obvious from the perspective of a service provider, but please don't forget the individuals (e.g. school teachers) who run their own little installation.

We use the multi-client functionality of ILIAS productively since some years and are very happy with it. In our platform for e-exams we create a new client with each new semester to keep the number of objects and the database tables of the test object small. Clients of former semesters are still available to selected users, e.g. to run test statistics. A bugfix update for ILIAS and all plugins is consistently done for all clients at once.

, November 10, 2017
Hi all, I would like to add our thoughts (IBI gGmbH) to the discussion. Short summary: We completly oppose the statement 2.7 "Multi-Client capability can go away".
1 Agreed
2.1 Agreed
2.2 Agreed
2.3 Agreed
2.4 Agreed
2.5 Agreed
2.6 Agreed
2.7 Our Installations runs in an wide-spread intranet with more than 100 Clients, each in a different network. Our project relies on a strict separation of clients. We point users to their client via origin ip. Theoretically it would be possible to realize it via subdomains and whatever... But the effort to maintain  the client feature centralized would be multiplied and spreaded to all ILIAS providers relying on clients. Instead of clients, all of us have to take care of many subdomains, certs, webspace, modified Skins...
Our biggest concern is, we invested a lot of time and money to be able to have on "template client" from which we can replicate content to all clients. So we don't have to add a lessons to >100 clients, but add it to our template and we can copy it via SOAP to all clients. Our project uses ILIAS since arround 2004 and we use clients heavily since 2010. In my understanding, all our investments are in vain if the community drops client support.
We would have to switch our LMS.
We suggest to keep the small effort to maintain the clients feature in ILIAS and don't disappoint all of a sudden heavy users, who rely on it! This would be a slap in our face and bad politics =(
We are confident, the community will decide wisely.
Regards and thanks for making ILIAS possible
Michael Hartmann and Team

Kiegel, Colin [kiegel] 2017-11-24: At Qualitus we often use the multi-client capability.

• In public call for bids for LMS, we often see the requirement of multi-client capability. We are concerned that dropping multi-client capability will reduce the competitiveness of ILIAS.
• We also run a lot of installations with multiple clients. Dropping this functionality sounds like a huge "pita" to us.
• A lot of our customers want multiple instances of ILIAS, which are only different in some configuration aspects, like default language, default skin, login page, etc.
• Old database snapshots can be easily made available as a different client for comparison, if something went wrong.
• A shared code policy can lead to cost savings, if testing etc. only needs to be done once. We often use the multi-client capability to "enforce" this shared code policy.
• One concrete example is ILIAS CI performance installation. It has three different clients, which are guaranteed to run on the same code base and and are updated in lock-step:
  • public client (self registration possible)
  • temporary client (to run all the measurements, only automated users login here)
  • template client (this is a snapshot used to reset the temporary client before each measurement cycle)
• Other concrete examples are customers with clients for each federal state or country.
• Among our customers roughly every 2nd to 3rd uses multiple clients

That being said, multiple installations running on the same git branch are of course very close to mulitple clients. But you loose the guarantee that the code will always be identical. You need more webspace, and you need to checkouts multiple times. Also all existing multi-client configurations need to be adapted. And last but not least it might reduce the competitiveness of ILIAS in public call for bids. It might not be the end of the world, but for us these are clear disadvantages.

Also, please don't drop a GUI based setup completely. Sometimes it's just easier to change something via a web frontend than via VPN/SSH etc. Also the learning curve is not as steep for new admins.

All other points have our full support. :-)

Klees, Richard [rklees], 2017-11-27:
Hello everybody,
thank you all for your feedback. Most of my initial items seem to be uncontroversial, which I'm happy to see. Two items gather a lot of attention, so I want to adress them specifically. I'm sure we will be able to settle both, so I'm really looking forward for the things to come regarding the setup.

at "Complete setup and update process must be automatable. CLI for setup is paramount to web based GUI":

There seem to be two mayor concerns:

1. That the setup process may only be triggered via CLI.
2. That a basic installation will be harder to create than it is atm.

Concern 1) is already adressed in my initial proposal. To be clear: I'm not advocating to drop the GUI based setup completely. I only propose to think of an automatable CLI-based setup first and then build a GUI on top of that. This may look a lot like Mircos proposal.

Regarding 2) I definitely see the requirement that ILIAS should be easily installable to let users try it out. I cannot see how this really depends on a GUI based setup. I mean, is there anyone who really wants to argue that the current GUI-based setup makes it easy for newcomers to install ILIAS? On the other hand we could provide users with different configuration files for different purposes or OSes in an automated scenario. That will make it easier to install ILIAS even if there would be no GUI-based setup. An automated setup also opens the doors for other deployment scenarios that would be even simpler, like a docker image or packages for linux distros. That said, I'm all in making it easy to try out ILIAS, but a GUI-based setup does not necessarily achieve that and also is only one of many possible ways to get there. If "ease of install for newcomers" is a real requirement we should adress it seperately. When building a GUI based setup we should of course at least think of possible newcomers as a target group for such a feature.

at "Multi-Client capability can go away":

Some people voiced concerns about the multi-client capability that imo are only tangentially related to that feature of ILIAS:

1. Some installations use the overview-page for the different clients to lead their users to the correct client for their purpose.
2. Some users create new clients frequently for specific use cases.
3. Some users use single instances of Lucene and the chatserver for multiple clients.

For 1) I cannot see, why a page like that could not be build for multiple ILIAS installations as a static or dynamic page, like Alex suggested. 2) seems to be a perfectly valid use case, but will be even easier to accomplish with an automated setup, no clients needed there. Removing multi client capabilities will even lead to a better separation of clients, because logs and directories won't be mixed up as in the current multi client implementation. I also cannot see why copying standard lessons to an installation via SOAP should depend on the multi client capabilities of ILIAS. Wouldn't that just work exactly the same with two separate installations? 3) was already answered by Alex: neither Lucene nor the chatserver depend on the multi client feature to be used by different installations.

There are some concerns regarding the changes that would need to be done in the hosting environments if the multi client capability would be given up and how setups currently using the multi client features could be migrated to single installations:

1. One would require multiple subdomains and thus certificates to run multiple ILIAS instances without the multiclient feature.
2. Migrating setups that use the multi client feature to separate installations will require some effort.

I cannot see why 1) needs to be true. At CaT we have multiple ILIAS-instances happily running on the same server in different subdirectories, all installed as with a single client, no subdomains, one certificate for the domain. I definitely understand why a setup with multiple subdomains (and thus potentially certs) could be desireable, but arguments for such a setup would equally apply to current multi client setups. I definitely see that 2) would be correct. But: keeping the multi client setup because of this would mean we are trading short term gains (no migration for some people) against long term gains (less complexity in the setup and the complete system for all people) which imo would be wrong. Software development is a marathon, not a sprint.

I also think that a migration should be not particulary hard. It's basically:

• Per $CLIENT (i.e. subdirectory in ./data) copy the original ILIAS-folder in a new directory $CLIENT in the docroot (or similar).
• Throw away all subdirectories in ./data that are not about $CLIENT.
• Install a rewrite rule in your webserver or proxy of choice that maps the query-param CLIENT_ID=XYZ to a subfolder XYZ
• Done already.

There is one argument I definitely understand: Instead of one "git pull origin release" (or similar), one would need to do a similar step per installation. Maybe from my concept summary this is not totally clear, but: A CLI-based setup will be able to do just that, and also update the database, and also download and update plugins. Also currently you are not done just git pulling the code. You need to got into the setup, check for hotfixes, which you would need to apply to every single client in your setup per clicking buttons on the web interface. Wanting multi client for only git pulling once to me is not worth the effort if there are no other reasons to keep that feature.

That said, I also want to give some more reasons why I consider the multi client setup as something we could easily give up. The feature introduces a lot of complexity in the setup and in the installed software as well, and I just cannot see that there are enough benefits that feature has. I understand the "git pull" and "similar code" arguments, but consider this minor. I aim to guarantee reproducible and automatable setup via a script, with a possibly similar mechanism as composer has via it's composer.json and composer.lock file. That would guarantee similar code and also do "git pull" for the admins. Yes (again), I also want plugins to be installed and updated by the automated setup.

I definitely would see a benefit in multi client capability, if it would in fact be "Mandantenfähigkeit" as described here. General idea of such a feature is that one can define objects (in a broad sense) that are shared amongst clients, e.g. tax rates, language variables or other standardized catalogs of items. I just cannot see such functionality in ILIAS, the only data that is shared is in the "Basic Settings" of the system, the rest is stuff that comes via code. If one really wants "multi client capability" we would need some serious investment in that topic. From my perspective checking "multi client capability" on requirement sheets atm is close to lying. We frequently find that requirement as well, but on closer inquiry we always found that our customers really want some roles that give access to different parts of the repository and maybe a second or third skin.

I do not ultimatively think we need to abandon multi client capability. I just do not see the benefits and thus my institution won't invest any effort or money in supporting it. Thus I cannot use my own ressources to consider it when seriously reviewing the setup. So the question to me is: is there any proponent of the multi client feature that want's to put money to where its mouth is?

JourFixe, ILIAS [jourfixe], 04 DEC 2017: We highly appreciate this suggestion and schedule the setup revision for 5.4. We see a clear advantage of this concept because a setup would be fully automatable and much easier in the end. A GUI for running the setup would be highly appreciated to allow also users to setup an ILIAS installation that do not have access to the command line of their server. We see that there is an additional effort to migrate multi-client installations to separate installations, but Richard's suggestion above feels like a feasible solution and should be evaluated. Giving up multi-client capabilty is acceptable because it is only rudimentary implemented in the current ILIAS and only focused on the re-use of ILIAS code and some basic settings data in the setup. We are thankful for the current maintainership of Alexander Killing who has maintained this component and are very happy that Richard Klees takes over maintainership of the Setup with 5.4.

Neumann, Fred [fneumann], December 5, 2017
Will there be separate feature requests for the seven points to discuss their conception in detail?

Killing, Alexander [alex], 5 Dec 2017: I agree with Fred that we need to clarify some points so that everyone (users, Richard and other developers involved) get a better idea what to be expected. I attended the meeting yesterday, so its also my fault, that some things are a little bit vague. Most important: the GUI is now "highly appreciated". Does this mean a setup would be accepted without GUI for 5.4 or not? And the GUI (which needs funding) would need a specification, too, of course (views, functionality, etc.).

Klees, Richard [rklees], 2017-12-05

Hi everybody,

my current understanding (and I'm almost sure that matches the understanding of the JF and the TB) is this: This FR is about the general direction the setup should take to allow me to decide whether to take the maintenance and invest further effort in a setup revision. I will now preceed with further planning and design, which will surely take some time and also involve the other ILIAS developers. I will also create according and more concrete FRs as I see fit. I'm not sure that these will be about the seven points and I'm also not sure that all things I propose can be discussed in the form of a FR. PRs might be better for some stuff. I surely won't remove anything without further notice to the and approval by the JF.

Regarding the GUI I understand the decision as follows: We all (and the JF especially) want the GUI. Matthias announced that he will give funding for it from his society budget.

I hope we can agree on that description of the situation. Please feel free to ask for further clarification.

Best regards!

Kunkel, Matthias [mkunkel], 07 DEC 2017: This Feature request is about the concept of a new setup, not about the setup itself. We scheduled this concept for 5.4 to show that the Jour Fixe agrees with Richard’s conceptual suggestions. Richard also mentioned in the meeting that the implementation might even take until 5.5. Next steps in the Feature Wiki would be to create related pages for a new setup feature and to present and discuss them. One of these pages would be the GUI front end, incl UI elements.  IMHO, the number of articles depends from the development packages. It doesn’t need to be seven pages - but of course more than one. 

Neumann, Fred [fneumann], December 8, 2017:
Dear Alex, Richard and Matthias, thanks for the explanations. Im happy tho read that society is willing to invest in a setup GUI that will be reduced and user-friendly and that feeds an automated setup under the surface.
A revision of the setup will also have the chance to revise its relationship to the language service. I already try out some internaly refactoring there that speeds up the reading of language files and the setup would benefit from these improvements when the language classes are directly used instead of being duplicated.

Schenk, Ralf [rschenk] 08.12.2017
Dear Richard: matching CLIENT_ID for Rewriting Rules in a +30 Client Setup will be a pain in the ass and will never catch all links that i.e. point to ILIAS externally. The CLIENT_ID Query-Parameter isn't used everywhere in ILIAS and every change of the URL of a ILIAS Client or Instance in an institution with multiple 1000 Users will provoke a bad user experience anywhere. 
You will save my income, but I've more interesting things that I would like to do rather than discussing that ILIAS e.V. decided to abandon Multi-Client capability and that next upgrade will take at least double the effort because we need to implement these workarounds.

Killing, Alexander [alex], 8 Dec 2017: @Ralf, There are two classes of links in ILIAS. The so called permanent links, which always contain the CLIENT ID. It should be possible to write rewrite rules for them in a way that does not require separate rules for each client (if subdirectories of resulting installations match client IDs).

All other links in ILIAS are non permanent links and almost all of them contain parameters (cmdNode) that change regularly between releases. So the "bad user experience" if they users bookmark these is already a fact now.

There are some special cases like login.php, feed.php, calendar.php which we need to take care of in the rewrite rules, too. And SOAP clients would need to be reconfigured.

2018-03-15:
The benefits of this [the multi client support] (lesser storage requirements, easier checkout of the code) seem to have been greatly reduced with dropping storage prices and git checkouts.

This is not quite correct for some installations. A git checkout needs about 1 GB; even without the .git directory it is about 400MB.
That does not look overly much, but if this figure is multiplied by a factor of 100, it is no longer negligible, especially if one wants to serve the files from a fast SSD.
A larger in memory cache for the file system needs more RAM, which is not for free especially on the amounts of approx. 100 clients.

Also, while it does not happen often, sometimes one needs to do some "hot fixes" to the installation in production.
Not everyone has an own git repository to maintain a clone for these things.
With 100 different instances it becomes difficult to apply these patches to all instances uniformly, and to maintain them is close to impossible.


I also cannot see why copying standard lessons to an installation via SOAP should depend on the multi client capabilities of ILIAS.

It depends on the fact that the single installation knows its clients. All you need to do is to configure which one is the template, and then maybe an (easily maintainable) list of "special clients" that are excluded from replication for some reasons.
If all clients are separate instances one needs to maintain a redundant list of all installations on the "template" client.
Updating database hotfixes is also easily managed with client replication as a side feature, btw.


Neither Lucene nor the chatserver depend on the multi client feature to be used by different installations

Well, but they need to know that they are used by different instances. Somehow each instance needs to maintain an unique id when reusing the same Lucene index (And we surely do not want a separate index for each client; Lucene is a RAM hog already.)
So there is no advantage on dropping the feature here either.

I do not know the code base well enough to understand why keeping this feature adds that high maintenance costs.
But to me it looks like this feature should be pretty orthogonal to everything else, and add very little maintenance costs.
If this is not true for the current code, maybe a better alternative is to refactor the code so it is maintainable?

Especially as far as the setup is concerned, I cannot see the impact.
One usually sets up the system starting with only one client anyway, so where are the extra costs? (Sorry if being a bit dense here; I do not want to sound snobly ignorant, I just really do not see it.)

Sorry for a rather late reply. I plan to show up as the DevConf next week, so maybe we can discuss it there?

2018-03-19:
A short note about a talk I had with Richard Klees while being at the DevConf in Halle.
Richard: please correct me if I misunderstood you.

First, while the multi-client feature can go away, this does not mean that it will go away instantly with the new set up; it is simply that the set up will not support setting up new clients. This makes the feature de facto unavailable to users, but the corresponding code elsewhere will not be removed when implementing the new set up. If someone else is willing and able to provide an extension to the set up, e.g. by means of a separate command line tool, then this would make the feature available again.
I will look into the options of how to implement such an extension.

About having something like a few hundred clients we agreed that the idea of copying the complete code base for each client is somewhat impractical. Richard suggested copying only the ilias.ini.php for each client, and use e.g. Apache RewriteMagic to reuse the actual Ilias code. It needs to be evaluated if this is a viable alternative to the multi-client feature in case of many clients.

Klees, Richard [rklees], 2018-04-30

@Clemens:
Thanks for the summary. I mostly agree. The multi client code won't go away immediately. I slightly disagree regarding a setup extension. The general goal would still be to be able to remove the multi client code perspectively. An extension would thus potentially only work as long as the code is still included.

@All:
I want to reassure you, that, during discussions on the DevConf, I became aware of two use cases for the multi client setup that I want to maintain even if the multi client feature might go away:

• The possiblity to share code between different ILIAS installations (what Clemens said)
• The possiblity to easily and quickly spin up additional ILIAS installations.

Klees, Richard [rklees] There won't be any (visible) changes for 5.4. I remove this from the feature suggestions for 5.4 and resume the discussion in the end of 2018.

2018-05-31:

A small update from my side on alternatives for a quasi-multi-client setup; i.e. having several instances without having to copy the complete code base.

The first attempt was to copy only the configuration files (basically ilias.ini.php) to a new directory, and use an apache directive like

RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_URI} !-f
RewriteRule /iliasN/(.*) /var/www/ilias_main_site/ilias/$1 [NE,QSA,N]


This did not work as expected, as the second instance still did read in the configuration file from the main site; i.e. it behaves like a copy of the main site, not showing the data from its own instance. The reason for this has something to do with the way the ilias.ini.php is looked up by the system.
It might be possible to fix this in the code, but it has to be done in several places (and maybe some plugins, too.)
Also the information where the second instance is located cannot be guessed easily by the code, which means most likely to add some extra configuration to the web server. As this is redundant information from an admin point of view I would rather not like to go that way.

The second attempt was to use symbolic links. All files and directories in the root dir of the other instances are replaced by symlinks to the main instance, with the exception of the configuration files and the data directory. It is possible to set up new instances this way manually, and likely by a script, too. I am not sure how this will work from a Web-UI.

If a new ILIAS verison comes out that has new files or directories in the base directory, then the missing symbolic links need to be refreshed for all other instances. Also some consistency checks like "are the symlinks all ok" are certainly needed. I have no clear idea at the moment how to implement this in a reliable manner. This looks like writing some non-trivial scripts to support and maintain this solution. I have to admit that I have not looked into the multi-client parts of the current code base to weight the trade offs between maintaing the multi-client feature vs. maintaining this alternative emulation of the multi-client funtionality.

Kunkel, Matthias [mkunkel], 28 JUN 2018 : A workshop to discuss the future of client support in a revised setup takes place at July 13, 2018, 10:00 - 12:00 in a VC, see here.