Show Advanced KnowledgeHide Advanced Knowledge

Concept for a Notification System

1 Requirements

Our goal was to devise and describe a basic concept for handling and processing notifications in ILIAS in a system-wide consistent way. The basic idea is to create a layer of abstraction by providing an internal interface based on xAPI statements, which is addressed by all modules and services in ILIAS that want to trigger a notification. That means, ILIAS modules and services themselves control when a notification is triggered and create a statement that is then processed by the system and eventually delivered as a message of some form to a set (1..n) of users.

Another central goal is providing more flexibility and better usability for both system / course administrator and "simple" users of ILIAS when dealing with notifications. Users ought be able to configure which notifications are delivered using a choice form a selection of different nofification channels, while system administrator should be able to set systemwide preferences that can or cannot be overwritten.

The concept includes:

• a general description of the xAPI based notfication system including various components and processes neccessary for handling notifications
• a description of the xAPI statement grammar and how it can be used in the context of ILIAS to model notification "events"
• a description of how different "delivery channels" for notifications can be addressed and configured
• table shemas for the basic new database tables
• three delivery channel types: mail, instant on-screen-display, timeline
• GUI descriptions for the central (system-wide preferences) and the user-based preference configuration interfaces concerned with notifications
• a extensive listing of the existing notification functionalities up to ILIAS 5.0.x
• ...

What the concept is not...

• a detailed layout for the actual implementation of the components, since we were supposed to create a basic concept
• a comprehensive list of all notification functionalities that should be provided by ILIAS, since this is a separate process that has to be gone throgh with each component maintainer, a well-chosen user group, possibly a SIG that is concerned with the component and the ILIAS Jour Fixe
• the solution to getting further feature-focused conceptual work and the actual coding implementation(s) that derive from them funded
• concerned with features that will enter ILIAS with version >= 5.1.x

1.1 The Concept Files and other Documents

Initial concept for a new notification system in ILIAS

This is the initial version of the concept as presented to the ILIAS Jour Fixe on May 11th 2015. Furthermore, the basic ideas of this version of the concept have been presented and briefly discussed at the ILIAS Süd meeting on June 18th in Konstanz.

Revision of the initially presented concept

The current version of the concept for a new notificaton system in ILIAS comprises a revision and extension of the concept as it was presented by Uwe Kohle at the Jour Fixe meeting on May 11th 2015 together with feedback from attendants of the meeting and the discussion entries posted on this feature wiki page until 27.09.2015. In addition, the concept was briefly presented and discussed at the ILIAS Development Conference in Dortmund on Sept 8th 2015  (feedback and suggestions form the discussion have been considered for the revised version fo the concept as well).

2 Additional Information

• Concept: Uwe Kohnle (Internetlehrer GmbH), Marko Glaubitz, Philipp Kröpelin
• Funding of concept: ILIAS-Verein
• Funding of implementation: Partly by ILIAS-Verein
• Concept is written by: Uwe Kohnle, ,
• Tested by / status: (name, e-mail), (status information set after implementation)

3 Discussion

Kunkel, Matthias [mkunkel], March 17, 2015: Based on the result of the tender for a concept of a notification system, the Advisory Council commissions Uwe Kohnle (Internetlehrer GmbH) and his partners Philipp Kröpelin and Marko Glaubitz to develop a concept for a notification system. A rough concept and a detailed concept will be discussed in the Jour Fixe in time.

JourFixe, ILIAS [jourfixe], 11 May 2015: Uwe gave a presentation of the current draft concept today. We discussed its main points and think that some major issues need to be adressed in a final concept (currently in German):

• Begriffe müssen klarer herausgestellt werden: Event vs. Notification vs. Statement vs. Reminder. Bitte ein Glossar ins Dokument aufnehmen und dort die Begriffe kurz klären
• Hilfreich wären UML Klassen- oder Sequenzdiagramm (um Schnittstellen und Kontrollfluss besser zu verdeutlichen), sowie ERD
• (Mehr) Mockups wären wünschenswert
• Aggregationen werden noch zu wenig behandelt, gerade in Hinblick auf Content in unterschiedlichen Kanäle
• Performance sollte behandelt werden
• Content-Ertellung sollte behandelt werden (Transformation von Statement zu Text)
• Konkreter Kontrollfluss ausgehend vom Event, hin zur Notification, Subscription-Prozesse, etc
• Prozess des Notificationversands ist nicht ausreichend geklärt/beschrieben (Ablauf, Schnittstellen)
• Unklare Abgrenzung zu bestehendem Event-System (soll es komplett ersetzt werden?)
• Es ist gut für den Ereignisteil, xAPI zu nutzen. Aber im aktuellen Konzept steht xAPI (statt Notifications) zu sehr im Focus (Verwendung von xAPI für etwas für das es nicht vorgesehen ist?)
• Befürchtung, dass Regelwerk xAPI Statements in DB-Tabelle auf Dauer zu unflexibel ist (Indiz z.B. context_instructor_type, context_team_type, spiegeln Flexibilität des ILIAS Rollensystem nicht wieder)
• Reminder-Problematik passt nicht ganz ins Konzept (Notwendigkeit der stetigen Aktualisierung/Löschung von Einträgen mit Reminderverhalten)
• Komplexität der Settings für Benutzer zu hoch?

We ask all core developers to read the concept and to add their opinions and open issues as comments to this page.

Jansen, Michael [mjansen] 12 May 2015: Most of my objections are already mentioned in the list above. In my opinion the rough concept (this term might be a bit misleading, because the call for proposals actually claimed some detailed aspects) should include something like a "big picture",  which shows the lifescycle of notifications and the stream (with subscriptions/unsubscriptions).
The requirements of the concept concerning interfaces (technically) are not fullfilled. I expected something like a visualization of the system's architecture, based on an event triggered in an existing service / module, to the abstract message is transformed into different target formats (email body, sms, tweet, on screen markup), to the point of delivering the resulting message on different channels to a person/other system. I did not expect code but at least something like an abstract model (UML class diagram, entity–relationship model). The following questions should be answered by the concept:

• How can a notification type be registered by a service / module maintainer?
• How can a new channel be registered (and think about channel plugins)?
• How can a user be subscribed to a certain notification?
  • Active subscription
    • The user should be able to cancel the subscription
  • Passive subscription
    • The user should be able to cancel the subscription
    • Forced, the user should not be able to cancel the subscription
• How is a notification triggered in an existing module / service? The concept should desribe the API (public interfaces) of the notification service a service / module can communicate with. How does the notification system can contact the consuming service / module to fetch and inspect further information (bidirectional commumincation)?
• How long does a notification exists in the persistence layer of ILIAS (time to live)?
• How exactly is a abstract message transformed into the target format, depending on the delivery channel and aggregation level?
• How are aggregations handled in general (daily, weekly)?
• How does a author (e.g. a global administrator) define the readable contents (let's call it template) of a notification for different channels in different target languages?
  • How does a certain notification provide placeholder information (the placeholder itself and the information, how this placeholder is substituted)?
• How does the notification system handle recurrent notifications?

MB, 12 May 2015: I was required to share my thoughts on this.

I have to admit first that my remarks are biased towards a concept sketch that was not considered during evaluation of the proposals.

While a majority of notifications should be sent at times when an event is raised, I do not see that events are a suitable concept to trigger notifications. The main reason is that an event based notification relies on something similar to an observer pattern but that is not "where the requirements for sending a notification come from". Use cases that require notifications to be sent are usually formulated alongside the location of the "primarily acting code", so moving these parts of specification contents into "distant locations" like event handleresque constructs doesn't sound like a good way to deal with them.

Furthermore, the simplified call to an event handleresque construct doesn't seem to be appropriately addressing requirements of different transmission modes and notification types.

The message content as it is defined by the stakeholder should be reverse-engineered and the data required forward-propagated:

"Dear Mr. Kohnle,

Your tutor Peter Müller created a virtual classroom in course BWL A. Please review the next session dates."

What data is required:
1. Recipients
2. The actor and his role in another object at the time of the subject of the notification is emitted
3. The type of the object created
4. The title of another object at the time of the subject of the notification is emitted

The emitting service - the virtual classroom or the course - must know what the stake holder wants to have sent out, so the data passed into the notification system can (7 days later? Weekly e-mail digest?) be used to unfold to placeholders:

- Recipient- -Lastname, -Firstname, -Title, -Salutation, -Gender (for each recipient!)
- Actor- -Lastname, -Firstname, -Title, -Salutation, -Gender
- Actor-Role on another object
- Object-Title
- Object-Title of another object
- Object-Type of another object

In order to make the translation of notification-calls to notification-messages, the observing event handler is required to inspect the state of the calling code to get the data that is required to be sent. It should be the event handleresque, because in scenarios with delayed transmission of the notification, the state may have changed: Ownership may be transferred, objects may be deleted again and the like. So in my opinion the notification that is to be sent should be instructed to be sent with all necessary data included. This requires a very flexible data model because the relevant content for a notification differs greatly between different types of notifications. The simplified "who did what to whom and when" doesn't cover all aspects. Instead, it should be "code wants the appropriate audience to know a list of keyed data".

The concept describes a table il_xapi_statement which requires every output channel to modify a core table: Against this beyond words. Just don't!
The concept describes a table il_xapi_verbs which defines one content for all notification channels. This is not going to work nicely with Mail vs. OSD and will not work fatally with channels that enforce limitations on content length. (E.g. a course title in ILIAS can be longer than the allowed character limit on Twitter.)

In my opinion using xAPI as a concept in this context doesn't work out.

OT 18. Mai 2015:
Most of the issues/ideas may have been mentioned before but I'd like to communicate my concerns as well. In the first part I'll discuss the concept and in the second I append a wishlist what in my opinion should be embeded in the concept before development starts.

I like the concept of notifications being stored in an abstract form e.g.
Create;  Uwe  Kohnle;  Wiki;  Kurs  "Grundlagen  BWL" [+ additional Parameters]
This will allow the client of the notification system not to care how the notification in the end will be handled and the actual handler can push the notifications highly configurably. Important to notice is, that all data to create the notification must be present in the abstract form, as the lookup of e.g. the title may not be the same when the notification is sent.

However, I'm not sure how close we should keep this to xapi and wether to call it xapi at all. For example the notification that someone has uploaded a new file or the reminder for you to fill out some survey is, in my opinion, not a learning experience. Thus the xAPI concept is stretched out to work as a notification system. My experience with xApi is very limited, maybe some concrete examples why it should be used as a notification system would help. In my opinion those are two different things, you can implement xAPI into ILIAS and on top of that you can implement a Notifiaction System but the specified concept looks like a hybrid thingy xApi with a mixed in notification system.

I also don't like the idea to listen to the current ILIAS events when adding notifications. If a notification is sent, the code for it should be there. This allows to: Order the calls (first the EventHook is called, to enable plugins, and then the notification gets created) and it is more transparent for the developer. He/She knows a notification is created here. While I really appreciate the current ILIAS Event system for Plugins I think it should not be used too frequently for ILIAS core features.

[Edit: On second thought this is pretty much what Micheal Jansen already wrote down, so you can see this as an emphasis on his list :-)]
The Wishilist (the examples are just from the top of my head, not to be taken seriously):

•  A clear distinction in the Document what "Events" are. Is it the current Event system of ILIAS? If not then the terminology collides, it needs specifications.

•   What does the interface look like if I want to send a notification from the ~datacollection?

   $notifications = ilNotifications::getInstance();
   $notifications->add('dcl', 'entry_created', array(
    'creator' => 'Oskar Truffer',
    'entry_title' => 'Kochrezept: Wurstsalat'
   )); // In my dreams this would throw an Exception when the parameters are not as defined.

•  How does a Module/Service define what notifications it can send? How can a plugin define what notifications it can send? Edit a table in the db? Something in the module.xml?

•  What does the interface look like to save user preferences for the notification system?

   $notificationSettings = ilNotificationSettings::getInstance();
   $notificationSettings->set('dcl', 'entry_created', ilNotificationSettings::WEEKLY | ilNotificationSettings::NONE)

•  Is there an abstract interface for a channel? How does it look like? What needs to be written everytime a new channel is created, what can be shared? Will there be a plugin slot to create new channels?

•  A cross cut from when a notification is triggered, to when it is sent.

  1.) Notifiaciton is created by Module xy. Add entry in notification_event: (123, 'dcl', 'entry_created', NOW()). Add entry to notification_parameters (123, 'creator', 'Oskar Truffer'). If user setting of 'dcl', 'entry_created' is ilNotificationSettings::IMMEDIATLY, then no database entries are made and continue with 5.)
  2.) ... maybe something timed? How do you bundle notifications?
  [...]
  5.) Lookup which channels should be used for 'dcl', 'entry_created'
  6.) Instantiate each channel
  7.) send event identity plus parameters to create actual notification -> Channel sends out notification.

•  How does the User Interface look like to enable/configure different notifications? How does it look like to customize channel output?

Philipp Kröpelin, 18.05.2015
Vielen Dank an alle, die sich im Jour Fixe und hier so leidenschaftlich, engagiert und umfangreich zu dem Thema geäußert haben.
Wir werden alle Anregungen und Fragen aufnehmen und beantworten, benötigen dafür aber noch einige Wochen, da sowohl Marko als auch Uwe aktuell urlaubs- und dienstreisebedingt nicht Stellung nehmen können. Ich bin sicher, dass im abschließenden Konzept alle Vorstellungen zufriedenstellend berücksichtigt sein werden.

Jansen, Michael [mjansen] 15 July 2015: Any news here ;-)?

Killing, Alexander [alex], 15 July 2015: Some points to the "how are notifications transformed into the target format" issue:

• The emitting service needs to "store" some key/values for the notifications when the triggering event occurs. As I understand we mostly agree in this point (We want the historic data in the notification that is send days later).
• Now I understand that it is the responsibility of the notification service to control when the notification is finally transformed into the presentation for the different channels.
• But who is responsible to determine the content of the notification "per channel"? Is there some generic logic within the notification system that allows it to do this on its own, or will it call the emitting service again to fullfil some "transformations"?
• My understanding is, that the emitting service needs to be called again (not sure, if this is part of Max comments, he lost me). Only the emitting service is able to transform the key value pairs into a presentation form, e.g. it "knows" the data types (e.g. date/time) and the actions needed to transform this to a presentation form. Also the "content templates" are part of the responsibility of the emitting service (whereas the abstract structure of the templates is defined by the channels, right?).
• If I am not wrong with this, I think it will be very difficult to support a good extensibility on the side of the channels. For each channel new placeholders (key/values) and new templates may be appropriate. Since this lies in the responsibility of all emitting services, it is not feasible that a new channel is quickly supported by a plugin. Personally I would drop the goal of a good extensibility for channels. I would be quite happy if we manage to get mails and OSD messages right. Of course already these two types should be based on a sound abstract foundation, but I would not try to support a "channel plugin slot".

JourFixe, ILIAS [jourfixe], September 28, 2015: Uwe Kohnle, Marko Glaubitz and Philipp Kröpelin presented an revised and extended version of their concept of a notification system, followed by a long discussion about several aspects of the concept. We could not finish the discussion and decide upon the concept in the meeting as several participants had to leave to get their train home. We will discuss open or unclear aspects in the feature wiki.

Klees, Richard [rklees], 2015-10-12: 

I was asked to clearify my concerns regarding the current proposal for a new notification system based on xAPI, so here we go.

First of all, i'm not objecting against the introduction of an xAPI implementation to ILIAS. I just see no fit for xAPI in the implementation of a general notification system. If somebody wants to have an xAPI implementation in ILIAS it should be put on the feature request list and discussed in the JF and not be the hidden agenda in the introduction of a general notification system.

In the proposed concept, a xAPI-event is the trigger for any notification. A notification is defined as the readable form of an event when the event is send to a user. There surely are cases, where a notification is a direct consequence of an event, e.g. a user joined a course. But there surely are also cases, where a notification is not caused by an event, e.g. a user needs to be reminded to participate at a survey. It is of course possible to generate an artificial event in such cases (user did not participate in the survey), but it seems to be questionable how this is an event (as nothing happened). The moment the event is generated is arbitrary regarding the noted fact, as the user did also not participate in the survey 10 minutes before and (maybe) 10 minutes after. This is of course a possible way to use the event mechanics for sending a reminder, but also looks like a hack, as i need to turn a non-event into an event just to send a reminder.

This is even more severe when one focusses on xAPI-events rather then some general notion of event. The xAPI-spec defines the Experience API as "a service that allows for statements of experience to be delivered to and stored securely in a Learning Record Store (LRS)". Clearly "user did not participate in a survey" is not a learning experience. We thus would introduce some kind of non-learning events in a LRS.

Regarding the proposed concept, we would also introduce ILIAS as an actor in learning records. The proposed xAPI-statement for "Reminder for consulation" is "ILIAS (actor) reminded (verb) consulation_obj (object) in crs (context)". This again is of course a possibility to use the proposed event mechanics for notifications, but again looks like a hack. In the moment said event is generated, ILIAS did not in fact remind anyone. It rather might remind someone as a consequence of the generated event, which therefore is an outright lie in the moment it is generated. It is also questionable how ILIAS is an actor in the sense of xAPI.

By using a general LRS in ILIAS for notifications, we would pollute that LRS with statements that are not about learning experiences, either because they are not about experiences ("user did not participate in survey") or because they are about entities that most certainly do not make expeciences ("ILIAS reminded consulation_obj"). We would also put things into the store that did not happen yet or, in the worst case, will never actually happen ("ILIAS reminded consulation_obj"). To me it therefore seems very very wrong to use a general LRS for notifications and i see no way to unify an implementation of xAPI with a general notification system without damaging concepts like "event", "learning record" and such and jumping through hacky hoops to make xAPI fit notifications.

This does not mean, that notifications could not be caused by xAPI events or be generated based on xAPI events. But this is a different question, as it would mean we have a clear concept of a notifications system and a (concept for an) implementation of xAPI.

Even if one disregards the concerns i raised above, there are many open questions in the proposed concept for notifications based on xAPI. These were formulated in the comments above and in the discussion on the JF as well. To me that is a strong indicator, that the proposed concept is not ready for an implementation, as it seems to raise more questions than it answers.

I therefore propose that the JF thanks the authors of the concept for their research regarding an implementation of a notification system via xAPI, rejects the concept and the idea to implement a notification system based on xAPI and then discusses how to get to a better solution. 

4 Implementation

...