Proposal of a Widget Set independent Event Queue implementation

White paper: centralized implementation of the code doing the Event Queue handling in TApplication

Large parts of this in no way describe the current situation, but a possible way to change the LCL.

Terms

For platform-independence, the often used term "message" and "message queue" are avoided and instead the more general terms "Event" and "Event queue" are used unless in fact Windows Messages are discussed.

Moreover the term "Widget Type" is used for the Lazarus implementations of the API provided to the application program, while "Widget Set" is the general term for a collection of GUI control definitions

current situation

Widget Types

Lazarus provides a library of several "Widget Type" implementations that form an interface to the underlying OS-depending infrastructure.

As the name "Widget Type" suggests, here GUI controls - often called "Widgets" are handled. Some Widget Types use an external "Widget Set" libray that needs to be accessible at the application's run time. Here we have "gtk" and "qt" (usually on Linux), "cocoa" (on Mac) and "Win32/Win64" (using the Widget Set API of the Windows OS). Moreover there are other Widget Types that don't use an external Widget Set such as "fpGUI" (directly accessing a "painting" API - such as "X11" - and creating the look of the GUI control elements in Pascal code) or "NoGUI" providing no usable runtime functionality at all.

The Widget handling needs two types of functionality:

1. providing an API to the application program for displaying the GUI Controls
2. firing application code Events when a GUI control is accessed by the user (e.g. by means of mouse or keyboard)

The Event Queue

The "Object Pascal" (Delphi like) way of application program makeup is called "event driven" programming. Here the program is completely created as a set of "Event Handler" procedures, that are called by the infrastructure (done in the run time libraries) when appropriate. With this, the program sleeps most of the time and gets active only when necessary, and - unless explicitly coded - no concurrency (i. e. multiple threads) occurs, preventing the necessity of protecting objects from conflicting accesses, while providing a kind of cooperative multitasking within the application.

With Lazarus (and Delphi), these events are only available in the main thread of the Application and an "Event Queue" guarantees that all events that are originated by external processes (such as the user working with the GUI, a timer, a worker thread requiring attention of the main thread, ...) are scheduled one after the other in the order as they are fired.

Each event handler is a "run to completion" object. It is never preempted by another event but is known to run undisturbed until the procedure exits. If the application programmer feels like, (s)he can use "Application.ProcessMessages" to allow for handling all scheduled events while the running event handler is temporarily halted at this point of execution (i.e. calling all scheduled event handlers like subroutines.)

Obviously the Event Queue functionality is provided by the TApplication object, i.e. by the "Application" instance that is auto-created for the main thread of a user application by the initialization code in the LCL.

Placing Events in the Queue

With Lazarus (and Delphi), the Event Queue is fed from multiple sources

1. GUI events
2. Timers
3. Events generated by other threads of the application by means of "PostMessage()", "PostThreadMessage()", "TThread.Synchronize()" and (right now not implemented in Lazarus) "TThread.Queue()".
4. Only with Windows: Events from other Application, that do "PostMessage()", sending a Windows Message to the main Window of our application.

Moreover, Lazarus (but not Delphi), with "TApplication.QueueAsyncCall()" offers a "non Windowish" method to queue an event that explicitly schedules an event handler function given as an argument. Other than TThread.Queue() same can provide a large data record with the call.

Handling Events from the Queue

The runtime libraries see that the sleeping application main thread gets woken up whenever the Event Queue is not empty. Now the events are taken from the queue and handled one after the other in the same order as they had been queued.

When scheduled, the event handlers are called by the LCL (or Delphi VCL) by means of the "Dispatch" mechanism of the main form, thus reaching the code of all controls it owns. The user code can use the Controls' event properties to create derived handlers for the predefined "On..." events or do "procedure ... message" to create an event handler for "Windowish" message-type events not defined in the LCL. Additionally, events generated by TThread. Synchronize(), TThread.Queue() or TApplication.QueueAsyncCall() fire the event handler function given in the argument.

Ways of implementing the Event Queue

The Event Queue is done in different ways for the different OSes.

Windows

Windows provides a "Message Queue" to user applications. In fact it provides a Message Queue with any of the application's windows, so a Windows application can have multiple message queues. Accordingly the LCL (and the Delphi VCL) uses this Message Queue to implement the Event Queue for the application's main thread. Windows automatically places GUI events, generated by application's main window's controls, and Timers in this queue (by means of Windows Messages). Application programs (the owner of the window as well as other applications) can schedule events ("send user Messages") by the "PostMessage" API call. Worker threads additionally can use the slightly faster "PostThreadMessage" call to notify the main thread (or the owner of another window of our application's).

Windows provides an API to allow a user application (or a thread of same) to wait for Events (here: Windows Messages) in the queue and take them from the queue (aka "receive a queued Message").

other OSes

With Linux and Mac OS the event queue needs to be done in Pascal code within the LCL. The GUI events generated by callbacks of the different widget set libraries are used to feed this queue, together with events generated by TTimers, and the "PostMessage()", "PostThreadMessage", "TThread.Synchronize", "TThread.Queue" (if once implemented) , and "TApplication.QueueAsdyncCall"functions.

Some Widget Sets don't provide an event queue at all.

Suggestion

Due to historical reasons the code that creates/manages the event queue is done directly in the appropriate incarnation of the Widget Type code. This makes creating new Widget Types more complicated than necessary and up to now prevented the creation of a Widget Type that does provide an event queue but does not need a binding to a local GUI.

ToDo List

Theoretically it is possible to enhance this proceeding in a way that a worker thread, too, can instantiate a TApplication object and thus get it's own message queue and thus allows to be programmed in an "event driven" as run to completion objects, as well. Of course the event queues of worker threads are not supposed to receive GUI events (which would ask for a reentrant implementation of many complex LCL functions), but it could receive TTimer events and events generated by other threads.

(Thread programming for SMP)