Proposal of a Widget Set independent Event Queue implementation

From Lazarus wiki
Jump to navigationJump to search

White paper: centralized implementation of the code doing the Event Queue handling in TApplication

Large parts of this in no way describe the current situation, but a possible way to change the LCL. Maybe some assumptions in the "Current Situations" paragraph could be erroneous or misleading due to incomplete research. Here I hope for appropriate discussions and corrections.

Terms

For platform-independence, the often used "Windowish" term "message" and "message queue" are avoided and instead the more general terms "Event" and "Event Queue" are used unless in fact Windows Messages are discussed.

Moreover the term "Widget Type" is used for the Lazarus implementations of the API provided to the application program, while "Widget Set" is the general term for a collection of GUI control definitions

Current Situation

Widget Types

In the LCL, Lazarus provides a library of several "Widget Type" implementations as a platform independent layer between the user code and the underlying Widget Set and OS-depending infrastructure.

As the name "Widget Type" suggests, here GUI controls - often called "Widgets" are handled. Some Widget Types use an external "Widget Set" library that needs to be accessible at the application's run time. Here we have "gtk" and "qt" (usually on Linux), "cocoa" (on Mac) and "Win32/Win64" (using the Widget Set API of the Windows OS). Moreover there are other Widget Types that don't use an external Widget Set such as "fpGUI" (directly accessing a "painting" API - such as "X11" - and creating the look of the GUI controls in Pascal code) or "NoGUI" providing no usable runtime functionality at all.

The Widget handling needs two types of functionality:

  1. providing an API to the application program for displaying the GUI Controls
  2. firing application code Events when a GUI control is accessed by the user (e.g. by means of mouse or keyboard)

The Event Queue

The "Object Pascal" (Delphi like) way of application program makeup is called "event driven" programming. Here the program is completely created as a set of "Event Handler" procedures, that are called by the infrastructure (done in the run time libraries) when appropriate. With this, the program sleeps most of the time and gets active only when necessary, and - unless explicitly coded - no concurrency (i. e. multiple threads) occurs, preventing the necessity of protecting objects from conflicting accesses, while providing a kind of cooperative multitasking within the application.

With Lazarus (and Delphi), these events are only available in the main thread of the Application and an "Event Queue" guarantees that all events that are originated by external processes (such as the user working with the GUI, a timer, a worker thread requiring attention of the main thread, ...) are scheduled one after the other in the order as they are fired.

Each event handler is a "run to completion" object. It is never preempted by another event but is known to run undisturbed until the procedure exits. If the application programmer feels like, (s)he can use "Application.ProcessMessages" to allow for handling all scheduled events while the running event handler is temporarily halted at this point of execution (i.e. calling all scheduled event handlers like subroutines.)

Obviously the Event Queue functionality is provided by the TApplication object (that in turn uses the underlaying OS an widget set APIs), i.e. by the "Application" instance that is auto-created for the main thread of a user application by the initialization code in the LCL and initialized in the project source code.

Placing Events in the Queue

With Lazarus (and Delphi), the Event Queue is fed from multiple sources

  1. GUI events
  2. Timers
  3. Events generated by other threads of the application by means of "PostMessage()", "TThread.Synchronize()" and (right now not implemented in Lazarus / fpc) "PostThreadMessage()", and "TThread.Queue()".
  4. Only with Windows: Events from other Application, that do "PostMessage()", sending a Windows Message to the main Window of our application.
  5. Moreover, Lazarus (but not Delphi), with "TApplication.QueueAsyncCall()" offers a "non Windowish" method to queue an event that explicitly schedules an event handler function given as an argument. Other than TThread.Queue() or PostMessage(), same can provide a large data record with the call.

Handling Events from the Queue

The runtime libraries see that the sleeping application main thread gets woken up whenever the Event Queue is not empty. Now the events are taken from the queue and handled one after the other in the same order as they had been queued.

When scheduled, the event handlers are called by the LCL (or Delphi VCL) by means of the "Dispatch" mechanism of the main form, thus reaching the code of all controls it owns. The user code can use the Controls' event properties to create derived handlers for the predefined "On..." events or do "procedure ... message" to hook into the Dispatch mechanism and create an event handler for "Windowish" message-type events already defined or not defined in the controls owned by the main Form. Additionally, events generated by TThread. Synchronize(), TThread.Queue() or TApplication.QueueAsyncCall() fire the event handler function given in the argument. Without using the Dispatch mechanism.

Ways of implementing the Event Queue

The Event Queue is done in different ways for the different OSes.

Windows

Windows provides a "Message Queue" to user applications. (In fact it provides a Message Queue with any of the application's windows or threads, so a Windows application can have multiple message queues.) Accordingly the LCL (and the Delphi VCL) uses this Message Queue to implement the Event Queue for the application's main thread. Windows automatically places GUI events, generated by application's main window's controls, and Timers in this queue (by means of Windows Messages). Application programs (the owner of the window as well as other applications) can schedule events ("send user Messages") by the "PostMessage" API call. Worker threads additionally can use the slightly faster "PostThreadMessage" call to notify the main thread (or another thread of our application's).

Windows provides an API to allow a user application (or a thread of same) to wait for Events (here: Windows Messages) in the queue and take them from the queue (aka "receive a queued Message").

other OSes

With Linux and Mac OS the event queue needs to be done in Pascal code within the LCL. The GUI events generated by callbacks of the different widget set libraries are used to feed this queue, together with events generated by TTimers, and the "PostMessage()", "PostThreadMessage()", "TThread.Synchronize()", "TThread.Queue()" (if once implemented) , and "TApplication.QueueAsyncCall()" functions.

Some Widget Sets don't provide an event queue at all, and thus "Event Driven Programming" can't be done, TTimers can't be used and thread communication can only be done in the programmer's propriety way. These Widget Types can be used to create programs like "command line tools" and "run once per call" CGI applications.

Suggestion

Due to historical reasons the code that creates/manages the event queue is done directly in the appropriate incarnation of the Widget Type code. This makes creating new Widget Types more complicated than necessary and up to now prevented the creation of a Widget Type that does provide an event queue but does not need a binding to a local GUI.

The dual target of the Widget Type code (and with it the TApplication object) — 1.: GUI Controls and 2.: general Event processing — and the fact that the GUI Control handling depends on the external widget set used, while the Event processing depends only on the OS the application is supposed to run on, clearly suggests to split this functionalities and implement them in their own objects and source code units.

By this move, there supposedly would only be two different Event Queue implementations, one for Windows and one for all other OSes. The Windows implementation would continue using the Windows Message Queue in it's Event queuing mechanism, while the other incarnations would implement an Event Queue in Pascal code, as before.

Supposedly, the "Windows" Widget Type implementation would not need a great paradigm change, but mainly some code would need to be moved to different files and directories and some interfaces between the different units need to be added. The actual code for the Event Queue implementation will be rather short, as Windows provides most of the functionality out of the box.

With the other Widget Types, the necessary changes would be a lot more drastic, as the Event Queue implementation needs to be untangled from the other code and only one completely independent implementation is supposed to be maintained. It should have a well defined interface for the GUI-centric part of the Widget Type implementation and an OS independent interface towards an OS depending layer that provides the necessary system (OS and maybe Widget Set) APIs for having the process sleep and be waked.

On the Linux and Mac OSes, the Basis of the Pascal implementation of the event Queue could be the TApplication.QueueAsyncCall() functionality. On top of this, GUI event handling, Timer event handling, PostMessage(), TThread.Synchronize, and TThread.Queue() can be implemented for all widget Types, with code that is different per-Widget-Type only if necessary.

On Windows, the Windows Message Queue should be used (no other than with the current implementation), and TApplication.QueueAsncCall() — vice versa — needs to be implemented using same (supposedly this already is in place).

Pros and Cons

Regarding the Cons, it is obvious that it's a daunting task to restructure the mostly decently working code of the multiple Widget Type implementations, that in the end would all need to come up in a new incarnation.

Moreover it might be sensible or even necessary to rely on some small changes in the fpc RTL, as well (e.g. providing "TThread.Queue()", as already requested in the bug tracker). I feel that the best structure would be to move the basics of the Event Queue implementation into the fpc RTL, so that event driven programming and timers are possible even without Lazarus, but this of course would need a lot of cooperation between the Lazarus and fpc teams.

The Pros include:

  1. fewer code lines, due to the central implementation of the Event Queue code (not with Windows, though).
  2. a better structure and maintainability of the functionalities related to all Widget Types.
  3. creating a new Widget Type gets a lot easier, as the Event Queue mechanism already is in place.
  4. enhancing the TApplication class code in a way that additional Application objects can be instanciated by threads to allow for (optional) event driven programming in worker threads would be a lot easier than it is now. This in the end could be used to enhance fpc with "parallel loop" type of constructs to make decent use of SMP machines.

In fact, several new Widget Types could be created that could enhance the usability of Lazarus a lot:

  1. "Active NoGUI" for "headless" embedded applications (on Linux) with TTimers, TThreads, and event driven programming
  2. "Active FastCGI" allowing "free running" fast CGI application (on Linux) with TTimers, TThreads, and event driven programming. Same could be enhanced with a remote GUI based on technologies like HTML, Java Script (ExtJS), CIL-Pascal (Delphi-Prism on Silverlight/Moonlight), ...
  3. "Android": a "near-remote" GUI done in Java, based on Android's Java interface
  4. "ifi": ( (c) Martin Schreiber) a remote GUI done by a remote Lazarus program using the normal Lazarus GUI controls, accessed by the embedded program via a proprietey bidirectional stream of codes.
  5. ...

Possible Proceeding

The "Windows" Widget Type could quite easily be modified according to the paradigm described here, but (other than a slight code cleanup) the benefit would be rather small, as with Windows, dropping the GUI binding does not make any sense, as it is always in place anyway, and the Event Queue code is Windows specific (and rather minimal) anyway. (In fact I do know close to nothing about Win CE, so somebody more knowledgeable might be inclined to comment.)

As the "normal" Linux and Mac Widget Type implementations are decently working, very complex and have a close "legacy"-coded binding to the appropriate external widget set library, it will need a lot of programming and testing work to transform them according to this new paradigm.

So a good candidate for a first implementation seems to be the "fpGUI" Widget Type on Linux. Same does provide a working Event Queue implementation, is already very promisingly functional but still is "bleeding edge" code, so that it does make sense to perform a small paradigm change. Moreover it does not use a complex binding to an external widget set library, but uses a much less complex interface to a "painting" library, which even is intended to be reprogrammed, so that not only X11, but other frame buffer implementations can be accessed.

Additional Comments

Theoretically it is possible to enhance the code of the TApplication class (and friends) in a way that a worker thread, too, can instantiate an Application object, with this get it's own message queue and thus allows to be programmed in an "event driven" way as a set of "run to completion" objects, using the same programming paradigm as when programming main thread code. Of course the event queues of worker threads are not supposed to receive GUI events (which would ask for a reentrant implementation of many complex LCL functions: possible but not worth the effort), but it could receive TTimer events and events generated by other threads. This would help the average programmer a lot when (s)he needs threads for his/her application.

Moreover this could be the basis of introducing parallelism into the fpc compiler, hopefully following the syntax already existing in Delphi Prism, such as parallel loops, future variables, etc. For this, obviously, the Event Queue implementation should be done in the RTL rather than in the LCL.

Retrieved from "https://wiki.freepascal.org/index.php?title=Proposal_of_a_Widget_Set_independent_Event_Queue_implementation&oldid=47297"