Informatics Informatics Department Methodology of Window Management

Jump To Main Content

Jump Over Banner

Home

Jump Over Left Menu

6 Issues in Window Management design and Implementation

Brad Myers

6.1 INTRODUCTION

Sapphire (the Screen Allocation Package Providing Helpful Icons and Rectangular Environments) is a very powerful window management system running on the PERQ personal workstation. Design for the system started early in 1982 and it has been a product of PERQ Systems Corporation since mid-1983. A description of the user interface for Sapphire is given in [44] while some experiments to judge the effectiveness of the progress bars is given in [43]. This paper will concentrate on the design decisions that were made during the development, how such design decisions interact and, finally, the issues that need to be addressed by future designers of window management systems.

The term window will be used in this paper for the rectangular environment manipulated by the window manager. Window is a term that causes confusion due to its use in graphics with a different meaning. The same applies to viewport which more closely approximates to an area of the screen in graphics. It is probably necessary at some stage to come up with a different term.

Adjectives associated with windows that also cause confusion are active and current. Most window managers are used on computer systems with a single keyboard and mouse with multiple windows displayed on the screen. Many systems connect the keyboard to one window at a time and that window is called the active or current window. However, all the windows are current as they are being displayed. In some window managers, including Sapphire, output can be sent to multiple windows concurrently, no window is more active than another. Consequently, I will use the term listener to define that window which is currently associated with input coming from the keyboard. Under certain circumstances in Sapphire, input from the mouse will be associated with the listener irrespective of whether the mouse coordinates are within the listener's window boundary or not. Later, it will be necessary to consider the problems that arise when multiple input devices are provided.

Sapphire is a much more complete window management system than most. The options available at the application program interface are the minimal set that provides the necessary functionality. On the other hand, the user interface provides a high degree of functionality based on the view that the window manager should provide more information and control to the user than most of the earlier systems. Consequently, this paper will concentrate on those aspects of the system associated with the user interface.

USER INTERFACE

6.2.1 Sapphire Icons

A major design goal for Sapphire was that the window manager should provide more information to the user than just the output displayed in the rectangular window. Sapphire, like many window managers, adopts the covered window paradigm which allows windows to overlap on the screen. Windows can be thought of as pieces of paper on a desk. A window may be on top of another window just as one piece of paper may be on top of another piece of paper. Commands must be provided to allow the user to bring certain windows to the top and to specify which window is the listener. Even with these commands, it is sometimes difficult to manage a large number of windows. The screen gets cluttered and finding a desired window is difficult. Some systems (SUN, Macintosh) have defined an icon as a little picture, often at the bottom of the screen, to represent the window. The window can be shrunk until only its icon is visible. Unfortunately, the icons contain very little information and this limits their usefulness.

Figure 6.1

Figure 6.1
Full Size Image

Icons in Sapphire differ in that they provide a great deal of status information. The 64 × 64 square icon contains the following eight pieces of status information as shown in Figure 6.1:

  1. error indicator: a small bug appears in the top left of the icon to indicate an error has occurred.
  2. Keyboard: a keyboard appears at the middle top of the icon to indicate that the process is waiting for input.
  3. Exclamation mark: this is reserved for specific application-defined attention signals.
  4. Process name: the application program may optionally replace this with some other useful names.
  5. First progress bar: this is used by the application program to indicate what percentage of the current task has been completed. This progress bar is repeated underneath the title of the window itself. A random sequence of vertical lines changing in position is used when an application program is unable to identify how much of the task is complete. The changing random image at least indicates that the application is still executing.
  6. Second progress bar: this is used to indicate how much of the entire job has been completed. For example, it might indicate how much of a command file had been processed if this was how the application program had been started.
  7. Off-screen indicator: three dots at the bottom of the icon indicate that the window has been moved off-screen.

The border of the icon is shaded grey if the associated window is the listener.

The main reason behind Sapphire icons is the view that a user running many tasks at the same time gets confused or does not remember what the various tasks are doing. He needs some aids to control and monitor the different processes. Of these, the progress bar is the most important in that it tells the user whether the task is running, how long it has been since it started, and approximately how long it has still to run. A formal experiment [43] demonstrated that users preferred systems with progress bars. Novices felt better about the system since they knew that the command had been accepted and the task was progressing successfully. Experienced users had sufficient information to allow them to estimate completion times and, therefore, to plan their time more effectively.

Icons provide a good example of how one design decision affects another so that there is much less freedom in designing a system than is apparent when you start. For example, it was decided early on that the icons always provide useful information and so should always be on the screen even when the window is visible. The alternative paradigm of the icon as another representation of the window, with one or other appearing but not both, was rejected as this could lead to the position where the window was on the screen but covered and the icon off-screen. Consequently, no status information would be available to the user.

With a Sapphire icon, you have the problem of deciding whether commands such as move (reposition the item at a different position either on or off the screen) when applied to an icon is referring to the icon or the window associated with it. Is the command applied to the icon or effectively sent through it and applied to the associated window? One solution would be to duplicate all the commands so that you had both 'move icon' and 'move window' commands. Rather than do this in Sapphire, the decision was taken to group all the icons together in a single window. Commands such as move then only apply to windows. The decision to associate icons with windows rather than have only one or the other on the screen led to the decision to group the icons in a window.

The icon window behaves much like other windows so that the icons as a group can be moved around or removed altogether if the user does not like icons. The icon window can be covered by other windows. Its size and position can be changed. The icons in the icon window are not rearranged except on user command. When a window is deleted, the icon is removed but the hole is not filled until another window is created. Thus users can remember which icons go with which window by position.

User Control

To provide more user control in Sapphire, each function provides three different interfaces to the user:

  1. pop-up menu;
  2. direct pointing device button press;
  3. keyboard.

Sapphire has a conventional pop-up menu interface and all commands can be initiated in this way. A button press causes the menu to appear. The user picks an item from the menu and the menu disappears, restoring the picture that was underneath. Advantages of pop-up menus include the fact that they do not take up screen space and novices do not have to remember all the commands before they can get started. With a system such as Sapphire with a rich set of commands, the size of the pop-up menu can get very long and users find that it takes a long time to select a command.

The Sapphire accelerators provide a faster interface to these functions. The goal is to make the most common window manager commands very easy to specify, for example, with a single button press or keyboard key being sufficient. The title line of a window is divided into three sections with the left and right sections having the same effect. The puck has three buttons so that the two areas and three buttons give six commands that can be initiated by a single button press. Sapphire sets these six commands to the most commonly used ones. The actual assignment of commands to these six alternatives can be changed by the user.

Sapphire also provides a keyboard interface to the commands. As there is a large number of commands, a special prefix key is used which effectively sets the system in Window Management Mode and the next key press indicates the command to be obeyed. This facility was provided primarily for people who preferred not to use a pointing device. We were surprised to find that many users preferred to use the keyboard mode of input. However, it should not be surprising if you analyse the alternatives using the keystroke model. This is especially true for commands not available on the title lines.

One problem with keyboard input is that it turns the system into a modal one and people then become confused as to which mode they are in. Sapphire attempts to alleviate the problem by having the cursor picture change depending on the mode. Also, the cursor picture shows what command will be given, allowing users to abort commands before they are executed.

6.2.3 Issues

The experience with Sapphire has raised a number of design issues with respect to window management systems which can be categorized approximately as:

  1. user adaptation;
  2. integration with User Interface Management System (UIMS);
  3. interference between application and window management models;
  4. differences in window management models.

Particular users may have a mode of working which is desirable for them but which may not be generally applicable. For example, some users may like the top and listener commands to be distinct. The first makes a window uncovered and the second selects a particular window for accepting input. Other users find this quite confusing. They expect that when they bring a window to the top it is the one that is listening to them. The system must allow the users to tailor the user interface and this might be implemented with a User Interface Management System.

How does the User Interface Management System relate to the window manager? To tailor the window manager requires a UIMS. However, UIMSs require window management type functions. Real questions arise as to whether you integrate the two and, if not, which do you implement first?

Another issue is the interference between the user interface of the window manager and the user interface of the application running under the window manager. No matter how simple the user interface of the window manager is, it needs to specify functions such as which window is the listener, and commands such as top and move. For example, Sapphire requires a button press to change the listener while other systems just require the cursor to be moved into a window for that window to become the listener. Thus, in Sapphire you require an explicit press to change the listener and moving outside the window does not change things. The Sapphire window manager model is, therefore, one where presses are important events and movements are of less significance.

Compare this with applications where movement is the main operation. Many applications use pop-up menus where movement is important as it changes the highlighted menu item. In a Bravo-style editor, moving the cursor may well cause the cursor picture to change. We may thus have an application where movement is the dominant operation running under a window manager where button presses are the dominant operation or vice versa. In such cases, the user is likely to get confused.

A related question is whether commands to the window manager get passed to the application. For example, should a command to change the listener also get sent to the application for information? If it gets sent through and the application is not expecting it, that will cause problems. If it is never sent through, the user may do something expecting the application to respond and it does not. Interlisp-D allows the application to choose whether it gets told. However, this is equally confusing as that also can lead to inconsistent and confusing responses.

Finally, there is the point that all window managers are different. If you walk up to a random window manager having used another, you really do not know what to do. Thus, you can have portable applications which cannot be used because the window manager between the user and the application is completely different.

6.3 INPUT

An important problem is how input from the user is dealt with and passed to the application program. On the output side there are still many issues but it is basically a choice between several options all of which we know how to implement. On the input side, we really do not know the right thing to do nor do we have an adequate model of what we are trying to achieve. There are really only two choices - polling and queuing - for the input model and the third question is the complication caused by multiple input devices.

6.3.1 Polling

If polling is used as the input model, either the application program or some other process must sit in a tight loop asking all the relevant devices if they have any input. This works reasonably well in an environment where there is a single process. However, as soon as you have multiple processes it is not very efficient as you can use up a lot of processor cycles and lose characters and events.

Queuing

Queuing presents a number of problems due to the lag between the time the event occurs and the time it is interpreted. Existing operating systems queue keyboard input. Certain errors require the input buffer to be cleared and the lack of synchronization makes it difficult to know to what point you should clear out the buffer. A similar problem occurs with pop-up menus when you queue up the button presses. The user hits a button to select a particular menu item, nothing happens immediately so he hits it again. Meanwhile the system acts on the first button hit and then acts on the second button hit in the environment after the first button hit has been accepted. Consequently, the user gets totally confused.

In many systems, specific events such as abort cannot or should not be queued. The user does not want it to happen later. He may be trying to close down the system now or stop output going to a printer. To handle such events, you need an entirely separate mechanism.

Type ahead is also a problem in non-homogeneous systems. On a loaded system, different things may happen if the command is interpreted immediately by whatever is currently running rather than waiting until it reaches the front of the event queue. This occurs frequently in Unix systems with a program like Emacs. The Unix shell interprets everything immediately while a program like Emacs requires the Unix kernel to stop interpreting the characters and give them to it direct. Consequently, you have to learn not to type ahead when using Emacs. As we go to more sophisticated shells and more sophisticated applications, such problems are likely to arise even more.

6.3.3 Multiple Windows

A separate problem occurs when interfacing different processes to the same set of windows. If you think of the windows as corresponding to different terminals, they obviously need to have different queues and characters go separately to each window. However, when you have multiple windows for the same program, do you want one queue for all the windows so that you can keep the events synchronized or do you have separate queues for each window but timestamp the information so that you know which event came first?

A related issue when a program is using several windows is how you identify the listener. Two windows may be cooperating and input sent to one is passed by the application to the other for execution. The listener has to be specified as the first window yet the user sees the input being acted upon by the second and he gets a very confused model of the system. The problem is even worse if windows are divided into subwindows as the user may not even be aware of the subdivision.

6.3.4 Multiple Input Devices

While the window manager only has to deal with a single keyboard and pointing device, it is relatively easy to define the listener as a single window and allow both input devices to move as a group from one window to another. In the future, people are likely to realize that they have two hands, two feet, ears and eyes. Speech I/O, foot pedals etc will mean that the user has a variety of input devices from which to choose.

In such an environment, it is not clear that it is sensible to move all the input devices from one window to another as the listener changes. Some devices may only be used with certain windows in which case you require to move some devices with the listener while leaving others pointing at the old listener. The problems are: how do you specify which devices are attached to which window, and how do you let the user know?

6.4 IMPLEMENTATION

6.4.1 Picture Updating

Sapphire allows windows to overlap and that immediately raises the question whether you allow updates to the parts of the windows that are covered. Sapphire allows this and there are several ways in which it can be done depending on what you are optimizing for. The main issue is how to deal with refresh when a window becomes uncovered.

  1. Speed: the fastest way to refresh the picture is to have the complete picture off-screen and RasterOp it back on to the screen when it becomes visible. The disadvantage is that it takes a lot of memory for the off-screen images of windows.
  2. Memory: the application is asked to refresh the parts of the screen that become visible. The application probably has a much more concise way of representing the data held in the window. Usually, the application knows what it has displayed in the window. There are very few programs that cannot refresh the picture. After all, they generated it in the first place.

A compromise between the two solutions above is to save only those sections of the windows that are covered. This avoids having two copies, one off-screen and one on, of the information on the screen that is not covered.

Sapphire provides both of the choices. The application can choose to do the update or request Sapphire to save off-screen those parts of windows that are covered. This is done at the time the window is created. Sapphire also provides a third alternative which allows the application to specify neither. In this case, the screen is blank when the window is uncovered.

Interlisp uses a different scheme whereby a window saves the picture which is underneath it. This is very useful with pop-up menus which then appear and disappear very quickly. Sapphire also provides this facility calling such windows courteous. Thus windows in Sapphire either remember what is in them or what they are hiding.

Our experience has been that trying to provide both courteous and normal windows does not work very well. If you have windows that have off-screen memory and courteous windows overlapping each other, and you move one of the windows to the top, there is no order in which you can update the windows that will be correct. They all have to be updated first or you have to use separate buffers into which you move information temporarily.

6.4.2 Rectangle Lists

To identify what parts of windows are covered so that graphics can be clipped appropriately, most systems keep a rectangle list with individual items marked as covered or not. On a graphic operation, the window manager goes through and updates the covered portions in off-screen memory and the uncovered portions on the screen. Sapphire has such a list.

Deciding the order in which the rectangles are processed is analogous to the problem of shifting elements in an array. In that case, you either shift the elements from the top of the array first or the bottom depending on whether you are moving elements up the array or down. In the case of moving rectangles, the order in which RasterOps occur is different depending on whether you are moving the rectangle up and to the right, up and to the left, down and to the left or down and to the right. There are four different orderings in which the three subparts of the rectangle are moved. An example of moving the rectangle up and right is shown in Figure 6.2.

A B C move up and right slightly Rectangles must be moved in the order B C A

Figure 6.2

6.4.3 Graphical Output

The window management system needs to provide fast updating of graphical information in the windows. In Sapphire, we differentiate between graphical output to covered and uncovered parts of windows. For uncovered parts, drawing is performed directly to the screen memory with a clipping imposed by the kernel. For covered parts, updating is performed using the inter process communication mechanism of the Accent operating system.

The problems arise when you want to support even faster graphics by adding microcode to draw circles and other special graphics. It is difficult to see how protection of the various windows can be achieved and still make good use of the hardware.

6.4.4 Move and Grow

Some theoretical results give some order of relative cost of operations in a covered window environment. If N is the number of windows, a RasterOp is inherently of order N2 which contradicts the claim of Rob Pike that it is of order N log N.

Move and grow, which Pike decided not to provide in BLIT because they are too complicated, are provided in Sapphire and they are extremely complicated. The reason for the complexity is that an attempt is made to retain all the information that is possible. The picture is automatically redrawn in the new window. As both the source and destination of the window may be covered or partially covered, the picture update is complicated. A major question is whether it is sensible to attempt to redraw the picture. As the window has changed in size, the application is likely to want to redraw the window contents immediately to fit the new size window in which case the updating is immediately overwritten by the application. However, if the application has requested Sapphire to provide memory to remember the picture, you cannot assume that the application has the ability to redraw the picture.

6.4.5 Separate Processes

Several problems arise when you consider how the window manager is implemented with respect to the operating system with which it is associated. The Accent operating system is based on a system of asynchronous message-passing primitives and Sapphire uses these as its inter process communication mechanism. It is natural in this environment to think of the window manager as a process to and from which other processes send and receive messages. However, messages take a finite time to arrive and this can cause problems.

For example, imagine a file is being typed into a window with the text scrolling up the screen. Suppose another window partially obscures this window so that text disappears and reappears as it scrolls underneath this window. If the window manager is requesting the application to redraw the uncovered parts, by the time the message is received by the application, the text has moved and it is likely that the application will refresh the wrong piece of text. There is no way round this fixup problem without providing more synchronization. This may not be possible with some operating systems.

Another problem that the application has in a window management environment is whether it can get sufficient of the processor time to ensure reasonable feedback when it is working in a real-time environment. This may affect the way the application is structured as well as the window manager.

Finally, there is the issue of how closely the graphics package should be integrated with the window manager. Cedar, for example, is built on top of a fairly sophisticated graphics package and the APPLE Macintosh has a similar structure. The alternative approach is to assume the graphics package (say GKS) sits on top of the window manager. The problem is that the window manager itself does graphics. The move operation, titling of windows, drawing borders, constructing icons are all graphical operations and the window manager would like to use the flexibility of a full graphics package to do this. This is shown in the Macintosh where the windows are much more elaborate in terms of the pictures presented because it has a graphics package it can use. Integrating the graphics package and window manager produces a large system. If the window manager is to be portable it should sit on top of the graphics package and not vice versa.

6.5 SUMMARY

When you are designing a window manager, there are many tradeoffs and the design decisions are interrelated. Sapphire attempts to provide a rich user environment and this leads to complexity. We have found that users do make use of the options provided. In particular, progress indicators and cursor-picture changing have been much appreciated and used. As a result, applications running under Sapphire tend to have a richer user environment than if these facilities had not been provided by the window manager. Users tend not just to port an application without change but rework their application to make extensive use of the facilities provided.

6.6 DISCUSSION

Chairman - Austin Tate

Teitelman:
In Sapphire, you allow the application to tailor the environment including changing the icons. Will this not leave the user with seeing many different environments together rather than a consistent one?
Myers:
Sapphire is similar to the environment on the Macintosh in as far as you provide a bunch of facilities which you allow the application to change. The application is aware that if he does not use the standard facilities, it will look different to everybody else. There is a continuum between having no functionality and providing the entire interface. The claim is that the more you provide, the more the application is likely to use.
Teitelman:
You mentioned the additional complexity of specifying the listener when you have multiple input devices. Did all the devices move to a different window when the listener changed? Is this not an argument for not having so many devices?
Myers:
Not necessarily, the point is that the user is interacting with an application program far more often than he is interacting with the window management system. He creates a window, brings it to the top and then may interact with the application for half an hour. If you can provide a better interface to the application by providing six input devices, that is what you should be doing to make the user most effective. If another application also requires six input devices, that needs to be provided too. The window manager has to deal with this situation not because it is easy for the window manager and makes it better but because it makes the application programs better.
In Sapphire, we tried hard not to constrain what we gave to the application. It has access to all the buttons on the puck and all the keys on the keyboard while it is the listener so that it can have the best interface it can come up with. You should not say you can only have two buttons because the window manager needs the third. The window manager is not that important in the real scheme of things in terms of the user's requirements.
Rosenthal:
The property of being the listener for each input device potentially has to be separate to allow flexibility although it should be possible to cluster devices which move with the listener.
Butler:
How many Sapphire users are there?
Myers:
There are about 20 companies or so using Sapphire but I don't have any information on usage.