Much of the work on interactive systems design has been concerned with analyzing the interactive process with a view to abstracting and classifying those human factors that are considered essential to a successful system. A considerable effort has also been spent on describing individual devices and techniques. However, the system builder has been left largely on his own to decide what mix of devices, techniques, and human factors considerations he would bring to bear on any given application. Furthermore, techniques for ensuring that a system with desirable attributes is in fact realized are neither well understood nor documented. This paper takes the position that the time is ripe to establish a constructive methodology for the synthesis of interactive graphics systems. The topic of the algorithmic structure of systems is identified as one of the areas needing further attention before such a methodology can be established.
This position paper sets out several questions and topics for discussion, without providing any concrete answers. The reasons for this are twofold. First it is essential that a set of relevant and provocative questions be agreed upon at the outset to guide discussion and to measure progress at the Workshop; and second, I do not have any well developed answers to offer. My position, then, is on what is, and is not, a set of relevant topics and goals for the Workshop.
A great deal of effort has been spent on the subject of graphical man-machine communication. This work has been concerned with many aspects of the problem of understanding the nature of interactive systems, and a wealth of knowledge and opinion has been accumulated. Keywords and phrases which describe the areas of concern include: Human factors: comfort of the workstation, orientation of the display, continuity of conversation, correct pacing, user vs machine control of flow, understanding the user's model of the process, semantics in terms of the user's domain, provision of feedback at appropriate times and in the user's terms, benefits of modeless commands, provision of friendly and forgiving systems, ability to recover from mistakes, confirmation of dangerous actions, choice of expression, extensible command languages, use of hierarchical languages, use of novice and expert modes for prompts; Devices: terminal/satellite/stand-alone configurations of processors, buttons, tablets, light pens, joysticks, shaft encoders, virtual devices, locators, valuators; and Techniques: use of menus, windows, dragging, motion, depth cueing, stero pairs, color, feedback of input actions.
This effort may be categorized into the following two areas:
Some publications have gone further than this categorization suggests. For example, Foley and Wallace (1974) not only give a good discussion of many of the above considerations, but also give several principles for the design of natural interfaces to the user. They suggest, for example, that interactions should be expressed in sentences, which express semantically atomic actions. Consequently each sentence should be self-contained and expressible without interruption to search the screen for some intermediate piece of information or to search for a different input device. Several other recommendations of this type are given.
While it is true that each of the above areas has received considerable attention, and is the subject of continuing work, it is also the case that much of the work reported does little to directly help the system builder in producing a good interactive system. As Foley and Wallace say:
While these virtues are sometimes countervailing, and often difficult to achieve, successful graphic conversational languages include these properties in large measure and deny them grudgingly.
But this is precisely the difficulty - the virtues of good interactive systems are very difficult to achieve, either because the requirements are conflicting, or because system building techniques for realizing such properties are not obvious, and have not been adequately investigated or documented.
Interactive programs are well understood at the micro level, i.e. at the level of individual techniques and algorithms. However, the way in which the individual parts hang together to form a coherent whole is not well understood. This problem is addressed by Software Engineering, which studies the methodology of building large software systems. However the additional requirements and constraints imposed by highly interactive systems have not received much attention. Perhaps the most obvious of these troublesome constraints are concerned with performance, due to real time requirements and the large amounts of information involved in manipulating graphical entities.
The principal question underlying any investigation of interactive techniques is: How can better interactive systems be built?. The term better has various interpretations, but may be taken to mean better human factors. A direct consequence of this question is the need to identify what types of guidelines are needed by a system builder in order to help him to design better systems. This, in turn, implies the need for some model of the design process, and an understanding of what aspects of it are amenable to general recommendations and guidelines.
The basic problem facing the system builder is one of synthesis, whereas the majority of work on interaction has been either analytical in nature, i.e. has dealt with identifying desirable human factors, or has been concerned with tools as opposed to systems. Stated another way, present day recommendations tell what human factors a system should have, but say little or nothing (above the level of tools) about how they can be achieved.
For example, consider the recommendation that systems should allow each step of a multi-step transaction to be corrected, and even to allow for the whole transaction to be aborted at each step. While it is clear what this means in terms of the user interface, many systems do not provide such a facility. The reason for this is that facilities of this type require major overhauls of existing programs for their inclusion, and are not easy to incorporate even if considered at the outset.
Given the present state of understanding of interactive graphics systems, the goal should now be to produce directly relevant and constructive recommendations to guide builders of interactive systems in achieving some reasonable optimization of the many potentially conflicting requirements. This assumes a model of the design process in which requirements are established, these requirements are mapped onto techniques, which may conflict, such conflicts being resolved at the discretion of the designer. The recommendations, then, will take the form of a set of guidelines, keyed to a model of the design process, and presented in such a way that for any given application a system designer can readily find advice on appropriate techniques and recommendations for resolving conflicts. The combination of a model of the design process and the set of recommendations will constitute a design methodology for interactive systems.
It is proposed that first a model of the process of designing interactive graphics systems be established. The model must identify the various processes and abstractions involved in the design process sufficiently clearly to support the subsequent analysis and categorization of those processes and abstractions.
One area which is crucial to giving directly useful guidelines, but which has received very little investigation, is concerned with the program structure of interactive systems. After the requirements have been established, the formulation of the overall structure of a system is the first concrete step taken in realizing an implementation of an interactive system. System structure has a strong influence on the resulting system, not only in terms of subsequent modifiability and extensibility, but also in terms of the support of good human factors, since the behaviour of the system will reflect the system's structure and therefore the user's model of that structure. Furthermore, choice of system structure embodies many of the decisions concerning global conventions.
It is proposed that an effort should be introduced that is directed towards producing a taxonomy of system structures. Such a taxonomy would serve several functions. In addition to providing the system builder with a concrete starting point from which the implementation of any particular system can directly be derived, it would provide a framework within which design decisions and recommendations for the resolution of conflicts can be discussed.
It will be necessary to identify and classify the relevant parameters influencing structure. It is expected that the optimum structure of a system would be dependent on considerations such as hardware configuration, degree of interaction, type of end user, etc. Each structure must be responsive to three categories of needs:
Several models of the design process should be proposed and discussed. The important processes and abstractions must be identified. The generality and applicability of the models should be investigated using several case studies.
As to the work on system structure, first the dimensions of the space must be defined, i.e. the parameters that influence program structure must be identified. Initially these parameters will not be grouped into a minimal set. This is because the optimal grouping is dependent on the ability of the groups to partition system structures into a reasonable number of categories.
Second, the structures of existing interactive systems which are considered successful should be examined and classified according to the parameters. This exercise will no doubt cause the set of parameters to be extended.
Third, the resulting set of system structures will be examined to see how it may be made to better respond to the requirements listed above.
Fourth, the sets of parameters and structures will be condensed to their minimal forms.
Some preliminary work on system structures was done at the first Seillac Workshop, though there the motivation was to analyze the effects of various proposed standards on system structure and hence on portability. An important lesson from that exercise was that while interactive systems could be categorized for the purposes of distinguishing various structures, the effort required to complete the task was considerable. The goal of the present exercise is to identify system structures for the purposes of showing how various desirable properties can be combined and realized in an actual implementation. This may or may not involve a major effort over an extended period of time.
1. Alden, D. G., Daniels, R. W. and Kanarick, A. F. Keyboard Design and Operation: A review of the Major Issues. Human Factors 1, 4 (1972), 275-293.
2. Armit, A. P. A language for interactive shape design. In Graphic Languages, Nake and Rosenfeld Editors, North-Holland, Amsterdam, Netherlands, (1972), 369-380.
3. Baker, C. L. Joss: Introduction to a Helpful Assistant. Memorandum RM-5058-PR, Rand Corporation, Santa Monica, CA., July 1966.
4. Baskin, H.B., and Morse, S.P. A Multilevel Modelling Structure for Interactive Graphic Design. IBM Systems Journal, Vol. 7, -#3/4, 1968, p. 218.
5. Batter, J. J., and Brooks, F. P. Jr. GROPE-1: A computer display to the sense of feel. Proc. IFIP Congr, Booklet TA-7, 135-140.
6. Bebb, Joan. An Interactive Environment for Scientific Development. Computers and Graphics 1, 4 (Dec 1975), 303-307.
7. Bcnnct. J. I.,. Integrating users and decision support systems. Proc. 6th and 7th Ann. Conf. Soc. for Man. Info. Sys. While Editor, Chicago, III. (1976), 77-86.
8. Boehm, B. W. Seven. M. J. and Watson. R. A. Interactive Problem-Solving - An Experimental Study of 'Lockout Effects'. Proc. SJCC (1971). 205-210.
9. Brcmer, Robert W. The Frictional Interface Between Computers and Society. Computers and-People 24, 1, (Jan 1975), 14.
10. Carlson, E. D. A Summary of Graphics Terminal Requirements Derived From Application Studies. Ref. in Computer Graphics 10. 2. (Summer 1976), 67. Proc. S1GGRAPH 76. Third Annual Conference on Computer Graphics, Interactive Techniques, and Image Processing.
11. Chapanis, A., Ochsman, R. B., Parrish, R. N. and Weeks, G. D. Studies in Interactive Communication: I. Human Factors 14. 6 (1972), 487-509.
12. Chapanis, A. Interactive Human Communication. Scientific American 232, 3 (March 1975), 34-42.
13. Clark, J. H. Designing Surfaces in 3-D. Cumm. ACM 19, 8 (August 1976), 454-460.
14. Davis, J. R. Device Independent Graphics Software. Is it Possible? Computer Graphics 9, 1, (Spring 1975), 232. Proc. SIGGRAPH 75, Second Annual Conference on Computer Graphics and Interactive Techniques.
15. De Fanti, Thomas A., Sandin, Daniel J., and Nelson, T. H. Computer Graphics as a Way of Life. Computers and Graphics 1, 1, (May 1975), 9-15.
16. Dunn, R. M. Graphics, Problem Solving and Virtual Systems. Proc. NCC (1973), 23-30.
17. Edmonds, Ernest, and Lee, John. An Appraisal of Some Problems in Achieving Fluid Man/Machine Interaction. Proc. The European Computing Congress Conference, (May 1974), 635.
18. Ellis, T. O., Heafner, J. F., and Sibley, W. C. Interactive Man-Machine Communication. Instrumentation and Control 44, 1, 92.
19. English, W. K., Engelbart, D. C. and Berman, M. L. Display Selection Techniques for Text Manipulation. IEEE Trans. Human Factors Electron., HFE-8, (1967), 5-15.
20. Flood, Merril M. Coupling Man and Computer. In Man and Computer, M. Marois, Ed., 421-428.
21. Foley, J. D., and Wallace, V. L. The Art of Natural Man-machine Conversation. Proc. IEEE. (April 1974), 462-471.
22. Forrest, A. R. Curves and Surfaces for Computer-Aided Design. Ph.D. Thesis, Cambridge University, UK, (July 1968).
23. Franklin, J. L., and Dean, E. B. Interactive Graphics for Computer Aided Network Design. Proc. AFIPS 1973 National Computer Conference, Vol. 42, 677-68.
24. Gaines, B. R. and Facey, P. V. Some Experience in Interactive Systems Development and Application. Proc. IEEE, 63 (1975), 894-911.
25. Gammill, R. C. Applications of Relational Data Structure Models in Man/Machine Systems. Proc. ACM 1972 Annual Conference, 460-467.
26. Gammill, R. C. and Robertson, D. Graphics and Interactive Systems Design Considerations of a Software System. Proc. AFIPS 1973 National Computer Conference, 657-662.
27. Gilb, Tom. and Weinberg, Gerald. Humanizing Data Entry by Default. Datamation 22, 8, (Aug 1976), 73.
28. Gordon, I. E. and Winwood, M. Searching through letter arrays. Ergonomics 16, 2, (1973), 177-188.
29. Griffin, J. Design Considerations for a Low Cost Graphic Computer Terminal. Hardware and Software Compromises. International Conference on Remote Data Processing, (March 1969), 52.
30. Haines, R. F. and Gilliland, K. Response Time in the Full Visual Field. J. of Applied Psychology 58, 3 (1973), 289-295.
31. Hansen, W. J. User Engineering Principles for Interactive Systems. Proc. AFIPS FJCC 39, AFIPS Press, Montvale, N. J., (1971), 523-532.
32. Hebditch, D. L. The Human Component in On-Line Systems. Proc. The European Computing Congress Conference, (May 1974), 689.
33. Herot, C. F. Graphical input through machine recognition of sketches. Proc. Third ACM/S1GGRAPH Conf. on Computer Graphics, Interactive Techniques and linage Processing, (1976).
34. Hoffmann. M. A. and Heimstra. N. W. Tracking Performance with Visual, Auditory or Electrocutoneous Displays. Human Factors 14, 2 (1972), 131-138.
35. Hornbuckle, G. D. The computer graphics user/machine interface. IEEE Trans. Hum. Factors Electron. HFE-8, (March 1967), 17-20.
36. Howden, W. E. Solution Plans and Interactive Problem Solving. Computers and Graphics I, (May 1975), 21.
37. IEEE Transactions on Human Factors in Electronics, Vol. HFE-8, 1, Special Issue on Human/Machine I/O.
38. Institution of Electrical Engineers (England). Conference on Man-Computer Interaction, (Sep 1970).
39. Irani, K. B., Wallace, V. L., and Jackson, J. H. Conversational design of stochastic service systems from a graphical terminal. Proc. 1970 Int. Symp. on Computer Graphics, Parslow and Green Editors, New York, Plenum, (1971), 91-107.
40. Irby, Charles H. Display Techniques for Interactive Text Manipulation. Proc. AFIPS 1974 National Computer Conference, 43, 247-255.
41. Kaplow, Roy, and Schneider, David. Computer Assistance for Writing Interactive Programs, TICS. Proc. ACM 1973 Annual Conference, 309-315.
42. Keast, D. N. Survey of Graphical Input Devices, Machine Design, (August 1967), 114-120.
43. Lecarme, Olivier. A System for Interactive Graphic Programming. Proc. IFIP Congress 71, 440-444.
44. Long, J. Effects of Randomly Delayed Visual and Auditory Feedback on Keying Performance. Ergonomics 18, 3 (1975), 337-347.
45. Mabry, Frank J.; and Levy, Alien H. Integrating Small Computers Within a Conversational Timesharing Environment. Proc. ACM 1972 Annual Conference, 183-186.
46. Marois, M., Editor. Man and Computer. North Holland/ American Elsevier Pub. Co., New York, (1975).
47. Martin, James T. Design of Real Time Computer Systems. Prentice-Hall, Englewood Cliffs, NJ, (1967).
48. Martin, James T. Design of Man-Computer Dialogues, Prentice-Hall, Englewood Cliffs, NJ, (1973).
49. Meadow, Charles T. Man-Machine Communication. John Wiley & Sons, New York, (1970).
50. Miller, R. B. Response time in man-computer conversational transactions. Proc. AFIPS Conf. 33, AFIPS Press, Montvale, N. J., (1968), 267-277.
51. Miller, S. W. Display Requirements for Future Man-Machine Systems. IEEK Transactions on Electronic Devices, Ed-18, 9, (Sept 1971), 616.
52. Neisser, U., Novick, R. and Lazar, R. Searching for Ten Targets Simultaneously. Perceptual and Motor Skills 17, (1963), 955-961.
53. Neroth, Chacko C. A Graphical Programming System with Speech Input. Computers and Graphics 1, 2/3, (Sept 1975), 227-231.
54. Newell, A. and Simon, H. A. Human Problem Solving. Prentice Hall (1972).
55. Newell, M. E, Man Machine Communication in Three Dimensions. Computer Aided Geometric Design, Barnhill and Riesenfeld Editors, Academic Press (1974), 303-315.
56. Newman, W. M. A System for Interactive Graphical Programming. Proc. SJCC 32 (1968), 47-54.
57. Newman, W. M., and Sproull, R. F. Principles of Interactive Computer Graphics. McGraw-Hill, New York, (1973).
58. Nickerson. R. S. and Elkind, J. I. Human Factors and the Design of Time-Sharing Computer Systems. Human Factors 10, 2 (1968), 127-134.
59. Online 72. Proc. of the International Conference on the Design and Application of Online Interactive Computer Systems. Online Computer Systems, Ltd., Middlesex, England, (1972).
60. Ortony, A. A system for stereo viewing. Comp. J. 14, (May 1971), 140-149.
61. Over 2600 Computer Applications. Computers and People, 23, B, (Aug 30, 1974), 145.
62. Pager, David. On the Problem of Communicating Complex Information. Comm. ACM 16, 5, (May 1973), 275.
63. Pask, G. Conversation, Cognition and Learning. Elsevier Press, Amsterdam and New York (1975).
64. Pooch, Udo W. A User-Oriented Computer Graphics System. Computer Graphics 10, 2, (Summer 1976), 25. SIGGRAPH 76, Proc, of the Third Annual Conference on Computer Graphics, Interactive Techniques, and Image Processing.
65. Prince, M. David. Interactive Graphics for Computer-Aided Design. Addison-Wesley, Reading, Mass., (1971).
66. Pyke, T. N. Jr., Treu, S. and Hudson, J. A. Performance Measurement of Interactive Graphics Systems in Support of Computer Aided Design and Engineering. National Bureau of Standards Report 10451 (June 1971.)
67. Reiber. J. E., and Shaw, A. C. Interactive Picture Generation and Manipulation through Formal Descriptors. Computers and Graphics 1, 1, (May 1975), 95.
68. Reisner, P., Boyce, R. F. and Chamberlin, D. D. Human Factors Evaluation of Two Data Base Query Languages - Square and Sequel. Proc. NCC (1975), 447-452.
69. Robbins, M. F., and Beyer, J. D. An Interactive Computer System Using Graphical Flowchart Input. Comm. ACM 13, 2, (Feb 1970), 115.
70. Rouse, W. B. Design of Man-Computer Interfaces for On-Line Interactive Systems. Proc. IEEE, 63 (1975), 847-857.
71. Sackman, H. Time-Sharing versus Batch Processing: The Experimental Evidence. Proc. SJCC (1968), 1-10.
72. Sackman, H. Man-Computer Problem Solving. Auerbach Publishers, Inc., New York, (1970).
73. Sexton, J. H. Multi-Level Software for Interactive Graphics and Computer Aided Design. Conference on Man-Computer Interaction, IEE (England), 181.
74. Shapiro, Stuart C., and Kwasny. Stanley C. Interactive Consulting via Natural Language. Comm. ACM 8, 8, (Aug 1975), 459.
75. Sprecher, L. T. Interactive Computer Graphics for Assisting Human Programmers. Computers and Graphics 1, 1, (May 1975), 17.
76. Sterling, Theodor D. Guidelines for Humanizing Computerized Information Systems: A Report from Stanley House. Comm. ACM. 17, 11, (Nov 1974), 609.
77. Sutherland, I. E. SKETCHPAD A man-machine graphical communication system. Proc. AFIPS SJCC 23, (1963), 329-346.
78. Taggart, J. Sketching, an Informal Dialogue between Designer and Computer, in Computer Aids to Design and Architecture, Negroponte Editor, Mason/Charter, New York (1975).
79. Testa, Charles J. Behavioral Factors in Information Systems. Computers and People 23, 4, (Apr 1974), 13.
80. Testa, Charles J. and Dearie, Douglas B. Human Factors in Design Criteria in Man-Computer Interaction. Proc. ACM 1974 Annual Conference, 61.
81. Thornhill, D. E., Stotz, R. H., Ross, D. T, and Ward, J. E. An Integrated Hardware-Software System for Computer Graphics in Time-Sharing. MIT Project MAC, TR-56, (Dec 1968.)
82. Tomeski, Edward A., and Lazarus, Harold. People Oriented Computer Systems. Van Nostrand Reinhold Co., New York, (1975).
83. Treu, S. (Editor). User-Oriented Design of Interactive Graphics Systems. Proc. ACM/SIGGRAPH Workshop, Pittsburgh, Pa. (October 1976).
84. Vartabedian, A. G. Human Factors Evaluation of Several Cursor Forms for use on Alphanumeric CRT Displays. IEEE Trails, on Man-Machine Systems, MMS-11, (1970), 132-137..
85. Wasserman, Anthony I. The Design of "Idiot Proof Interactive Programs. Proc. AFIPS 1973 National Computer Conference, M34-38.
86. Wright, Thomas J. Practical Computer Graphics for Scientific Users - Philosophy and Implementation. Computer and Graphics I, 2/3, (Sept 1975), 157-160.
87. Yourdon, Edward. Design of On-Line Computer Systems. Prentice-Hall, Englewood Cliffs, NJ, (1972).
