Graphic input classification based on devices and techniques is generally accepted today. This classification however is not adequate to accommodate as yet unknown graphic input devices and techniques, it also does not differentiate between graphic input and graphic interaction techniques. In this paper it is argued that classification on the basis of primitive and structured types of information input is more logical. One sucn scheme which has been incorporated in a general purpose interactive graphics system, SIGN is briefly described.
Graphic input lacks the well-established methodology that exists for graphic output, and to this date there is no complete solution to this problem. In the rest of this section I shall briefly state some of the basic reasons that I have felt are the cause for this. In the next section a possible solution to this problem is presented, one which we at NCSDCT have adopted in the design of a general purpose interactive graphics system, SIGN [7, 6, 11].
Basically graphic input has not received as much attention as it needs in general purpose interactive graphics system design [4,5,10,15]; while the graphic output process is burdened with a longish pipeline (segmenting, transforming, viewing, clipping, viewporting, etc. ), the input is usually shown by a single block designated as input handler. The number of general high level graphic output functions in any of these systems far outnumbers those available for graphic input. Or take the number of pages devoted to graphic input in any general text book on interactive graphics!
At the fundamental level at least in all information handling systems, the types of information handled will be the same for both input and output; this is reflected in general by the availability of complementary functions for input and output. Most graphics systems of today, known to me at least, are deficient in this regard, it is my feeling that the graphic input and even the well-established graphic output process at the basic information handling level need a more formal framework [8]. This paper however restricts itself to graphic input only.
Device interchangeability through specially programmed techniques is commonly used as an example of the problem in formalizing graphic input [1,3]. Consequently most of the earlier work appears concentrated on attempts to classify graphic input in terms of virtual and physical devices [3, 4, 5, 13] , or in terms of high level input functions (techniques) [10]. I would like to argue in this paper that these are not very logical classifications. Certainly, even more convenient devices and techniques are bound to come in future, which may be characteristically different from currently known devices and techniques. Classification on the basis of virtual devices and high level input functions lacks the essential feature of extensibility which is inherent in most accepted formalisms.
In non-graphical programming the keyboard device may be used to input:
and a variety of other kinds of information. This difference in the type of information input is not resolved by considering these inputs as coming from different virtual devices!
Analogously, the lightpen device can be used to input:
and a variety of other types of information. I do not think it is logical to assume that every type of information input is from a distinct virtual device. After all the set of the types of information being input via the lightpen or any other device is determined by the needs of the application. This set certainly should not be restricted to the types in use today. The solution, I think, can be found in much the same way as in conventional programming systems [2,14], that is:
The set of primitives and the structuring mechanisms as available in SIGN are discussed in the next section.
Perhaps the greatest cause of confusion has been due to our inability to distinguish between graphic input and graphic interaction. For every known interaction technique which results in the input of a value, an attempt is made to categorize it as one of a set of virtual devices. Worse still is the suggestion that all or some of these be included as part of the basic system software. Somewhat extreme examples are the light(pen) handle categorized as a valuator device and the successive entity brightening technique used in conjunction with 3 Keys as the pick device [3]. I would like to argue here that these input techniques are fairly complicated interaction procedures, which do not lie within the domain of basic system software for interactive graphics. The main reasons being:
These types of complex interaction techniques should logically be considered as programmable processes in the application software and not as basic general purpose system facilities.
Yet another extremely troublesome issue is that of echoing or providing visual feedback for the user's action of inputting a value. This feedback is as essential as character echoing is for an alphanumeric display terminal. Unfortunately providing graphical feedback is not as trivial as character echoing. In fact the inability to distinguish between different kinds of feedback for the same basic type of input (tracking, rubber band line, zig-zag line all input a coordinate position on the screen) has resulted in considering these as different input functions or as distinct virtual devices.
Ideally feedback should be provided by hardware and the user must be able to turn on and off this hardware feedback. For graphics devices at least this is not very feasible, particularly because, as I just stated, the same input action can have different forms of visual feedback associated with it.
The next best solution appears to be inclusion of feedback as part of the basic system software. This is not a very general solution for there is a limit to the number of different forms that the system designer can include.
The simplest solution then seems to be to let the user program the form of visual feedback desired as a user procedure. Unfortunately this leads to concurrent programming problems. The input actions are asynchronous with the user program execution; the feedback should be immediate and hence the associated feedback procedure may have to be executed concurrently with the user program. Most graphics applications are written in sequential programming languages.
An elegant but not very flexible solution has been incorporated in SIGN. Feedback to a structured type is provided by a segment of the display file known as the vf-segment. vf-segments are created for a specific graphic display device by users as a sequence of vf-output primitives and vf-modification operations. The vf-output primitives correspond to simple graphic output primitives (relative and absolute, visible or invisible vectors, text strings etc. ) except that they bypass the output pipeline and are specified directly in normalized device space. The vf-modification operators continuously modify the vf-segment which includes these operators. The actual modification carried out depends on the semantics associated with the operator with results of the input actions effectively forming the operands. The vf-modification operators compile into processor code that is executed repeatedly, i.e., at every vf-modification cycle, every vf-segment has a user defined reference position which is not modified as part of the vf-modification cycle. For further details the interested reader is referred to [12].
In SIGN there are classes of input and within each class there may be one or more units, which have physical associations. A SIGN user can access (read) the report of a unit. An input class is either an event class or a sampled class. The following points should be noted.
There are 3 primitive event classes:
and 2 primitive sampled classes:
In addition to the above five primitive classes the SIGN user may construct new structured classes as shown below:
The Set Class is an ordered set of known classes, called as the component classes. e.g.
position := <value, value> location := <position, clock>
Please note that this and other examples in this section are declared in a notation specific to this paper. The exact syntax of these specifications in SIGN is available in [12].
The following points about the set class should be noted:
The attributes of the set class are the attributes of the individual component classes.
The Sequence Class is an ordered limited sequence of events of any particular event class known as the component. The sequence class is always of the event type.
An event of the unit of the sequence class is said to occur when the terminating event of the sequence has occurred. The terminating event is one of the following
The report of a unit of the sequence class is an ordered sequence of reports of the individual events. e.g.,
curve := <sequence of location;
length := 50;
terminator := button;
location limits := {(0.,1.), (0.,1.)};
As already stated units of classes may be declared and units of primitive classes would have to be associated with physical devices before the reports of units are accessed. An important point to note is that the set of known physical devices do not directly correspond to the primitives classes in SIGN. In fact some of the known devices correspond to a structured class in SIGN. For example the tablet with a stylus and switch is the set class declared as below:
tablet stylus switch := <value, value, button>
In addition the SIGN user may declare a procedure known as the conversion procedure which converts the report of a unit of a particular class into the report of a unit of another class. The conversion procedure is invoked only at the time when the user requests access to the report: e.g., character-recognizer (character, curve).
The SIGN user can also declare own units of a class. The report of an own unit is set by the user program and not by the system. e.g., lightpen cursor may be an own unit of the class 'position' declared above.
Access to reports of input units may be enabled/disabled. If a structured class unit is enabled then access to reports of component units of that class unit is disabled. SIGN includes simple mechanisms for accessing the report of a structured class unit.
As already stated the visual feedback for a structured class unit is provided by a user defined vf-segment. vf-segments may be active or inactive. Only active vf-segments are modified. Whenever a unit is enabled the associated vf-segment becomes active. When an event on that unit occurs this vf-segment becomes inactive; however a new active vf-segment identical to the inactive one is created. When an event report is accessed by the user the associated inactive vf-segment is deleted from the display file. A vf-segment need only be associated with the set class. The visual feeddack for the sequence class automatically appears as part of the above vf-segment activation mechanism.
It is a pleasure to acknowledge the many helpful discussions I have had with my colleagues working on SIGN, namely Cu Sharma, SC Gupta and S Ramesh. I also gratefully acknowledge the quick and accurate typing of this paper by Shri J Britto.
1. I W Cotton 'Network Graphic Attention Handling' Proc. ONLINE, 1972.
2. O J Dahl, B Myhrhaug, K Nygaard 'The SIMULA67 Common Base Language' Publication 3-22, Norwegian Computer Centre, Oslo. 1970.
3. J D Foley, V L Wallace 'The Art of Natural Man-Machine Conversation' Proc. IEEE, April 1974.
4. Graphics Standards Planning Committee of ACM/SIGGRAPH 'Core Graphics System Specification', Computer Graphics Fall, 1977.
5. D Groot, E Hermans, L C Caruthers, J Schwartz "GPGS Reference Manual ' Th Delft and University of Nijmegen, The Netherlands, 1977.
6. S P Mudur, S C Gupta , C U Sharma, S Ramesh 'SIGN: Detailed System Specifications', NCSDCT Technical Report July 1978.
7. S P Mudur, S C Gupta, C U Sharma, S Ramesh 'Environmental Independence in a Graphics Programming System', Proc. Conference on Interactive Techniques in CAD, Bologna, Italy, September 1978.
8. S P Mudur 'Towards More Formal Interactive Graphics Software Specification' (under preparation).
9. W M Newman 'A Graphical Technique for Computer input' Computer Journal, May 1968.
10. W M Newman, iR F Sproull 'An Approach to Graphics System Design' Proc. IEEE, April 1974.
11. S Ramesh, S P Mudur, S C Gupta, C U Sharma 'Language interface and Error Recovery in SIGN : Data Structures and Procedures' NCSDCT Technical Report (submitted for publication).
12. C U Sharma, S P Mudur, S C Gupta, S Ramesh 'A Formal Framework for Graphic Input' WCSDCT Technical Report, October 1978.
13. V L Wallace 'The Semantics of Graphic Input Devices' Computer Graphics, April 1976.
14. N Wirth 'The Programming Language Pascal' Acta Informatica, January 1971.
15. P A Woodsford 'The Design and Implementation of the GINO-3D Graphics Software Package' Software Practice and Experience, October 1971.