5. VISUAL COMMUNICATION

• 5.1 Visual Communication and Interaction - N. Negroponte
• 5.2 Discussion

5.1 VISUAL COMMUNICATION AND INTERACTION - N. NEGROPONTE

The presentation addresses five topics. Following that, there will be a short discussion of two specific projects that characterise a certain style of research. The following areas will not be discussed but must be kept in mind:

1. There are many situations where people are dealing with computers when they only have vague ideas which they want to develop further using computers. Existing equipment such as the insidious rubber band line, light pen and calligraphic display absolutely destroy this kind of activity. They are totally unproductive in any area where the problem is incomplete, ambiguous or wrongly posed. At best, interaction has been done badly.
2. There has been, in the late 60s, work by Sutherland and people at the University of Utah, of making pictures, but completely ignoring factors like interaction and man-machine communication. As a result, very beautiful pictures were produced, but that is all.
3. New technologies are coming from large consumer communities (in particular, television) because the large market makes them economically possible. For example, you can now buy large video discs with a spot size of 0.8 microns and a tolerance of 120 angstroms. Such a video disc can contain up to 54000 images. This is more images than a PhD student in Art History will see in his entire education. A video disc contains more pages than the Encyclopaedia Britannica and all of its annual updates since it was founded. At home, I have a video disc containing 20000 art slides, 20000 architectural slides and 15000 travel slides. I have had it for a year and a half and I have not even begun to see every image on it.
4. Old technologies are now available at very low prices. For example, the Japanese have just announced the 1 Mbit chip. It is now possible to market a Speak and Spell toy consisting of 32 Kbytes, VLSI chip and voice synthesiser for only 36 dollars. Also of great importance is that it is portable. You can tell when my 9 year-old child goes to sleep. It is when Speak and Spell stops talking!

I would have liked to call this paper Can a Eunuch discuss Sex. We are all eunuchs as far as interaction is concerned. Very few people have yet experienced real interaction with a computer. Also, there are going to be new categories of users like 6-year-old children and presidents.

5.1.1 Personalisation

I have used this example before, how do people select hand calculators? All calculators cost about the same ($7-$12), they have common functions. You can play with them at the counter. People pick them up, play with them and finally select the one that feels good. The message here is that the human interface is the reason for selection. This also means that computers at home will have to meet different criteria. Users will be much less tolerant with respect to human factors.

Personalisation can come in two flavours; easy and somewhere between hard and impossible. Easy means to give sufficient modes and media so that people can select the one they prefer. Selection provides an important kind of personalisation. You can offer them either a tablet or a lightpen rather than one or other. Visual selection is also important. Some people prefer to work using plan projections while others would like perspective projections, photographs, or cartoons.

The hard flavour means that the machine knows the user. To do something about it is a classical AI problem. One can write good papers about the subject, which should be done. Yet it has been impossible to cross the gulf between trivial experiment and reality. Consider the following dialogue when the husband arrives home and greets his wife:

Husband:

OK, where did you hide it?

Wife:

Hide what?

Husband:

You know.

Wife:

Well, where do you think?

Husband:

Oh!

There is a great deal going on in that conversation. Both parties know what it means. The two people have shared experiences, shared metaphors, and can infer things from having access to the world at large.

The aim of our group of 35 people at MIT is to build computers for generals, presidents and six-year-old children. What they have in common is that they want things done immediately.

One of our interests is touch sensitive input devices. We have used flat and cylindrical, large and small devices. Touch sensitive displays are riddled with misunderstanding. The argument goes that the end of a finger is about half an inch across and the display area is quite small, therefore, it must be a low resolution device.

To refute that argument, take a one inch square piece of metal plate and place it on a table with a grain of salt underneath. It is possible to balance the plate on the grain after about a second of trying. We have extremely good motor control. On a 4096 × 4096 display, you can move a cursor with your finger with great accuracy. Fingers have two other important properties. Firstly, you do not have to pick them up. Secondly, you have ten of them. Both are important properties.

5.1.2 Quality

It is a myth that standard television has low resolution and that NTSC stands for Not The Same Colour. We need to take advantage of the 65 million colour television sets in the USA. The mass production of TV forces us to stay TV compatible. It turns out that one can trade z for x and y, resulting in an effective resolution of a million by a million pixels. By introducing 4 levels of grey, it should not be thought of as a dejagging artefact but as a method of increasing the resolution by trading z for resolution. Also, the grey tone stops the image scintillating and makes it much gentler for the NTSC encoding process.

A second myth says that one should not write programs in proportionally spaced character fonts. A proposition that we could make is that if your programming language does not allow proportionally spaced text then it must be the wrong language. Another experience about text is that as soon as text scrolls, you no longer know where you are. A book is very good in that sense. It indeed gives a sense of place. We simulate this by letting a user flip the pages (with sound!).

5.1.3 Modes and Media

Xerox and IBM are responsible for getting rid of colour from our working environment; Xerox for the inability to produce colour copying at reasonable cost; IBM for the absence of Red/Black carbon ribbons on their typewriters. The old typewriters used to be far superior. I have an old typewriter at home. I can write a memo and you can get an abstract of what it says by just reading the red words. My secretary, who has a 19in platinum, self-erasing, self-correcting, spelling, upside/down, ... typewriter does not know what to do when she gets to the red words!

Colour is becoming a standard way of working. The argument against colour of the high costs is now going away. Another resolution that could be made is that vector graphics is dead. A vector is just a degenerate rectangle.

5.1.4 Environment

A year and a half ago, we decided to get away from people sitting in front of displays. Instead, the whole space should be part of the environment. To do this, we produced a complete room with large display and octophonic sound. Sound is very interesting. Sound space is huge - considerably larger than picture space. Even the graphics off screen can still keep talking to you. Also, sound is not sequential as people imagine. If you associate a number of messages with a particular image, it is very easy to pick out the one you require if they are all played back together. We call this the Cocktail Party effect.

5.1.5 Projects

Here are two projects that we have been looking at recently.

In using a touch sensitive screen, the coefficient of friction between finger and glass is often too high to allow efficient drawing. You get a kind of sticking effect called stick-slip. Since we did not know how to solve this problem we turned the bug into a feature. We built pressure sensitive displays where the forces in three directions drive a three dimensional cursor. An arrow and a square indicate the x, y and z values. You can give the user feedback on the properties of objects. You can make it harder to push a tank than a jeep. It can also be used as a virtual knob. People are already running out of real estate on their front panels (for example, cockpit displays and oscilloscopes). We will need to go to soft front panels.

Flatness, bigness and thinness are three other properties that we have looked at. Of these three, flatness is a very special one. What does interaction with a flat screen mean? By that we mean absolute flat without any parallax. By looking through a half silvered mirror at a tablet, one can superimpose a flat display onto the tablet face. Normally, a user can tell that a line drawn on a display is not precisely coming from the drawing implement. With this new device, the user is really drawing on the surface of the paper. A fringe benefit is that his hand is transparent. As you draw, you do not obscure the image. Of great value to left-handed users.

The presentation concluded with a film of the Dataland system.

5.2 DISCUSSION

van Dam:

In your Dataland system you used to have a number of levels and now you talk about all the objects being at the top level. Did the layers cause problems?

Negroponte:

Yes, it tended to be confusing. We used to have an Alice in Wonderland type of system. You did go crashing through pictograms and icons, and you found things behind them. It was like lifting up a rock and finding a little subculture of bugs, ants, and worms underneath it. That had a certain impact but we felt it was the wrong way to go. You were working your way down a tree and, even if you could came zipping up to the surface, it was very much like scuba diving! Every time you wanted to go somewhere else, you had to come up for air and go back down again. We felt that was not as successful as trying to put everything right up front - right at the surface that you are dealing with.

We have broken the Spatial Data Management System (SDMS) into two parts, finding the data and perusing it. The data is up front but it may be just the front of the filing cabinet that is there. When you want to search through the filing cabinet or read a book, there is a whole new perusal mode. Not all of the pages of the book are up front.

Sancha:

Do you have a coherent set of software with aids for defining and designing interaction? Does the user work at the pixel level or in terms of areas? Do you have a syntactic approach to input?

Negroponte:

There is nothing particularly coherent about the software. People work at the level of putting ICs into sockets right up to writing PL/I or LISP programs. We have a terrific set of interrupt handlers and a certain amount of device independence. The user can work at any level from pixels to shapes. The central frame buffer is 7 years old and a lot of software has grown up around it. People have tools for their own use but there is no coherent set generally available. There are no 15 cabinets in the front office full of robust documentation. I wish there were.

Engelman:

Your system provides something analogous to a library. In a personal library system, I would expect to insert information. The decision not to have a keyboard on the system surprises me.

Negroponte:

You are correct. Input of full documents is clumsy. My normal way of working is at a typewriter. I even type postcards! I would be uncomfortable in a work environment with no keyboard. However, our sponsors are more interested in command and control than input. In this environment, the keyboard is not so important. I would not advocate the omission of a keyboard in different environments.

Kay:

I think what you are doing is very good in a research environment. We have some 3-button mice. Researchers tend to use the buttons for anything that seems reasonable at the time. This has made systems difficult to use. About two years ago, we decided to restrain ourselves to use just a single button so that we could define a decent user interface. You could learn a lot from not having a keyboard.

Bo:

You mentioned observing people purchase hand calculators. Have you observed the attitudes and changes in people who have used your system?

Negroponte:

No, we haven't done this yet. We hope to look at this when we get our new media room which is due to be completed today. However, people who used our Dataland a year ago can still remember where certain information is. For example, that a certain media file is in the bottom right hand corner. We do not intend to do formal evaluations until we have got all possible sensory inputs into our system. The right approach to interaction must be to make a giant leap not a small step. Hence we will look at things like eye tracking, body sensing, voice recognition. The really big pay-off comes from combinations of them.

Hayes:

I found your TV display attractive. Is that because it is novel? It raises the issue of device oriented development. Are there people now who prefer to do things your way and are prepared to abandon the ways they are used to? If so, what tasks fall into this category?

Negroponte:

We have not looked at this aspect in detail. Only one third of our laboratory is working on this project. Some of our equipment is outdated. Dataland uses five machines in parallel and it is difficult to get all five working simultaneously! Everyone else in the laboratory uses these machines. When we run Dataland they have to stop working! We hope to solve that with the new media room.

We have been asked to do some experiments with the media room to assess its suitability for use in teleconferencing. To that end we have built a smaller version. The experiment will be deemed a success if two people prefer to communicate through their media rooms rather than face to face. We are really working with a test environment to explore what the norm might be in 10 to 15 years time.

Bono:

Graphics standards activities have been concentrated at the vector level; your system seems far removed from this. Now that hardware prices are coming down, is the concept of machine and device independence obsolete because one will just go out and buy the appropriate hardware?

Negroponte:

The search for device independence may partly be a consequence of the bad devices available. The notion of independence might be replaced by more intelligent programs.

Baecker:

The CORE was aimed at cutting down on the investment involved in building the same system several times. You could not afford to reimplement large expensive systems. If you are making several thousand copies of the implementations, it does not matter.

van Dam:

Your work is brilliant and trend cutting but I object to your notion that vector graphics is dead and your dismissal of standards work. For a certain class of problems, you may be right. However, standards should not be concerned with promoting the future but with fixing the present. Most of today's displays, and for a little while to come, are calligraphic displays and will continue to be so. The SIGGRAPH group are now working on standards to encompass vector graphics. We must help those who are struggling now.

Negroponte:

I did not dismiss the idea of standards. A lot of my work is hampered by the lack of a video tape standard.

However, to think in terms of vectors is an archaic way of thinking about graphics. We only think about lines because we have hands and with pencil and hand movements we draw lines.

Seeing surface intersections as lines is the wrong approach and is counter productive. In the research community a lot of people are interested in geometrical modelling in terms of surfaces and solids. TV systems are not high technology anymore, they are cheaper than most calligraphic systems, with the Tektronix 4010 being a possible exception. Thus, I contend that vectors should be dismissed; they are not a reasonable approach to computer graphics in general.

Dunn:

I would like to comment on the dismissal of vectors and the CORE. This is an instance of the difficulty of making the change from one semantic category to another. We think of vectors because of the fact that we have machines that work in this way. As long as we talk about tool paths, we need to consider vectors.

Nick's world is a different semantic category. It is people's concern with managing data for themselves. It is not the world of getting machines to do what you want them to do to produce materials. Both have value and importance. Nick's work should not make us downgrade the mundane but important world of machines producing materials.

Standards for the representation of physical objects as shapes exist. Mechanical Engineering has taken the time to do that and it is just in publication form. ASME (American Society of Mechanical Engineers) has sponsored it in the USA. There is no such standard for portrayal. Nick's work will prompt this.