This paper is based on a talk given at a Symposium on "The Human Use of Computing Machines" held at the Bell Telephone Laboratories, Murray Hill, NJ, on June 20-21, 1966, where the author was a resident visitor during 1965.
The use of films to foster the education of students has a long history. Within the past decade, as part of the curriculum revisions being made throughout the educational establishment, films have come to occupy a place of growing importance.
Inspired by the successful effort of the Physical Science Study Committee of Educational Services, Incorporated (ESI) to develop a series of 60 films as an integral part of its high school physics course, other groups have undertaken similar programs in various subject areas. Many of these programs are being carried forward with the help of ESI, a non-profit organization in Newton, Massachusetts, that provides administrative services, professional film production facilities, and technical staff.
Support for these efforts is for the most part coming from the National Science Foundation. Some notion of the magnitude of these efforts may be gained by noting that the production of a scientific film may cost about $2,000 per minute of finished product. If, to the film programs in elementary science, social studies, college physics, semiconductor electronics, fluid mechanics, meteorology, and electrical engineering - all currently underway at ESI - one also adds the films in mathematics, physics, and other disciplines being done elsewhere, the total effort is impressive. Apparently, someone must believe that films are a valuable aid to education and worthy of support.
The justification for much of this development is that films provide an unusually effective way to enrich the perceptual side of subject matter for the student. Experimental demonstrations, which in times past provided the percepts to go with the concepts, have faded or vanished from most modern classrooms. Instead, the lecture is likely to be constructed of words, gestures, chalk, sketches, and mathematics with no supporting laboratory work. The emphasis is increasingly on the conceptual side of the subject, often expressed in mathematical symbolism. As a result, students all too often become proficient in the formalities of analysis without having any idea as to what the analysis is really about.
A major objective of these film programs has been to put demonstration experiments on film, taking full advantage of available techniques such as high-speed and time-lapse photography. With good camera work, a film can leave images even more vivid and meaningful than the experiment seen live in the classroom. Although costly to produce initially, the apparatus can be jettisoned once the film has been completed. The experiment, stored in a can of film, never fails to work promptly without difficulty, and may be easily replicated for use in any number of classrooms with little additional cost.
Although the major educational use of films has been to enhance the perception of reality, by showing physical apparatus, phenomena, and experiments, attempts have also been made to produce films that will convey abstract concepts. Here, films have been less successful. Geometrical notions, such as the slope of a curve at a point, the properties of the conic sections, projections, etc., provide photogenic material. Indeed, the Committee on Educational Media, of the Mathematical Association of America, is producing such films. However, as one shifts attention from the direct observation of the physical world toward the domain of concepts, the use of symbols (linguistic or mathematical) is unavoidable. And so it is that one may find it best to film an outstanding lecturer, such as Richard Feynman, discussing the important concepts of physics.
The very fidelity of films, which makes them so well suited for communicating specific properties of physical things, may interfere with the presentation of abstract concepts. An astoundingly large part of what we think of as reality is in fact not directly observable. Abstract concepts are made concrete by representing them by symbols which may be grasped, manipulated, and communicated as if they were physical objects. With familiarity and repeated success in checking conclusions reached by manipulating these symbols, we cloak them also with the manifestations of reality.
Information may be communicated either pictorially or symbolically, but with noteworthy differences. A picture of a dog, or the word dog, will evoke a response similar to that elicited by actually seeing a dog. However, the picture refers to a dog of a particular size, shape and texture; the word refers to any dog. The word is clearly more abstract and general in its range of applicability.
On the other hand, a symbol, such as the word dog, has signifying power only by convention and mutual agreement, whereas pictures derive their meaning from projection and replication. That is, the picture of a dog is perceived to stand for a dog by all people who have seen one, but the word for dog is different in most languages of the world. This is one reason why a picture is said to be worth a thousand words.
Thus, picture-type representations have the advantages of concreteness and fidelity. But the greater their fidelity, the lesser their capability to refer to abstract relations. In contrast, symbols are much less environment-bound and have greater flexibility and capacity. They can denote both abstract ideas as well as concrete objects. But the meaning of symbols must first be learned through arduous education.
In his theory of pictorial perception, J J Gibson [1] suggests that all representations can be placed on a continuum with linguistic symbols at one extreme and pictures at the other. Examples of mixed representations are diagrams and charts that represent objects and their relations. It is because words and pictures complement each other in this way that sound movies are generally more effective than either words or pictures alone [2].
To appreciate some of these problems, suppose that you wished to make a film showing the abstract concept of resonance. This is a property exhibited by many things around us, and is therefore of considerable generality and significance. To appreciate how dependent we are on words and symbols, try to develop a film that uses only pictures - no words or mathematical equations allowed! By presenting views of various mechanical systems subjected to periodic excitation, it should be possible to show that the amplitude of response for some excitation frequencies is much larger than at other frequencies. Hopefully, the student might himself see the underlying similarity and abstract this common property. Unfortunately, since he has seen only mechanical vibrations, he would have no way of generalizing to electrical, acoustc, economic, and other systems that also exhibit resonance.
To show resonance in an electrical system, the problem of displaying the values of the voltages and currents arises. Of course, one may insert meters in the circuit, thus converting these values to pointer deflections. The mechanical movement of the meter pointer may then be used to exhibit that resonance occurs at some frequency. However, the picture here is less persuasive than for the mechanical system, because the essential phenomenon is observed less directly. Furthermore, little has been done to explain the phenomenon of resonance. Even the function of the meter is hard to demonstrate pictorially. How do you show pictorially that the ammeter measures current and the voltmeter measures voltage, when the concepts of current and voltage are themselves often not clearly understood by the student? Perhaps at this point you might resort to animation with an overlay of big blue electrons moving back and forth along the wires and crowding onto the plates of the capacitor. But, as you introduce these artifices, you are cluttering the picture with increasing detail which, although pertinent to the electrical system, is in fact quite irrelevant to the concept of resonance itself.
The main point emphasized by this discussion is that the very specificity of physical things interferes with their use to communicate abstract concepts such as resonance. In contrast, mathematical symbolism seems ideally suited. Let the force acting upon a system be denoted by the symbol f. Suppose then that there is some other observable in the system, denoted by x, for which the following equation holds:
ẍ + ζẋ + ω02 x = f
Then the property of resonance is exhibited by finding the particular frequency for which the ẍ and ω02x terms are of equal magnitude and opposite sign. The fact that x may be an electrical charge in describing one physical system, and a mechanical displacement in describing another is irrelevant. What is important is the relation between f, ẋ, x, and ẍ, and the fact that certain terms cancel when x varies periodically with a particular frequency. This is a concept that is made more evident by study of the equation than of the physical system.
The difficulty here, of course, is that mathematical symbols are highly conventional. Until the student has learned their significance by studying calculus the symbols x, ẋ, and ẍ are meaningless to him. Traditional mathematical symbols lack the self-descriptive properties of a picture. It would be desirable to develop a symbolism that is so natural that the significance of the abstract relations portrayed would be evident to someone without formal mathematical education.
This article discusses a technique for conveying abstract ideas without using the traditional words and mathematical symbols. I shall call these representations computer pantomimes because the dynamical and pictorial symbols used in the movie are generated by the computer. Unlike ordinary mathematical symbols that are static and must be arranged in accordance with a rigid format, our computer-generated symbols are dynamic; they move about and change their shape with the passage of time so as to emphasize and highlight particular relations or properties.
Of course, similar happenings are already available in traditional animation sequences for movies. However, the computer offers unexplored possibilities of great promise because it can produce these visual representations of scientific and quantitative abstractions that are far beyond the capabilitv of traditional animation methods [3].
By using a computer to generate the signals for controlling a precision, television-like tube, it is possible to plot points and draw lines at speeds many orders of magnitude faster than can a human draftsman. These pictures may then be photographed by an automatic microfilm camera, also under the control of the computer. Current speeds of microfilm recorders, such as Stromberg-Carlson's SC4020, lie in the range of 10,000 to 100,000 points, lines, or characters per second. It is therefore possible to produce complete line drawings at the rate of several complete frames per second, or to produce half-tone images by typing a mosaic of closely spaced characters in only a few seconds. If the successive pictures are changed slightly from one frame to the next, the sequence forms a movie that may be viewed using a standard projector.
The quantitative results of research studies that are commonly plotted as graphs or charts may now be displayed dynamically in pictorial form so as to take advantage of the ability of the human eye to perceive changes and relations that would be impossible to find in pages of printed numbers. One of the earliest and best examples of this is the research film made in 1963 by E. E. Zajac of the Bell Telephone Laboratories. [4] This film shows the effects of the gyro stabilizers on the motion of a communication satellite. A sequence of many frames from that film, all superimposed in Fig. 1, shows the satellite tumbling as it orbits around the earth (which slowly revolves in the actual film).
In making this movie, the computer simulated the motion of the satellite and computed the necessary commands to the microfilm plotter so that it would draw the desired perspective view of the satellite's motion for different designs of the stabilizer.
The effect of motion in enhancing the signifcance of line drawings is quite striking. Complicated figures that seem a mere jumble of lines when static, suddenly become three-dimensional cage-like structures in space when made to rotate in animation. Further realism can be achieved by programming the computer to produce two stereographic views of the same data, thus making possible a true three-dimensional display of static objects as well. [5]
Special programming languages are being developed for instructing the computer how to draw the desired images. A major step was the development by K C Knowlton of a movie language called BEFLIX. [6] Using this special, easily learned language, the programmer may imagine himself to be draftsman, cameraman, and editor by using simple commands for drawing, copying, painting, panning, zooming in or out, dissolving, fading, etc. In addition, light-pens and other devices are being developed for simplifying the task of giving the computer its instructions. In this way, the great power of the computer to produce graphical pictures may be controlled in an easy and convenient way with only a minimum of computer expertise.
With the development of the computer and an appreciation of its uses, we are being forced to distinguish more clearly between the logical and empirical content of our knowledge. In the past, we have endeavored to construct scientific theories that insofar as possible would be in accord with the physical world. The truly revolutionary aspect of the computer is that for the first time in history, man has constructed a physical entity that functions insofar as possible in accord with his theory.
It is now possible to examine the logical consequences of a given set of assumptions without turning to analogous physical systems which, at best, are likely to be imperfect realizations of the assumptions. Instead, the computer itself becomes a pure, physical system in which a prescribed sequence of operations is performed precisely as specified, without contamination by the uncertainties and irrelevancies always encountered in traditional physical experiments.
When a computer is used to simulate a physical system, it is commonplace to regard the computer as an imperfect analog of the real thing. However, it must be emphasized that from the point of view of theory, the relationship is the other way around: it is the physical system, with its never-completely-described complexity and uncertainty, that is the imperfect analog of the pure system realized on the computer. This pure system is, in my opinion, destined to become the major tool for developing abstract concepts and related theories in science. Its emergence in this role has been delayed because of the abvsmally poor communication channels that have existed between the computer and the human. With the new graphical displays, light-pen inputs, and other presentations matched to the human senses (Maurice Constant has described light-gloves with which one could tactually mold computersimulated three-dimensional objects), the man-made world of the computer will become increasingly real and will provide a superb instrument for educational purposes.
Just as a symbol has greater generality than a particular thing denoted by that symbol, so also is this pure system more general than the world around us. This fact is made evident in F. W. Sinden's beautiful movie on Force, Mass, and Motion [7] in which he examines how the planets would move if the central force law varied with distance from the center of mass as l/r3 instead of l/r2. Using a similar technique, it would be easy to study the motion of the planets in a hypothetical universe having four spatial dimensions, instead of the familiar three. (Here, one might conjccture that the stable, natural, central-force law would be I/r", and that inverse, fourth-power, and higher-power laws would yield unstable configurations. In such a four-dimensional universe, do Kepler's laws still apply?) The motion of the planets in this four-dimensional space can be viewed, as A. M. Noll has shown [8], by projecting the space onto a three-dimensional subspace that may be viewed stereographically. The appropriate display of information about the goings-on in this pure system will generate new problems of considerable interest and importance.
In his remarkable book, Art and Illusion, Gombrich [9] examines in depth the notion that all art (and visual communication, generally) involves illusion. That which is accepted as a realistic representation always incorporates unrealistic conventions. These conventions have become so familiar to the viewer that he is oblivious of them. In our own culture, where photographic representations have been so highly developed, most people would regard a photographic portrait of a human head in profile as being more realistic than a Picasso drawing showing both eyes in a profile view. Yet, if you were to show the two representations to an aborigine, he might select the Picasso as the more realistic because the photograph has only one eye, whereas most people have two eyes (as realistically represented by Picasso). The conventions of perspective, which seem so natural to us, are quite unnatural to the savage.
The development of new representational schemes is an exceedingly slow and difficult business that goes on with fits and starts in all areas of human endeavor. We are often quite unconscious of the conventions that are being followed. In comic strips, for instance, the balloons that enclose the spoken words are drawn differently when a character talks to himself rather than to others. Yet, if someone who has just read a comic strip is asked to draw one of the frames from memory, he is likely to omit the balloons entirely and recollect the scene as though the characters had indeed been talking to each other and themselves.
To take an example of a more technical nature, lines of force, which are used to represent vector fields, are useful and meaningful because they seem to suggest other properties of the field such as divergence, curl, and the like. Yet, we should remember that one does not see force directly; rather, we see the effect of a force on the motion of a particle. The force must be inferred from the motion of the particle. And so it goes with most of the manifestations of reality with which we deal in science and engineering.
The ability of new computers to draw symbolic forms of unlimited variety suggests that here is a fruitful area for intensive research and study. As we have already seen, photographic likeness may be undesirable when portraying information that will lead to concept formation. It is important then, to delete all that is irrelevant or unessential, and to emphasize those features that are crucial to the concept. In this, technical people may be too literal-minded and inclined to underestimate the merits of unrealistic representations for conveying abstract concepts. Just as the artist distorts reality to bring out some quality or relationship that would otherwise be obscured, so also may distortion be used to advantage by technical people in making representational schemes for abstractions.
The essential idea that I am trying to convey is illustrated by the computer-generated film Harmonic Phasors that Professor D. Weiner of Syracuse University and I made last fall [10]. This film attempts to develop a few major concepts relating to the description of periodic signals by the sum of several sinusoidally varying components. This film is unusual in that there is a complete absence of traditional mathematical symbols and words in the body of the film. In an earlier version of the movie, we had used titles to explain the various scenes, but the fraction of the total film devoted to titles seemed altogether too large. Subsequently, we reworked the animation in an attempt to present several concepts using only pictorial images that would have some meaning to anyone who views them, regardless of his mathematical background. We soon found that by deliberately forcing ourselves not to rely on words and traditional symbols (even though a sound-track and titles might later be added to further strengthen the impact of the film), we were led to develop the full pictorial possibilities of the film.
Our experience illustrates an important difference between animation using traditional techniques and computer-graphic methods. Traditional animation involves an enormous number of man hours of labor by many skilled draftsmen and artists who reinterpret the ideas of the author of the films. Because this artwork is so costly, it is rarely feasible to explore alternative presentations or to alter drastically a lengthy animated sequence, once completed. Furthermore, because so many people are involved in this chain, the animation must follow rather strictly arranged logical lines. As a result, elegant and imaginative ideas become diluted and distorted as they are transferred from person to person. Computer animation, in contrast, enables the creator to come in direct contact with the display medium. Easily learned movie languages (such as the BEFLIX language described earlier) make it possible for the author to write his own instructions to the computer. And, because of the low cost of computer animation, he may try alternative presentations and choose those that are more effective. Of course, there is the danger that the artist and draftsman may be replaced by the computer programmer, but this is increasingly unlikely as more and more people become better acquainted with the simple languages developed for this purpose.
Our experience aiso emphasizes the importance of an appropriate problem-oriented language. To make the Harmonic Phasors film, I first developed a special language called PMACRO which was to be used for making films that were concerned with exponentially varying signals, modulation theory, and other topics of interest in electrical engineering. Although the development of this language took me about six months, my students and other users have been able to use it and make successful movies almost immediately, even without prior programming experience.
We are currently working on computer pantomimes for several different topics. Professor Weiner is making a film that explains why a simple resonant circuit responds in such a strange way when the frequency of the sinusoidal input signal is suddenly shifted. Frequency-modulation problems of this sort are very messy to describe by traditional methods of analysis. Yet, when represented in appropriate pictorial form, and shown as a computer pantomime, we believe the essential features will be made quite clear.
How effective are computer pantomimes? In an effort to obtain objective data concerning this question, Professor Lawrence Grayson of The Johns Hopkins University has made, with the help of ESI, a conventional film to demonstrate that the currents and voltages of a two-port electric circuit can be represented as vectors in a two-dimensional current space and a two-dimensional voltage space, respectively. Then the electric circuit is identified with the tensor that relates the voltage vector to the current vector. The apparatus used in this demonstration includes two electromechanical plotters for creating the images of the current and voltage vector spaces, and meters for showing the values of the currents and voltages in the circuit. Figures 2, 3, and 4 show selected scenes from this film.
A computer-pantomime version of this same demonstration experiment is now being prepared. The programming is being done by an undergraduate, Eugene Stull, at The Johns Hopkins University. In Fig. 5 are shown a few of the selected frames from the film's introductory sequence which is intended to define the voltage source and the voltmeter and ammeters. The meters are denoted by a circle and a line. In the case of a voltmeter, the line is placed beside the voltage source, whereas for an ammeter the line cuts the circuit through which the current is to be measured. The value of the current or voltage is represented by the displacement of the little circle from the line; the greater the value of the signal, the greater the displacement. Thus, in Frame 2 (of Fig. 5), the source voltage is zero, and both currents are zero. In Frame 3, the voltage has a positive value in the upward direction. In subsequent frames, the voltage source moves toward the resistor and, at the instant contact is made, a current passes through the circuit as indicated by the deflections of the two current meters in Frame 6. Subsequent scenes emphasize that the current is interrupted when the source is disconnected from the resistor and that the direction and magnitude of the currents are linearly dependent on the source voltage.
In later scenes of this movie, the circuit is enlarged to contain two voltage sources and three resistors. Kirchhoff's current law is demonstrated by showing that the sum of the currents in the vertical branches of the circuit total zero (this is done by geometrically adding the meter displacements end-to-end). Then, the meters are replicated and moved downward to form the horizontal and vertical components of a two-dimensional vector space-first for the voltages and then for the currents. All of this is done without using words and conventional mathematical symbols.
To compare the relative effectiveness of these two presentations, Dr. Doris Entwisle (an educational psychologist in the School of Engineering Science at The Johns Hopkins University) will join Dr. Grayson and myself in planning and performing an educational experiment in which both films will be shown to a substantial number of incoming freshman next fall. We are curious to see, among other things, if liberal arts students will show a significantly different response to the films than students going into science and engineering.The possibility of using the computer to generate pictures and graphical symbols opens new possibilities for clarifying and demonstrating relationships and concepts that are difficult to convey with traditional physical apparatus.
For removing the tyrannical crutch of words and for forcing one to rethink the meaning and implication of concepts in visual terms, computer pantomimicry is an excellent exercise - and fun too!
[1] Gibson, J J, A Theory of Pictorial Perception, A-N Commun Rev, 1, 1954, 3-23.
[2] May, M A, Word-Picture Relationships in Audio-Visual Presentations-A Working Paper, U.S. Department of Health, Education, and Welfare, Office of Education, Contract No. OE-5- 16-006, July 20, 1965.
[3] Knowlton, K C, Computer-Produced Movies, Science, 150, Nov 26, 1965, 1116-1120.
[4] Zajac E E, Computer-Made Perspective Movies as a Scientific and Communication Tool, CACM, 7, Mar. 1964, 169-170.
[5] Noll, A M, Stereographic Projections by Digital Computer, Computers and Automation, 14, May 1965, 32-34.
[6] Knowlton, K C, A Computer Technique for the Production of Animated Movies, AFIPS Conference Proceedings, 25, 1964, 67-87.
[7] Sindon, F W, Force, Mass, and Motion, 10-minute, 16-mm black and white sound film, available on loan from Technical Information Libraries, Bell Telephone Laboratories, Murray Hill, N.J.
[8] Noll, A M Computer-Generated Three-Dimensional Movies, Computers and Automation, 14, Nov. 1965, 1-4.
[9] Gombrich, E H J, Art and Illusion, Bollingen Series XXXV, 5, Pantheon Books, 1961.
[10] Huggins, W H, Weiner, D, Harmonic Phasors, 7-minute, 16-mm black and white silent film, available on loan from Educational Services, Inc., Newton, Mass.