The session consisted of six parts:
Unfortunately, neither Lee Harrison nor John Whitney, scheduled to appear in the program, were able to attend.
The paper surveys key issues in the design and evaluation of current and future interactive computer animation systems.
The marriage of computer animation and interactive graphics I call interactive computer mediated animation. The phrase is cumbersome, but the words interactive and mediated are of critical importance.
In a medium, through a dialogue, we form and shape and represent content and meaning, the subject matter of the dialogue. An interactive language must mediateand enhance creative expression of that subject matter.
The matter of animation is first and foremost images, hence graphically-rich imagery should be mediated through graphic, pictorial interaction.
The matter of animation is picture change, picture dynamics, hence rich dimensions of dynamic variability should be mediated through interaction expressive in time and of time.
The matter of animation is rich sensory experience, hence dynamic intuition should be expressed through tangible, sensual interaction.
The matter of animation is symmetry and order and organization, hence precision and structure should be mediated through interaction expressed simply, clearly, cleanly.
The matter of animation consists of large collections of images, image sequences, changing images, hence the transaction time per interaction and the transaction time per mediated image should both be as small as possible.
We can therefore evaluate interactive animation systems along these five key dimensions - graphic, dynamic, sensual, structured, and immediate, by considering:
In the first half of the program, three panelists will describe, in words and with film, very different realizations of the interactive computer-mediated animation concept. I will present the GENESYS system and the picture-driven animation model. Lee Harrison III will describe the Computer Image family of animation systems. Prof. Charles Csuri will relate how he and a group of his art students have experienced and worked with a set of interactive animation programs. What are the essential features of each system? How does a user interact with it? How strong is the system along each of the five key dimensions? What does current use suggest about future evolution of animation systems?
Following the coffee break, Prof. Tim Standish, a computer scientist whose specialty is languages for interactive computing, Dr. Bert Sutherland, a computer scientist whose specialty is interactive graphics, and John Whitney, a film maker and pioneer in computer animation, will comment on the earlier presentations.
All six panelists, and I hope the audience as well, will then discuss future approaches towards enriching computer animation media and towards facilitating interaction between animator and computer. To stimulate this discussion, let me pose some questions, organized under three headings:
The choice of hardware is a major determinant of the graphic quality of images that can easily be produced with a system. The first part of the program features radically different solutions to the choice of hardware, some primarily analog and TV circuitry, the others primarily digital logic. These technologies should be married in future animation systems. How can this best be accomplished? Can we characterize what is done most naturally by digital, analog, or TV techniques? Are there qualities or potentialities of conventional animation media that cannot be achieved with any of these approaches? How about interfacing with animation stands (a la Kar Liang)? or, should we design totally new animation stands? (Whitney's analog computer-controlled stand, and Scanimate, are both new animation stands in the sense that they are machines for transforming arbitrary still graphics into animated image sequences.)
The graphic quality of most current computer animation (aside from simple straight line drawings) is sufficient to identify uniquely its source. Just as we can distinguish cel animation from cut paper animation from direct painting on film, we can immediately recognize a Computer Image or a GENESYS or a BEFLIX product. Need each computer animation system have a unique graphic quality, or can we build a single system that will simulate a family of diverse animation media? I envision a computer animator of the future sketching (defining Images) with a flexible electronic paintbrush or electric pen, with which he can generate incredible varieties of line, shading, and texture, and which he can tune to achieve distinctive styles of his own. What is required to make this vision a reality? - new display hardware, integration of line drawing with video display capability, better control of the quality (intensity, thickness, modulation) of line or video signal, software generation of shading or texture in two or three dimensions,...?
What about storage and playback? Storage could be digital, on video tape, or photographic (including holographic). Playback could be from video tape, or by general - or special-purpose display processors, digital, analog, or hybrid. What are the advantages and disadvantages of each, in terms of flexibility, speed, storage capacity, resolution, cost, equipment amortization through use for other functions,..? How does the choice of storage medium and playback mechanism interact with the basic design of the animation computer?
How do we augment the sensual dimension of interactive animation systems?
Currently, with GENESYS, the animator expresses his intuitive, bodily-sensed dynamics through a pen on a planar surface. Using a set of push-buttons, he can tap a rhythmical sequence directly into the computer. Animators at the Computer Image Corporation can jump and dance, while wearing a light harness, their dynamics being transmitted directly into the computer.
The future will likely see the use of pressure-sensors and grip-sensors in electronic paintbrushes, three-dimensional wands for drawing and sculpting directly in space, and a variety of new devices to facilitate the transmission of expressive body movements into expressive synthetic moving images.
The analogies between the synthesis of graphic variation through time, or animation, and the composition of music have long fascinated Alexeleff, McLaren, Whitney, and others. How can sound best be integrated with imagery, and how should this problem affect the design of future systems? Computer Image already directly drives figures into motion by music or recorded vocalizations. The computer animator of the future will use systems in which sound and visual components can be integrated from the very start of a movie, in which real-time playback will include both audio and video, in which graphs of picture change and of sound and music will be superimposed on the same time frame in a unified editing system, and in which music waveforms can be sketched along with graphic waveforms.
The past five years has seen flurried activity in the building of systems for building interactive graphics systems. Early efforts explored alternative data structures for representing synthetic line drawings, and embedded minimal sets of display calls in available high-level languages. More promising directions have emerged in later years. Richer languages for describing structured data and user interaction have been embedded in high-level languages. Comprehensive methodologies for the design of interactive graphic systems have been developed.
Everyone is understandably eager to attain a critical mass for film making. Consequently, the design of written languages for expressing animation has not flourished into new directions. The same paths are retread again and again, without noticeable trends towards standardization. Standardization now would be premature, but careful analysis and language comparison should be undertaken. The embedding of the interesting and pioneering BEFLIX in FORTRAN is valuable for it will now be more accessible. Yet FORTRAN is archaic, and future embeddings of any animation language should begin with a base that has more powerful control and data structures, and more flexible and forgiving I/O. Let's not further propagate the obsolescence of rigid, card-oriented, format.
Even less work has been directed towards the development of graphical interactive languages. GENESYS is based on a concrete philosophy of pictorial animation languages, consisting of both static images and real-time actions (user-driven animation), but its current command set is ad-hoc and the implementation shoddy. Other efforts, such as those of Ohio State, IBM, and Information Concepts, have begun. The Computer Image approach is totally different - what is the language of user actions at a Computer Image console?
What is needed, I believe, are some fresh ideas, a thorough rethinking and reformulation of fundamental concepts for describing images and animator actions, and perhaps a touch of formalization. The concepts of conversational languages, extensible languages and picture grammars provide some new directions that I think will be of assistance.
A conversational language is one in which each user command is directly interpreted and either executed immediately or incorporated as part of a new procedure under construction. Conversationality enables one to enlarge the capabilities of a system with the same language in which he uses it. The concepts of language extensibility and picture grammar are closely linked. Both provide mechanisms for building complex structures out of primitive elements, and for testing if candidate objects possess a particular kind of complex structure. A picture structure defined by a grammar is a class of pictures that satisfy certain constraints, hence a language for defining picture structures is a powerful picture construction and description language.
I propose that we strongly pursue the development of extensible, conversational animation languages. (APPL, in my dissertation, is an effort in this direction, but is as yet unimplemented. Other cuts at language definition need to be made.) Pictures and interaction primitives must be carefully chosen; powerful mechanisms for extension must allow the definition of complex picture types and classes of animator actions. Equivalences between written (linear) and graphical (2 or 3 dimensions plus time) representations need to be developed. (One valuable by-product of such equivalences will be discussed below under systems.) The resulting language will be usable both for building interactive animation systems and for expressing animation.
One particular software feature deserves special mention. In working with GENESYS over an extended period, Lynn Smith has generated and filed away large collections of static images and dynamic descriptions, despite the Stone-Age tools she had for doing this. The library, data management function of an animation system, the facilities for naming, renaming, referencing, aggregating, and disaggregating, must be carefully integrated into any new system design.
I should like to subject the above viewpoint towards animation system software to critique from the panel, and solicit, from the audience as well as the panel, alternative views. To evaluate an approach we ask such questions as - Will it facilitate the description and generation of rich image graphics, the provision of powerful means of dynamically varying an image, the ability hierarchically to build complex images out of simple images, complex movements and rhythms out of simple movements and rhythms, and the provision of flexible, responsive service to the user. The final test, of course, is its demonstrated use in the building of a more elegant interactive animation system.
Systems built to date have been highly specialized. We already noted the diversity of current computer image styles, and posed the problem of realizing such diverse graphics in a single system. Should we reverse the trend towards specialization in other aspects of system development? What interactive capabilities do systems for making physics, or math, or biology teaching films require? Would algorithmic, or procedural, capabilities aid artists using systems such as GENESYS? What kinds of capabilities? Can we and should we build systems strong along all five dimensions - graphic, dynamic, sensual, structured, and immediate? This will not be easy, for the demands appear to conflict - bodily interaction versus formal interaction, graphic and structural complexity versus the immediacy that is facilitated by simple graphics, dynamics, and organization. Can we afford systems strong along all five dimensions?
Insofar as images are simple, or picture change is trivial or random or predictable or easily calculable, demands upon the quality and quantity of interaction can be relaxed. I should like to see an animation system that provides an array of interactive capabilities along all five dimensions, yet allows the animator to use and pay for only that subset of capabilities needed for a given application. How may this goal be achieved? Can animation systems be time-shared? What implications does this goal have for system design? For selection of a hardware configuration? For software design?
How do we make existent or planned animation resources more accessible to those who cannot afford their own? The concept of computer networks seems relevant here. Even though adequate bandwidth along all five dimensions cannot be realized through today's networks, animation can be roughed in, and only in the final stages need the animator visit the host computer to complete the film. It is therefore essential that an animation data base be modifiable from a variety of subsystems and a variety of terminals, and that we better understand the relationship of written and graphical languages, so they can be used more interchangeably. The power of the display terminal is a particularly critical cost factor; hence flexibility in its choice is desirable. Is such a concept of resource sharing viable for computer animation? Alternative concepts are solicited.
In conclusion, I solicit comments about the interplay of animator, film producer or director, system builder, and system resources, software and hardware. I shall make three brief remarks: Computer animation systems will increasingly challenge animators and film-makers to explore new vistas of dynamic graphics. As the demands of animator and filmmaker become both more free and more rich, an animation system builder and the system resources will be increasingly challenged to improve the flexibility and responsiveness of the service. Finally, I predict that as a number of individuals use and extend a computer animation base system of the future, these will evolve, symbiotically, towards a number of personalized systems oriented towards and responding to the needs of those individuals, personal animation-machines, true extensions of man.
The mind boggles. But much is still to be done.
A definition of real-time might be attempted, however, certain difficulties arise since there are few good examples one can refer to. Technical explanations are relatively simple but to convey the experience of a new application is quite another matter. Recently at the 1970 Spring Joint Computer Conference the Evans and Sutherland Corporation demonstrated several real-time computer programs on their graphic console. One such program is a simulation of an aircraft taking off and landing on an aircraft carrier. The user can graphically fly his aircraft in a real-time environment communicating with the program through an electronic tablet and light pen. He can make changes in the elevation of the movement of the aircraft as the event occurs in time. He has the visual representation of these relationships on the oscilloscope as if it were a real physical event of flying the aircraft. To the observer it appears as if the user takes off in his aircraft from the aircraft carrier and the program shrinks the carrier's size and tilts the horizon line depending upon the user's maneuvers. What the observer does not usually appreciate are the utility routines and the perspective program and the clipping divider program which handle the transformations. A very dynamic and interactive mode of communication is offered in which one can test out maneuvers and experiment with alternatives without serious consequences.
In our real-time system at Ohio State the user can draw images on the CRT or he can use mathematically generated images for film animation. Not only do his images move in real-time but the images can be controlled as they move with the light pen and switches and buttons on the function keyboard. An artist can draw key frames for an animation sequence and a computer program fills in the additional drawings required to make the images move. To control speed one can adjust switches on the function keyboard which represent different rates of speed, and the moving images instantaneously speed up or slow down. The axis of rotation of a moving drawing can be adjusted through the function keys and light pen. The size of a moving drawing can be changed by pressing a function key to reduce the size of the drawing while another function key is used to increase its size. These function keys are programmed to behave like potentiometers. With the light pen one can draw the path they wish the drawing to follow, and as soon as the path is completed and one presses a function key, the drawing moves at motion picture speed. Several drawings can be displayed simultaneously on the CRT and the user has independent control over the path, the speed of movement about the path, and the size of each drawing. By pressing keys on the 2250 keyboard that represent x, y, z, directions, a software joystick can be used to control the path of 3D drawings. For instance, a 3D drawing representing a turtle can be walked in real-time throughout three dimensional space. The animator also has independent control over the movement of the turtle's head, the tail, and each of the four legs in real-time. Alphanumeric characters for words and sentences can be typed in through the 2250 keyboard, and as soon as the word or sentence is typed, the light pen can be used to sketch the path, and then the word will move about the path. Essentially in our real-time environment the user can fly, walk or model his drawings through two and three dimensional space.
At the risk of being redundant this concept of real-time might be described in the following way to artists and film makers. The process of creating images which we have just described involves an exciting new concept of time. Essentially the computer is only programmed so that it can respond to the decisions of the artist about the image and its movement. Then whatever the artist decided to do is transmitted almost instantaneously to the screen. This allows for a full interaction between the artist and his images. It occurs in what we call real-time, that is, time which is real because the moment of the artistic idea is also the moment of its materialization. This is a revolutionary idea in film-making, rather like editing the film before it is made. It makes the creative process a very spontaneous thing, for all the problems of execution are solved in advance. It is also by no means as mechanical as it might seem, for different artists will produce a variety of images and different relationships between images to portray their concept of reality.
In a real-time environment there are limitations as to the number of transformations that can be linked together before the real-time display is lost. Even with a sophisticated use of a compound transformation, and special approaches to the relationships between transformations and data structures, the computer's storage size and cycle speed, with its word length and instruction set, will impose additional constraints.
One of our concerns is not only one of developing new tools or options in graphics but how does the non-specialist use them in a natural way. Perhaps the question might be stated in the following manner. How can computer graphic tools involving chains or links of transformations be integrated for real-time animation so that the user can be involved in a spontaneous and interactive mode of communication to generate animated film sequences? Here one would hope to minimize the delays in response time and the computer programs organized in such a way that the user can give his full attention to the concepts and ideas he wishes to illustrate.
An example of the difficulties one can encounter with a rigid relationship between computer programs for real-time animation is as follows. The user has available to him the following programs that may represent only a small fraction of a graphics software package. (1) A 3D rotation program which handles hidden line removal {the data described as edges and intersecting planes) and which simultaneously handles straight line path zoom effects, as images are moving in real-time. (2) A 3D rotation program which handle hidden point removal (the logical structure of the data is a description of surfaces as points) and it has a software joystick and also the option to draw curves on the CRT with the light pen which can represent the zoom path of data. If the user wants the data described as edges and intersecting planes in program (1), and only the software joystick to control its movement from program (2), another new program will need to be written by a programmer to accommodate a new relationship between these routines. The solution to this problem may be a simple one but it is an inefficient approach and the storage space and speed of the computer system can soon be exhausted. Each user in the computer film animation environment has a different need and an approach is highly desirable which offers flexibility and new options and one which is not programmer dependent.
An alternative to the preceding approach is to treat the transformations as discrete units or modules. Conceptually, at one level they would appear to the user as items on a shelf or as a menu that he could string together as a chain of transformations. There would be a discrete unit for 3D to 2D or 2D to 3D data conversion, straight line zoom, drawn curve zoom, joystick, random walk, hidden line program, hidden surface program, scaling, 3D rotation, data conversion of hidden surface and hidden line, and many other units. At another and more important level transformations might be treated as building blocks by the user to formulate new transformations. A modular approach to computer programming is not new and it may appear simplistic but in real-time animation there are a number of special considerations. These considerations involve the relationship between data structures, transformations, and data management, and storage allocation.
The following example might illustrate a more complex problem for real-time animation and it is one which we have solved with software. Let us use as a data deck, a line drawing representation (x, y, z, coordinates) of a helicopter. The helicopter has two rotors, one for the main assembly section and another one on the tail section. We have a 3D rotation program which has as part of its structure, transformations to change the axis and the center of rotation of the helicopter. The problem is to make the helicopter move on the CRT and the computer must calculate and display pictures at a rate fast and smooth enough to appear to the user as a motion picture or video film. It should appear to move about an axis of rotation and the effect is as if the helicopter is moving in perspective about the path of an ellipse. While it is moving about this path each rotor is to move at a different speed in relationship to the helicopter itself. Then in addition, hidden lines are to be removed as the helicopter is changing its position in relationship to the user of the program. The user should be able to communicate to the program with a light pen and function keyboard to immediately change the speed of the helicopter's movement and the speed of each of the rotors. He can at any stage change the axis of rotation of the helicopter or even draw a new path.
In this example, let us give our attention to the most important problems and perhaps set aside a number of minor details about programming in a real-time environment. For instance, the transformations involved in a 3D rotation program are important but not the issue at the moment. Neither are the utility routines which handle the interrupts through the light pen or the function keyboard. There are a number of questions we must ask ourselves. (1) How do we define the data structure so that the program can recognize the helicopter as a whole and the individual rotors as data? (2) Once a transformation is developed to move the helicopter, how is another transformation introduced which will take into account that the helicopter is already moving at some rate of speed constantly changing its location, but now the main rotor must move at another rate of speed? (3) How do we link the transformation to the data structure and other transformations which will look at the main assembly that is moving and at the same time have another speed for the tail rotor? (4) While we are dealing with three motion variables, how can hidden lines be removed as the helicopter is changing its position? (5) How do these transformations look at the data structures to recognize the helicopter, main assembly and tail, main assembly rotor, and tail rotor? How does one set up the pointers to recognize data and establish the appropriate links between the transformations? At the same time, how does one allocate space for programs and data and move them in and out of the computer's core memory fast enough to provide a real-time display?
Algorithms to solve the hidden line problem in a batch processing environment have been in existence for several years. Most of them have been written in a higher level language such as FORTRAN. The calculation time for one view of an object may vary from 30 seconds to several minutes depending upon the complexity of the object and the efficiency of the algorithm - one would assume access to a high speed computer. If the programmer could reduce the calculation time from seven minutes to two minutes, he would naturally be enthusiastic about the improvement. There are differences in the algorithms used by researchers Bouknight, Warnock, and Loutrel. Although these are excellent solutions to the hidden line problem, they do not seem fast enough as solutions to produce a real-time hidden line display (the convex case), at least on 32K machines with medium speed core.
At Ohio State on our 1130/2250 system we have a real-time hidden line program which can handle the convex case (600 edges). In the process of doing an analysis of the hidden line problem, we discovered some procedures which reduced the number of tests that had to be made with the data. Another significant factor in our real-time hidden line display involves an extremely fast three dimensional rotation program. Our hidden line routine is also fast enough to permit several drawings to be displayed simultaneously moving at different rates of speed, and at the same time each drawing has a different zoom path and zoom rate as hidden lines are removed. The technique and concept used for this rotation program coupled with an improved hidden line algorithm accounts for our success with real-time display. Our results were produced on a 32K machine with a 3.6 microsecond cycle time but the same techniques and concepts used on a machine such as the PDP10 with its greater speed and better instruction set would produce far more complex figures moving in real-time. It would be reasonable to expect that many figures in the concave case could be displayed in real-time on a PDP10 machine interfaced to an Evans and Sutherland graphics display processor.
Extensions or projected plans of our work into the hidden line problem include not only refinements on the convex case but solutions to the concave case which will make it possible for us to do the following type of animation sequence. It would probably not be possible for us to achieve this example in real-time but the display might move at stop action speed.
Under computer program control we would like to graphically move throughout this landscape. As the user communicates his position relative to the landscape, the program provides a display of his position at that location. The house, the two cars, and the airplane are three dimensional models which have a representation of their interior structure as well as their exterior structure. The user first adjusts with the light pen and function keyboard, the path and the speed of the two cars, and the airplane. Then he establishes a location x, y, z, coordinate in the scene from which he can view the scene. Let us assume, that he starts at a position several hundred feet above the ground level. Within a matter of seconds or less the CRT will display a representation of the scene from his viewing position. Through the use of a software joystick he can move in rapid incremental steps down to the house with the program changing size relationships, the perspective, clipping parts of the landscape on the x, y, and z planes, and at the same time removing hidden lines. As he reaches the wall of the house and then passed through it, the program would display the interior of the house. He could spend some time looking at various features of the interior. Perhaps he might look outside through a window. As soon as he passed through the house the outside scene would now be displayed. He could shift his position and drive the car down the highway. It is expected that, once we could display and control information at that level, the computer program could have addition options of a multiple point light source to provide gray scale representations and also to indicate the translucent and opaque appearance of surfaces depending upon the user's viewing angle.
The 2250 display has only one intensity level. In order for us to do gray scale representations of solid objects, a point density subroutine was written to achieve the variations of gray. The spacing of points on a given face of a geometric solid determines the shade of gray. Data structures were defined to have a logical description of surfaces as points. A solution to the hidden surface problem is part of this program. At this stage of the project we are restricted to the convex case and the representation of data must accommodate this constraint. Although our gray scale objects cannot function as real-time animation, a relatively fast algorithm has been devised to paint the object fast enough to make it reasonable to make animated film. We simulate a Flying Spot Scanner through software and control the CRT beam as it is exposed to motion picture film. Two representations of the data are involved to make a real-time animation a useful technique for gray scale figures. One data structure is made which described the geometric solid as edges, while another one described it as surfaces with points and specific shades of gray. With the line drawing to represent data, judgments can be made about the size, position, motion, and speed of the objects. Adjustments can easily be made in the real-time environment with those parameters. Such important decisions would be the precise path, size, speed, and time duration about the path. Then the values representing those parameters are used on the second data deck to make the final film sequence. The individual frames are painted in shades of gray fast enough to make stop action filming a feasible approach.
Our algorithm to write, points on the CRT representing geometric solids (convex) and simultaneously to do a double rotation while removing hidden surfaces, can display 225,000 points in less than 12 seconds. Once the first view is painted there is less than one second delay between the first view and the second view. It seems reasonable for us to make short sequences of gray scale motion picture of figures which can be represented by 75,000 - 100,000 points. Here one would have a new frame every 5 or 6 seconds.
Interactive graphic displays for real-time film animation and model simulation can be a dynamic and productive process for the user. The computer programming requirements to support and extend the capabilities of such a system is a sophisticated problem which involves basic research in computer graphics. The frontiers of research into the relationship between man and machine, as it is applied to interactive graphic display systems, indicates that additional modes of communication might be brought to bear upon this problem. New relationships between man and machine could involve a more total sensory-electronic communication system. They could influence his definitions of problems and how these problems could be solved. All of these modes of communication include special purpose electronic hardware and the appropriate computer programs to implement them. (1) Visual communication systems which involve the user's eye and body movements offer some interesting possibilities. Ivan Sutherland's video helmet for 3D perception of computer generated images in a real-time system is one example. Another possibility makes use of a special optical headgear electronically connected to the computer which scans the user's eye movements. At The Ohio State University a major research project (1966-1969) did research in eye movement studies on subjects as they watched commercial television programs. The subjects wore a head mounted electronically operated optical system which tracked their eye movements and these eye movements were recorded on video tape and then analysed. Manfred Knemeyer, my research associate, designed the electronic system for the eye movement studies and he also wrote the computer programs which did the analysis of their data. It seems reasonable to us to electronically wire the human so that he could communicate to the computer with his eye movements, information about the path, the speed, and even the size of the drawings displayed upon the CRT screen. Additional visual commands or instructions could be devised as a visual code to communicate to the computer program. (2) Computers that can recognize human speech are not as yet practical but there is evidence to suggest such a mode of communication may soon be a reality. This method of communication may initially be confined to simple commands, to control a graphic display but the potential for a more complex relationship between the human and the machine is an exciting prospect. (3) A tactile-kinesthetic mode of communication with a computer may seem somewhat bizarre but there is research already moving in that direction at Bell Telephone Laboratories, and at The University of North Carolina. The potential for form generation, form manipulation, and form perception is a promising area to investigate.
In the man and machine communication problem as it applies to visual representation by computer different modes of communication need to be examined to improve the quality of this relationship. Concepts about computer equipment design and the need for better communication systems are already emerging in the technology. With the advent of such a communication system (visual, tactile-kinesthetic, and verbal) basic questions of language description about a problem and problem definition itself are still difficult ones to resolve. In fact, additional modes of communication might further complicate these questions, at least, if one continues to use conventional approaches to data structure definition and linkages to transformations, data management, and the structure of transformations. However, these modes of communication and the ideas they will stimulate may well provide the conceptual breakthroughs that can make the computer process a more dynamic one.
In many ways the computer simulation of pertinent information from the real world is still an art and not a science. The effectiveness of any simulation rests on the human simulator's ability to abstract only those factors that affect the system or process he wishes to duplicate. Our research in computer graphics for real-time film animation, especially our success with the helicopter problem which involved a compound transformation, suggests that we may be able to develop additional computer graphic tools to deal with simulation models. It may be an ambitious goal but it is certainly a highly desirable goal, to expect that computer technology may be able to provide the appropriate tools to allow the scientist, the educator, and the artist to simulate their models of reality.
One of the reasons that computer animation excites me is because I feel it has the potential for becoming an important tool of the intellect. As science has matured, observation and deduction have grown more indirect. Nowadays one seldom uses ones primary senses, such as seeing, hearing or feeling, to make direct observations. Rather, instruments such as microscopes, spectrophotometers, oscilloscopes, bubble chambers and radio-telescopes apprehend phenomena that lie outside the range of human perception and translate them into a time scale and a sensory medium appropriate for human consumption. Theories have become richer and more complex as they attempt to describe symbolically the dynamics of complex systems. In fact, the inherent complexity of systems may be the single most important barrier to advanced scientific understanding in the latter half of this century. This is probably true in the life sciences, in computer science and in the social sciences, and may even be true of many industrial situations as the big northeast power blackout of a few years ago seems to illustrate. We must find ways of dealing with complexity and of rendering the behavior of complex systems intelligible in order that complex systems be made to serve man rather than man becoming the servant of the complex systems he creates. To what degree is man the master and to what degree is he the servant of his transportation systems, economic systems, industrial systems and the ecological system of the entire planet?
It is in helping to render the behavior-of complex systems intelligible that computer assisted animation has a largely unexplored potential. The computer can generate and present many species of visual images in many patterns and variations. But perhaps more important, it can translate sterile data into intuitively appealing pictures, it can change time scales and it can suppress details that impede understanding. Thus, like the oscilloscope, it becomes an instrument which transforms phenomena outside the range of human apprehension, into phenomena inside the range of human comprehension. But unlike the oscilloscope, the computer has a tantalizing potential for dealing with complex systems through its ability to be dynamically selective in what it looks at and to supply elegant translations of data unfit for human consumption into displays that are both palatable and revealing.
To give just one small example, suppose you are a biologist studying the effects of thermal and chemical pollution on a given local environment. You devise a set of 50 differential equations which express the population size of each species in the food chain in terms of the population size of other species in the food chain and in terms of some independent variables for thermal and chemical factors. Now how do you access the behavioral implications of your model under important sample conditions? You can try to do it by hand calculations, which is certainly tedious, and probably error-prone. You can write a computer program that prints reams of numbers for you to stare at, which is better. But if you really want to get a feel for the time dynamics of your model, for the rates of growth and decay, and for critical points at which the system will shift toward new equilibria, I know of no better idea than to animate it. Even a simple animation will do. For example, portray each species as a labelled circle whose area is proportional to the species population size. Drive the size of each circle off the differential equation for the population size of its associated species. Then choose an appropriate time scale, some initial conditions, set the beast in motion, and watch what happens. Your eye will be drawn to the olace where action is happening and unusual or surprising implications of the model will be likely to draw your attention. Now vary the initial conditions and set the system in motion again and watch what happens. If your model is wrong you will be able to find out how it is wrong so that you can tune it. If your model is right and agrees with important sample data, you may even begin to understand ecology.
There are many other situations where watching the behavior of something dynamic is maybe a better way to aid understanding than is reading a static description. Not the least of these is in understanding how computer software works. Why not animate operating systems, assemblers, compilers and loaders to see how they work and to get a feeling for where they are spending their time?
So, to summarize, while computer animation has many well known uses in art, in education, in entertainment and elsewhere, one of the most exciting largely unexplored applications of it is as a tool to help render intelligible the behavior of complex systems and the behavioral implications of complex descriptions.
I now want to explore some of the purely technical computing issues connected with animation.
Animation involves the use of the dynamics of picture change through time to express behavior. Thus, in computer-assisted animation one is always recomputing pictures. If the animation is non-interactive and is played back off-line from a recording medium such as a film or a video tape, then the response time of the computer in generating new pictures is not an issue, especially since the rates of generation and playback can be different. But if the animation is either interactive or is non-interactive but still generated on-line. Then the computer should generate picture updates at a rate that satisfies the human appetite for information. Thus the response time of the computer in supplying updates becomes an issue. Few batch or remote job entry systems have turnaround times fast enough to supply picture updates for the interactive consumption. Thus, ones attention turns to the systems that provide fast response times, these being single user systems or time shared systems with conversational or interactive facilities. The essence of these systems is simply this: Response time is proportional to the demand for computation and, in particular, there is rapid response to trivial requests.
Hence, if you want to couple your graphic animation system to some background programming language L, in order to do the computations relevant to animating some particular subject area, a rule for choosing L could be: If you are animating offline L can be any language, but if you are animating on-line L must be such that programs written in it can interact with your animation graphics program and can provide rapid enough response time to satisfy your demand for picture update rates.
One has to question closely the need for interactive computer assisted animation. Must animation be conversational? Obviously, in the movies it isn't, and movies are good things. I feel that there are two important cases in which computer animation should be conversational.
The first is that conversational systems which provide rapid response to trivial requests permit one to evolve and modify a program at a very rapid pace. Many of the changes you make in writing and debugging a program are trivial. One should not have to wait four hours and be charged for recompiling ones program just to change one semicolon. For example, if you make a mistake in animating a man walking, and his feet slide over the background because he walks at the wrong rate, maybe all you need to change is one rate parameter. Conversational systems permit you to observe the misbehavior, to request a trivial change and have it rapidly satisfied, and immediately to set the program in motion again to check whether the new behavior is correct. One should not have to go through the whole cycle of updating a source file, compiling, loading, initializing and then setting up the proper routine circumstances in order to sense the behavioral implications of trivial change. Conversational programming languages are especially valuable in this context. They should be chosen for convenient symbiotic relations with an interactive graphic package.
The second case in which interactive computer animation systems are important is to satisfy applications where man must be in the loop to supply decisions about how the animation is to proceed. For example, in using animation as a tool to understand the workings of a complex dynamic system - man must be selective in deciding what to look at in the fire-hose of information and detail that the computer is capable of generating. That is man must set the time scale, the focus and the level of detail. Since man has changing interests that the machine can't predict, he must be in the loop to decide what to look at, to compress the uninteresting, to expand the interesting and to jump in and out of the hierarchy of detail depending on whether he is excited or bored. Conversational graphics systems and conversational programming languages are structured to permit just this sort of thing. They make special provision for writing programs that converse with users, that is, programs that request that information be supplied by the user at certain points, and which use this information to determine their subsequent course of action. Thus, if interaction is important you should choose languages that permit one to express interactive algorithms with ease. The conversational programming languages thus become natural candidates for marriage with interactive computer animation packages.
Extensible languages directly concern the representational power of the medium in which animation-related computations get done. The principal contribution of extensible languages at the moment is in the realm of data definitions, but projected developments indicate a contribution in the area of control structures as well. The prototype extensible languages are in the cycle of construction and experimentation so it will be a while before they are available as commercial products. Here, as elsewhere, the lead time from conception to smooth application can be long. For example, transistors took ten years, and APL, which is a language of great beauty, took seven years.
Extensible languages help us in being flexible with respect to data, and data flexibility is important because it is related to the capacity of a representation to sustain variation.
In animation, we have to specify which parts of a picture are variable and how they are to vary. Do they change position, color, texture, size, orientation, shape or what? Do they move, and if so how continuously or discretely, and at what rate? So the decision about what shall be variable and how it shall vary in a picture implies a need for a data structure flexible enough to represent the set of possible variations. In reference to extensible languages, what we are really speaking about is a difference in scale and convenience in the effort required to set up new data structures. i.e. does it take six months of arduous programming in machine language to get the data representations you want, or does it take a few data definitions and three days of polishing to get going? And what happens if you want to change your mind. These are the issues that extensible languages address, and my own experience with a prototype language we have running in our laboratory indicates that a pleasant improvement in programming convenience is in store for those who graduate to extensible languages, particularly the conversational ones.
For example, six years ago three colleagues and I spent eight man years implementing a compiler for Formula Algol. This year one of my students was able to do all of the formula manipulation we did in Formula Algol one week in a prototype conversational extensible language he implemented as a student project in six months.
So if you are going to build a non-interactive animation package consider marrying it to either an extensible language or a low level list processing language like Knowlton's L6, and if you are going to build an interactive animation package, consider marrying it to a conversational extensible language, because having representational flexibility pays off.
Among other possibilities scan conversation, video discs and off-line refreshing give us a technique for flicker-free sharing of a computer resource that generates different pictures for each of many users. The question then boils down to a matter of dynamics - i.e. the rate at which the picture or its parts is being recomputed. This depends on the response-time of the computer for various amounts of computation. Additionally,, in the case or networking, band width and operating system service times may enter in. One should probably not count on getting updates fast enough to fashion continuous motion for sustained time intervals, but discrete picture changes every so often appear quite feasible. It is still possible to have an animation style without continuous motion of images, but the motion-related sensual qualities of the medium probably have to be sacrificed.