GENESYS: Interactive Computer-Mediated Animation

Ronald Michael Baecker

Illustrations by Lynn D Smith

Additional Statements by Lynn D Smith and Eric Martin

1974

Focal Press

Ronald Baecker is currently Visiting Assistant Professor of Computer Science at the University of Toronto, Ontario, Canada.

Lynn Smith is a free-lance animator, artist, and teacher of Cambridge, Massachusetts USA.

Eric Martin is with the Department of Visual and Environmental Studies, Harvard University, Cambridge, Massachusetts USA.

Work reported herein was carried out in part at the MIT Lincoln Laboratory, sponsored by the Advanced Research Projects Agency of the Department of Defense, USA, and in part at the Division of Computer Research and Technology of the US National Institutes of Health.

ANIMATION: ERIC MARTIN

Animation is the graphic art which occurs in time. whereas a static image (such as a Picasso etching or a complex graph) may convey complex information through a single picture, animation conveys equivalently complex information through a sequence of images seen in time. It is characteristic of this medium, as opposed to static imagery, that the actual graphic information at any given instant is relatively slight. The source of information for the viewer of animation is implicit in picture change: change in relative position, shape, and dynamics. Therefore, a computer is ideally suited to making animation possible through the fluid refinement of these changes.

- Eric Martin, Carpenter Centre for Visual Arts, Harvard University, March 1969

ANIMATION: NORMAN McLAREN

Animation is not the art of DRAWINGS-that-move but the art of MOVEMENTS-that-are-drawn. What happens between each frame is more important than what exists on each frame.

Animation is therefore the art of manipulating the invisible interstices that lie between frames. The interstices are the bones, flesh and blood of the movie, what is on each frame, merely the clothing.

- Norman McLaren, National Film Board - Canada

MOTIVATION AND GOALS: RONALD BAECKER

When, late in 1966, I began to explore new computer animation processes, I was lucky to have relatively free access to a technologically advanced, interactive computer graphics system.

What is meant by an interactive computer graphics system? Like most computer systems, there is a digital computer of some sort, supported by a hierarchy of storage mechanisms. This may be general-purpose equipment, or may be specialized towards graphics. In either case such systems differ from standard computers in the way in which they and users communicate. Unwieldy decks of cards, and line printers spewing forth pages of output, are superfluous. Keyboard mechanisms are demoted to a secondary role. Instead there is a cathode ray tube, or display scope, on which the computer paints images, and an electronic pen, with which the user draws and points. (See Figs 11.1 and 11.2.)

This kind of computer configuration, appropriately programmed, affords the opportunity of true graphical communication between man and machine. Furthermore, communication is immediate, instantaneous, real-time, encouraging the kind of man-computer symbiosis that Licklider has so eloquently described.

Fig 11.2: Minimal components required to realize an interactive computer-mediated animation system are:

Within this context, we can understand specific goals of my work:

• We must reduce to manageable proportions the non-creative, laborious components of animation.
• We must do this without forcing the animator to learn a written programming language. Rather we must enable him to work through a natural language of simple and direct computer console commands, sketches, and real-time actions, a language which uses graphic modes of expression to the fullest.
• A single animator must then be able to exercise total artistic control over the production of a film.
• The artist must be able to 'doodle through time, changing the film with a single sketch or action, then immediately seeing the consequence in a real-time playback of the movie. The process must extend the language of animation, opening new modes of expression for what Lee Harrison calls man's dynamic visual imagination. Thus, with McLaren, we seek the art of movements that are drawn; with Martin, we seek the capability for fluid refinements of relative position, shape, and dynamics.

I have therefore focused strongly on developing new techniques for controlling the dynamics, movement, and rhythm, of changing images, seeking techniques that are:

• Effective, in that they allow precise and economical dynamic control;
• Expressive, in that they utilize human dynamics and exploit graphic and dynamic intuition; and,
• Provocative, in that they stimulate and engage the artist's involvement.

My research of 1967-8 is reported in two 1969 manuscripts, one my Ph.D. dissertation and Picture-driven Animation. These include descriptions of three interactive animation systems, ADAM, EVE, and GENESYS, which were operational on the MIT Lincoln Laboratory TX-2 computer in Lexington, Massachusetts. In 1969 and 1970 GENESYS was used extensively by Lynn Smith, an artist and animator who produced the images that accompany this article (with the exception of Fig. 11.2, which was done by the author). It is important to mention that Lynn Smith had no previous computer experience, and very little mathematical training.

GENESYS: SCENARIO 1

Step 0:

The animator, Lynn Smith, sits at the TX-2 graphics console, and types the word GENESYS to the computer. This activates the GENESYS system, which types back the response, OK. BEGIN ANIMATION.

Step 1:

She turns to the scope, picks up the stylus and moves it on the tablet, producing on the display a tracking dot whose motion mirrors that of her hand. (It was not difficult for her to accustom herself to moving the stylus on one surface and observing the result on another.)

On the scope are displayed a number of words. With the stylus she points to the word SKETCH, which GENESYS blinks in acknowledgement. GENESYS then blanks the screen.

When she again touches the stylus to the tablet, her pen movements are recorded as an image on the display. She draws a frog, indicating its completion by lifting her pen for a moment.

Finally she types the name FROG 1', and GENESYS files this image, or cel.

Step 2:

She points to the word SKETCH, draws a second frog image, then types FROG 2. The new image is filed as the second cel representing the object named FROG. The appearance of the display at this point can be seen in Fig. 11.3.

Fig 11.3: This is the way the GENESYS display looks after Step 2 of Scenario 1, the hopping frog.

Words at the top of the screen denote modes of system operation. GENESYS is now in the DRAWINGS mode, in which it accepts and files new cel drawings, and modifies existing ones. The artist commands GENESYS to carry out these actions by use of the instructions at the bottom of the screen.

The artist has just drawn two images representing the object named FROG. This is recorded by the cel directory, which reads 2 FROG 2. The first 2 refers to the number of cels; the second 2 refers to the cel currently under construction.

Step 3:

She points to the word DYNAMICS, then to the word P-CURVE. Again the screen goes blank, awaiting a sketch. But this time GENESYS interprets it differently, and displays the result differently. Now the symbols comprising her sketch are not evenly spaced, as in the drawing of the frog. Rather, the spacing is a record of the dynamics with which her hand traces the path. Where she moves slowly, symbols are clustered; where she moves quickly, symbols are widely-spaced. This resulting p-curve, depicted in Fig. 11.4a, will determine a movement for the frog.

Fig 11.4a: GENESYS is now in the DYNAMICS mode. At the bottom of the screen is a new set of instructions, used to construct movie dynamics. This is the first p-curve drawn by the artist (Step 3).

Step 4:

She points to the word PLAYBACK, the screen goes blank, then GENESYS plays back the movie. FROG 1 hops across the screen, its movement identical to that which she has just sketched.

Step 5:

Not quite happy with what she sees, for it is more a bounce than a hop, she again points to P-CURVE, then mimics another hop motion. (See Fig. 11.4b.) Now PLAYBACK reveals a more satisfactory result.

Step 6:

So that the frog may react when it hits, FROG 2 should be substituted for FROG 1 in certain frames. The animator points to the word EDITOR, to which GENESYS responds by displaying the graphs of Fig. 11.5. These represent the dynamic behaviour of the frog through time. The lower graph is a waveform that portrays the vertical component of the p-curve or hop that she drew. She finds the lowest point of the trajectory, where FROG 2 should be introduced in place of FROG 1, and marks it with an arrow. She communicates the substitution, and also the number of frames to which it applies, to GENESYS. The change is reflected by the upper graph, the selections graph, showing which cel is selected in each frame.

Step 7:

She again points to PLAYBACK, producing the result shown in Fig. 11.6.

Fig 11.6b

The sequence has been constructed and refined in about 5 minutes. Furthermore, the frogs are now part of the image library, the hops are part of the movement library, and either can easily be re-used in a new context. Thus the animator can quickly develop variations on the hop theme such as those shown in Fig. 11.7.

The real-time playback system is in principle very simple. Stored in the computer's memory is a numerical representation of each of the frog images, that is, a collection of data corresponding to the location of the dots which comprise them. Each 1/24 sec. the computer selects FROG 1 or FROG 2. The choice is determined by the selection graph. Then it decides where on the screen to display the frog. The position is determined by the p-curve or by the equivalent waveforms. The resulting frame is displayed for a 1/24 sec. This process is repeated to construct all frames, thereby producing a real-time playback of the movie. Actually, the animator can slow down or speed up the playback process to refine the dynamic feel of the result. In this case she slows the frog hop down to ⅛ sec. per frame, or almost 3 sec. for the entire sequence.

The filming is similar to the playback process. A movie camera is mounted on the display, its frame-by-frame advance controlled by the computer and synchronized to the painting of each frame. So that the frog hops at the desired speed, GENESYS paints each frame three times on the film.

GENESYS: SCENARIO 2

The goal is a line balancing a series of objects. These bound and rebound, adhere to or roll on the line, interacting in playful ways through the intermediary of the line. This scene is an exercise in the relationship of movement, apparent weight, and plasticity, and in the effect of these qualities and the timing of interactions upon achieving the illusion of cause and effect.

Phase 1:

The animator begins by defining with single cels a line and two balls. Using the p-curve technique she roughs in motions for the two balls, such as one entering, colliding with the line, and propelling the other ball away. An illusion of causality is achieved by controlling the timing of each interaction. She can change this timing with a single GENESYS command, then immediately see the consequence in a playback of the movie. Nonetheless, after making these corrections the scene still has a leaden, mechanical quality - objects still seem to drift lifelessly through space.

Phase 2:

Now she adds plasticity to each object.

Several new line shapes, or cels, allow the line to distort when impacted by a ball. Several new ball shapes turn the rigid balls into rubbery or clay-like substances that flow around the edges of the line. From the library of images stored in the computer, the animator retrieves several other objects previously drawn and substitutes them for the balls into the movements. The scene is now coming to life, and she can act and react spontaneously to her work at each moment in time. (See Figs. 11.8 and 11.9.)

Fig 11.9b:

Phase 3:

Phase 3: The hardest part is yet to come, and yet we might not have come this far, dynamically, using conventional animation technique. Perfecting the scene requires detailed editing operations, corrections in shape, movement, and dynamics. GENESYS provides the animator with tools that help her in this process:

The DRAWINGS mode: The animator can at any time resketch all or part of a cel, or add a totally new cel. Erasing is accomplished by retracing over the part of the image that is to disappear.

The DYNAMICS mode: We have seen the artist work with dynamics in two different ways: through real-time operations such as the p-curve, with GENESYS transmitting her dynamics directly into the movie, and through the formation and correction of static images which depict picture change.

>Two such kinds of static images are shown in Figs. 11.9 and 11.10. The image series in the right column of Fig. 11.9 contains multiple exposures of groups of frames. The image of Fig. 11.10 graphs picture change through time, showing which cel represents each object in each frame, and showing one object's movement along one dimension.

These are two solutions to the problem of representing the complexity of a series of frames and portraying the essential features in a single image - one that can be easily comprehended and later manipulated and modified. Good representations of picture change bring power to the animator: power to control dynamics, because small corrections yield sizeable changes; power to conceive dynamics, because the dynamics are visualized.

CONTROL THROUGH ABSTRACT DYNAMICS

How does one manipulate or modify a movie through the use of abstract representations of dynamics?

Consider the symbols of the p-curve to be beads on a flexible string, or on a rubber band, one that can be repositioned and reshaped by grabbing hold of a bead with the stylus and pulling on the string. Each such reshaping changes the movement defined by the p-curve.

Look at the graphs plotted on a common time axis. The movement of one object, or several interacting objects, can be slowed down or speeded up by expanding or compressing the time scale on one or several of the graphs. It is easy to experiment with relative timing, by adding or removing, lengthening or shortening, the pieces of the action. An action, that is, a particular pattern of cel selections and movements in time, can be defined as a unit and repeated at arbitrary intervals. One such operation may affect every frame in the movie, yet can be executed as quickly as changing a single frame in conventional animation.

Thus some of the animator's refinements are made by typing new sequences of image choices for particular intervals, by sketching new p-curve and waveform fragments, and by inserting, deleting, shifting, copying, stretching, and shrinking sections of existing movements.

The EDITFRAME mode: Other effects are dependent upon achieving specific extreme phenomena in individual frames - particular distortions or unusual geometrical arrangements of familiar forms; or specifically co-ordinated phenomena - several objects which must align or relate in a precise way. It may be necessary to draw new cels just for that purpose. However, many effects can be achieved by carefully rearranging, realigning, and reselecting existing cels. This is the purpose of the EDITFRAME capability.

The animator first selects a frame for modification. She can then experiment with alternative selections of cels in that frame. She can move and reposition a selected cel by grabbing hold of it with the stylus, dragging it along, and then freezing its position by a flick of the pen. To aid in this positioning process, GENESYS will, if requested, superimpose adjacent frames upon the one being edited. Such frame-editing configurations are additional representations which facilitate the perfection of a movie. An instance which occurs while refining the balance scene is described in Fig. 11.11.

GENESYS: SCENARIO 3

The animation of the first two scenarios has been done in assumed two-dimensional form. The same effects could have been achieved on an animation stand, albeit much more slowly, by making appropriate substitutions of one cel for another and by moving them horizontally and vertically. But the computer can also transform cels along other dimensions. One such dimension is size - in fact, both the horizontal and vertical components can be varied independently. Another such dimension is orientation - a cel can be rotated about any point. These two simple additions give us a capacity for dynamically varying existing images in a totally new way.

Consider the simple circle and the hopping p-curve of Fig. 11.12b. The animator superimposes upon this two more movements, a smooth rotation of the circle about one point on its perimeter, and a smooth compression and expansion of one dimension of the circle, thus making it a changing ellipse. To be precise, the shape is first compressed along the horizontal dimension, then rotated about the fixed point, and finally positioned on the screen. (The order in which these operations are carried out is significant.) The result, shown in Figs. 11.12k and 11.12l, is the illusion of a flipping coin, turning and twisting in space.

The dynamics of the illusion are determined by the synchronization of the four movements, horizontal, vertical, turning, and compressing. The compressing looks like twisting, or turning into a third dimension. The animator controls these dynamics with the set of graphs shown in Figs. 11.12a, 11.12d, 11.12g, and 11.12j. The graphs depict variation through time along the four movement dimensions. By resketching parts of waveforms, changing the height or slant of linear portions, or adding interesting sections such as sinusoidal oscillations, she can develop an infinity of dynamic variations of this simple theme.

Infinitely more variations result from changing the centre of rotation and compression, and more significantly, from using a collection of cels, not merely a single one, to define the object. For instance, alternating between circular and star shapes before applying the above transformations yields a startling and unique result.

One final point need be made about size change and rotation. Like positional changes, their variations may also be mimicked in real time with a p-curve. To make a single shape pulsate, the animator need only move her pen in and out, in and out, in and out. The shape then expands and contracts with a movement embodying her dynamics. To make an object turn, she need only make circular motions with her pen, clockwise and counterclockwise, faster and slower. The object then carries out the same rotational movement.

Now, perhaps, we can begin to realize how the p-curve acts as a direct medium between expressive human dynamics and expressive image dynamics.

LINE QUALITY

Most cels consist of sparse dot drawings, or can one achieve richer and more varied line quality, texture, or tonality?

Currently, the animator controls with the set of toggle switches the spacing between the dots of each image. If the spacing is chosen small enough, and enough dots are used, the line appears solid. However, most images were sketched with the fewest dots required to convey an outline.

This is due to certain technical limitations of the current system related to the memory organization and display speed of the TX-2.

These limitations can be overcome. One simple solution is to allow the animator to sketch cels with as many dots and as complex a texture as desired, to save the full cel in auxiliary memory while using a sparse cel skeleton in main memory and for real-time playback, and then to reintroduce the full dot density while filming. This has not yet been done because in building an experimental prototype system the idea was to concentrate on achieving resolution and refinement capability in time rather than in a single image.

COMPUTING IMAGES AND DYNAMICS

Does GENESYS's emphasis on the freehand drawing of images and dynamics, rather than the ability to compute them (that is, to specify them with computer precision) imply that the latter capability cannot be smoothly integrated with the system?

Currently the animator can sketch cels, p-curves, and waveforms freehand, or can call up from a very limited library certain regular patterns, for example, a square cel, or a sinusoidally oscillating waveform. She cannot call up, for example, a cel that is a precise parabola, or a p-curve that generates the mathematically correct motion of a planet around the sun. She also cannot with reasonable effort sketch these accurately enough, and she should not have to. Thus we see that there are constraints on the class of moving images that can be naturally produced with the current system.

These constraints can be removed. GENESYS can easily be augmented so that a suitably skilled animator could, in a standard programming language, generate precise cels, movements, and rhythmical patterns, and then use them interchangeably with those sketched freehand.

Despite the limitations of the prototype system, I have used it to quickly produce some short precise diagrammatic animation sequences. These, on average, took 2 to 4 hours of console work to produce 10 sec. of film. This includes experimentation with different dynamic variations. The sequences contain simple moving points, straight lines, and text. Sequences of comparable complexity would be suitable for conveying concepts of language or mathematics.

With only these two straightforward additions, GENESYS could be used to generate a wide variety of films for education, entertainment, advertising, or pure art.

INTERPRETATION AND CONCLUSIONS: 1

Immediate playback encourages trial-and-error experimentation to achieve sophisticated visual effects. Instantaneous feedback is one of GENESYS's many tools for controlling and evaluating new dynamic combinations and transformations of static images. With these tools the animator can make powerful use of small numbers of cels.

Yet no computer experience is required to use the GENESYS system. The animator communicates with it by pointing at words that represent commands, by typing short messages, and by sketching a variety of images. GENESYS aids the naive user by removing commands from the display when their activation would be meaningless. Also it offers short helpful hints or detailed instructions when requested, or in response to obvious errors.

GENESYS is one realization of a new animation model called picture-driven animation. The animator sketches and refines static images that are used as components of individual frames of the movie, and static dynamic images that represent movement and rhythm. These latter pictures drive the former pictures into motion - hence the term picture-driven animation.

The driving images represent what might be called global dynamic descriptions: path descriptions, continuous variations of value; selection descriptions, recurring sequences of discrete choices, and rhythm descriptions, temporal patterns marking events. Dynamic descriptions are an evolving vocabulary for visual dynamics, vocabulary for expressing picture change. Work with GENESYS is evolving a language for dynamic control based on this vocabulary.

Dynamic control is exercised in two fundamentally different ways.

Sometimes, as with the p-curve, the dynamics of the animator's sketch or action are recorded by the computer, remembered, and later used directly as image dynamics. The dynamics are body-sensed, body-produced. The animator acts, the animator mimes. The creating is intuitive, synthetic.

At other times, as when the animator draws a waveform or selection graph, the sketch is not made in time, and the dynamics of the act of drawing are not recorded. What is remembered by the computer is only the resulting graph, not how fast it was drawn, or whether from right to left. Of course, the graph still determines image dynamics, since one of its axes represents the flow of time. The animator sketches the graph, views both it and the moving image it determines, resketches or edits the graph, and considers the effect again. The dynamics are conceived, visualized. The creating is intellectual, analytic.

Whether dynamic control is exercised intuitively or intellectually, a single driving image determines features of an entire sequence of frames, hence a single sketch or action controls timing, rhythm, and dynamics in that sequence. GENESYS gives animators great power - they can for the first time control movement and rhythm as easily as they control form. GENESYS provides power tools for the art of movements-that-are-drawn.

THE ANIMATOR'S VIEW: I: ERIC MARTIN

Animation quality is dependent on how a thing behaves rather more than how it looks, or so it ought to be. In fact, in animation, anything can move in any way we want it to, and that is the point and the glory of it.

The stubborn insistence of the GENESYS system on dynamics rather than superficial form is its genius. In GENESYS, the animator is led inexorably to the very quick of the matter, dynamics generating form - much as a muscle spasm articulates shapeless protoplasm.

That obscene joke, the typical American parked car, gives us a metaphor for the opposite: a static opera of steel doing precisely nothing and going precisely nowhere. In reality, things are functionally identical when they are at rest, despite the clever visual camouflage of apparent visual difference. Only when things move do they take on identity. Dynamics is the very stuff of animation.

Ideally, one frame of animation ought to appear to be a meaningless pattern which will only yield its coherence as part of a flow. For a precise, if mundane, illustration of this principle I offer the following example:

Imagine a man in a black suit walking in the dead of night. The only evidence of his presence is a set of luminous points attached at random to his body. At any instant in time we are aware of these points, but not of their significance. In motion, miraculously, we read a walking man as the synthesis of the evidence. It is almost worth stopping here to try to imagine this since it is the perceptual basis of animation, in extreme form.

Some current work with GENESYS is moving away from conventional animation problems toward experiments with similarly slender graphic evidence in which dynamics alone evidence form. This promises to provide radically new insight into the unique equilibrium between form and movement which characterizes the medium, and is at the source of its infinite expressive future.

INTERPRETATION AND CONCLUSIONS: 2

GENESYS also provides tools for the fluid refinement of relative position, shape, and dynamics, for controlling the unique equilibrium between form and movement. At instants where precise spatial relationships are critical, fine tuning of these frames is done in the EDITFRAME mode. Alternative cel selections can be tried, or the animator can return to the DRAWINGS mode to sketch a totally new shape. Of course timing and rhythm can be exquisitely tuned in the DYNAMICS mode.

Thus the GENESYS animator moves freely among the three modes, refining static images in the DRAWINGS mode, refining movements, timing, rhythm in the DYNAMICS mode, and working at the intersection of the static and the dynamic, the individual frame, in the EDITFRAME mode. The movie is neither conceived nor developed frame-by-frame, but rather in continuous layers of qualitative changes, first roughed-in, then refined as often as necessary.

As each movie is developed, the system itself grows. Cels, dynamic descriptions, and animation sequences all become part of an expanding library stored in the computer (Fig. 11.13). The scope of the library grows faster than its true size, for new dynamics can be applied to existing images, new images to existing dynamics.

The artist working in such a reactive medium is growing too, learning more about herself and about animation. She is also discovering, or uncovering, creative potential not planned by the system designer. She now discusses her experience and her process of discovery:

THE ANIMATOR'S VIEW: II: LYNN SMITH

I entered the computer world with an art background, TV experience, and two completed films done in the combined technique of direct drawing and collage. Toggles, switches, inputs and outputs, modes and circuit-breakers were in no way related to my previous experiences in art, animation or life.

Many people laughed when I sat down at the computer - concerned friends and doubting computer people. My concerned friends questioned whether my drawing style (not to mention my life style) was suitable to an inhuman machine-environment. The doubting computer faction questioned whether my lack of knowledge of toggles and tapes, and switches and glitches, was reconcileable with the computer graphics mode. I must admit there were times when sitting at the console with stylus in hand I found myself laughing.

But it did not take very long to become familiar with the machine, and the system turned out to be quite human after all. In fact, this system encourages the individual to tap some of his suppressed human dynamic and kinesthetic resources.

Remember the old riddle, how would you describe a spiral staircase? A person presented with such a question would inevitably find it difficult to choose the right words. When the poor fellow finally gave up and described in the air with his hand the unique coiling motion of the staircase, he was considered a loser. It is a popular misconception that physical movement and gesture to express an idea are indicative of an inarticulate person.

In animation also, there are certain rhythms and paths which should be defined by methods other than words and other than those offered by the conventional animation industry of today. The ability to mimic a path, and also to have the rhythm of this path immediately applied to a designated cel, demands that the animator rely on his body memory or his kinesthetic resources. The graphic dynamics which result from a successful empathy experience can have a vibrant life energy.

In the beginning, I had reservations about the freedom to draw into the TX-2 computer. Surely one did not want to just draw cartoon characters. It seemed only right that the images should reflect the medium in which they were done. Should the images be very simple and emphasize the dynamic possibilities of the system, or should they be ... what? - I didn't know.

But I found a clue. Even when recognizable such as those in Fig. 11.13 were played back on the scope, they looked as though they were generated by an electronic machine. This was because of the phosphor decay which preserved in levels of tone the vague and fading record of where the cel had been, while implying where it might be going. The gradation from the barely perceptible remnant of the cel to the vivid outline of the most recent cel was very beautiful and enhanced the sense of continuum. This alerted me and stimulated me to experiment with the other possibilities.

The type of imagery as is shown in Figs. 11.14 and 11.15 was discovered in the course of filming from the scope. In this process I disconnected the camera and played back sequences to check them before actually filming. When I did this, the film did not advance and the shutter remained open. This caused some frames to be drenched with dots, while others were just spattered.

Later, looking at the developed film in the editor, I was delighted at the infinite variety of dot textures. These were not merely textures, but patterns of light, ranging from faint overall dappling to super-flashes which exploded and could remain on the retina until they overlapped on to succeeding frames.

But it is when one combines these flashes, and orders them with other sequences, simulated decays, and flashes, that the excitement really begins. Look at Fig. 11.14 and imagine possible combinations, for example: a whole record of a movie could remain static and dim in the background as a bright burning cel lights up through it; clusters of dots could burst into flashes and create patterns which could interweave and form images and then return to lightly dotted space. Recognizable forms may be used as part of this evolutionary approach - from dots to patterns to things and back to dots. The resulting imagery preserves the electronic light-zapped quality of the computer while also reflecting human control in design and rhythm.

Fig 11.14: Examples of Light textured flash frames.

A PARADOX

Although both animators working on GENESYS are exploring the unique equilibrium between form and movement, their approaches, inevitably, are very different. Eric Martin wants to explore slender graphic evidence in which dynamics alone evidence form; Lynn Smith wants to use the dynamic capabilities of the system on more complex imagery. Her goal highlights the equilibrium that exists between the design and use of any computer system. It is important for those interested in computer animation to understand this equilibrium.

Currently, GENESYS will facilitate Eric Martin's goal and make Lynn Smith's difficult, forcing her to work outside the system. Slender graphic evidence can be played back in real time; flash frames cannot even be played back. Lynn Smith must first record the flashes on film, then order and combine them in a Moviola, a painstaking frame-by-frame process in conflict with two design goals of the system. Furthermore, there is currently no language for generating flash frames.

Despite these difficulties, beautiful results can be produced.

This tension is fundamental to the art of computer systems design. It may seem paradoxical, but one dimension of GENESYS's success is its ability to provoke Lynn Smith to explore the impossible and uncover the potential, which in turn provokes further exploration of the potential and make it possible.

Interactive animation systems must evolve in response to their use. A schema has been developed to assist thinking about dimensions of future evolution.

FUTURE ANIMATION SYSTEMS

The marriage of computer animation and interactive graphics might be called interactive computer-mediated animation.

In a medium, through a dialogue, we form and shape and represent content and meaning, the subject matter of the dialogue. An interactive language must mediate and enhance creative expression of that subject matter.

The matter of animation is first and foremost images, hence graphically rich imagery should be mediated through graphic, pictorial interaction. The matter of animation is picture change, picture dynamics, hence rich dimensions of dynamic variability should be mediated through interaction expressive in time and of time. The matter of animation is rich sensory experience, hence dynamic intuition should be expressed through tangible, sensual interaction. The matter of animation is symmetry and order and organization, hence precision and structure should be mediated through interaction expressed simply, clearly, cleanly. The matter of animation consists of large collections of images, image sequences, changing images, hence the transaction time per interaction and the transaction time per mediated image should both be small.

The job for the future, therefore, is to consider the five key dimensions - graphic, dynamic, sensual, structured, and immediate, and to increase: the representational power of the medium in how much it allows the animator to express along each dimension; and, the naturalness and flexibility and responsiveness of the interaction between animator and computer along each dimension.

THE GRAPHIC DIMENSION

In the future some will be content to explore the unique graphic qualities of the GENESYS electronic image. Others will cite the intricate textures that have been produced with BEFLIX, the University of Utah successes in generating synthetic shading, and the Computer Image Corporation's capability for animating real video images of arbitrarily rich graphic content, and want to work with these kinds of imagery.

I see the computer animator of the future sketching with a flexible computer paintbrush which will not only form new images but erase and transform existing ones, and which can be tuned by the animator to achieve distinctive styles of her own. The animation system will offer incredible varieties of line, shading, and texture, as well as powerful control over timing, rhythm, and dynamics.

DIMENSIONS OF DYNAMIC VARIABILITY

GENESYS stands for the GENEralized-cel animation SYStem. A generalized-cel is a picture description, expressed in some language, which is driven into motion by varying a set of features. GENESYS currently embodies a single simple generalized-cel definition: a group of cels is animated by selecting in each frame one from the group, then varying the chosen cel's position, size, and orientation. We enrich a picture-driven animation system by including more powerful generalized-cels, by adding new features whose variation through time also determines visible picture change.

It is important to note that the current techniques for dynamic control are applicable without alteration to all these new features.

Some simple examples are the intensity of the outline of an image, the density or brightness of shading, and the colour of an image. The first two would likely be continuously variable, and hence controlled by sketched waveforms; there would probably be a small number of colour choices, described by a selection graph. By controlling with a waveform the percentage of an image that is visible at each instant; one can vary the dynamics of growth (formation) and shrinkage (disappearance) of that image through time. Other interesting transformations will include distortion, decomposition, varying line width, shading, and texture quality, and adding controlled randomness.

THE SENSUAL DIMENSION

Currently, with GENESYS, the animator expresses her intuitive, bodily-sensed dynamics through a pen on a planar surface. Using a set of pushbuttons, she can tap a rhythmical sequence directly into the computer. Animators at the Computer Image Corporation can jump and dance, while wearing a light harness, their dynamics being transmitted directly into the computer.

The future will likely see the use of pressure-sensors and grip-sensors in electronic paint-brushes, three-dimensional wands for drawing and sculpting directly in space, and a variety of new devices to facilitate the transmission of expressive body movements into expressive synthetic moving images.

The analogies between the synthesis of graphic variation through time, or animation, and the composition of music have long fascinated Alexeieff, McLaren, Whitney and others. The computer animator of the future will use systems in which sound and visual components can be integrated from the very start of a movie, in which real-time playback will include both audio and video, in which graphs of picture change and of sound and music will be superimposed on the same time frame in a unified editing system, and in which music waveforms can be sketched along with graphic waveforms.

DIMENSIONS OF PRECISION AND STRUCTURE

Future developments will include the previously discussed integration of freehand sketching and precise definitional capabilities, and the ability hierarchically to build complex images out of single images, complex movements and rhythms out of simple movements and rhythms.

THE DIMENSION OF IMMEDIACY AND RESPONSIVENESS

It is necessary to preserve real-time playback capability despite increasing graphic complexity of the animated images. This can be done as suggested previously, with the aid of video tape, or in other ways.

Finally, we are concerned with the quality of immediate interaction, and not merely with minimizing the time lag to a response. Approaches towards increasing the flexibility and responsiveness of interactive animation systems have been suggested. I predict that as a number of animators use a computer animation base system of the future, these will evolve, symbiotically, towards a number of personalized systems oriented towards, and responding to, the needs of those individuals, personal animation-machines, true extensions of man.

ACKNOWLEDGEMENTS

Lynn Smith and Eric Martin have contributed significantly to the form which both the GENESYS system and the article eventually assumed. I am grateful to Lynn and Eric for their dedication, their insight, and perhaps above all their warmth and sense of humour. Group 23 of MIT Lincoln Laboratory, the Lincoln Photolab. and Miss Rose Najarian and Miss Jody Grande of the NIH, all aided beyond the call of duty in the preparation of the article.

It is with pleasure that I also acknowledge the, guidance, support and enthusiasm received from Professor Edward L Glaser, currently Chairman of the Department of Information and Computer Science at Case-Western Reserve University, Cleveland, Ohio, USA.

Fig 11.15