The creation of an animated film begins with a storyboard consisting of a series of frames. Between each pair of storyboard frames, the artist must draw many intermediate frames to produce smooth animation. This paper describes an interactive computer graphics program which automatically produces the intermediate frames for review or filming. The program runs on an IBM 1130/2250 with a Sylvania tablet attached to the 1130. The user can draw the storyboard frames on the tablet and ask for animation on the 2250. The intermediate frames are calculated using linear interpolation.
Karma: The forces or motions generated by actions, and their continuous effects on the physical and psychological planes. - Bhagavad-Gita
The creation of an animated film begins with a graphic outline called a storyboard, which consists of a series of frames and looks much like a comic strip. The purpose of the storyboard is to exhibit the story line and continuity of the proposed film. From the storyboard, the artist prepares a series of key frames (extremes), and then draws many frames (inbetweens) intermediate to each pair of extremes to produce the desired animation. The storyboard frames are not used in the animation, but serve only as an outline. Thus conventional methods involve a laborious frame-by-frame construction process. Most computer animation work has either automated the input or display of artist-drawn frames, or developed programming languages and non-interactive systems for artists - turned - programmers.
This paper describes an attempt at the computer generation of film sequences directly from the key frames drawn by the artist. The computer produces the intermediate frames, leaving the creative aspects of animation to the artist. The system, called KARMA, provides an interactive response to the commands of the artist, and permits immediate review of the animation.
The artist draws the key frames of a storyboard on a computer-controlled tablet, and the program automatically produces and consecutively displays the intermediate frames. Facilities are provided for input, storage, modification, and display of the key frames. In addition an animation review facility permits the artist to see the effects of the computer produced animation and also permits direct filming by a camera coupled to the display unit. The user can specify the number of intermediate frames produced between any pair of key frames. The intermediate frames are calculated using a simple linear interpolation which is described in the appendix.
KARMA is implemented on an IBM 1130/2250 system, a small (32K 16 bit words) general purpose computer with an attached display unit. KARMA accepts graphic information from an on-line Sylvania Data Tablet, and immediately reproduces on the display screen whatever is drawn on the tablet. Since the stylus contains a ball point pen, the user can direct his attention either at the paper on which he is drawing (or tracing a pre-drawn figure), or at the screen. The stylus position is indicated on the display by means of a cursor. Bolted to the tablet is a peg bar for registration of standard punched animation paper. The drawing area provided by the tablet is eleven inches square.
The objective of this work is to investigate the methodology and functions needed for a practical and powerful computer-assisted animation facility. A special emphasis has been placed on naturalness and ease of use by an artist, so that a long training period is unnecessary.
Two notable papers have influenced this project. The first by Miura et. al. (1) reported the first use of linear interpolation techniques for computer animation. The system was implemented on hybrid equipment and emphasized the interpolation process itself, rather than its use in a practical system. The pioneering paper by Baecker (2) described a powerful interactive animation system operating on a time-shared computer. In that paper the methodology of computer graphics was the primary topic. The notion of the interactive graphical specification of picture dynamics, developed there, has been used in this work.
Each key frame consists of a number of curves. As the curves are drawn they are stored in the computer and displayed as a sequence of line segments sufficiently short so that the curve appears smooth. This is illustrated in Figure 1. The interpolation process produces the curves intermediate to a curve in frame 1 and a corresponding curve in frame 2. Correspondences are necessary since legs should be interpolated into legs, hands into hands, etc. The KARMA system establishes a correspondence of the first curve drawn in frame 1 to the first curve in frame 2, the second curve to the second, etc. If two key frames have a different number of curves, only those curves which have a correspondent are displayed; the extra curves do not appear in the interpolation process.
A similar correspondence exists between line segments of corresponding curves. Normally two curves will not consist of the same number of line segments, and in this case the curve with fewer line segments is redrawn so that the number of segments agree (see appendix). Then the first line segment of a curve recorded in frame 1 is associated with the first line segment of its corresponding curve in frame 2. The direction in which a curve is drawn is therefore significant.
Figure 2 shows the result of interpolating the input of Figure 1. Another example is shown in Figure 3, where the input was the top and bottom lines and the computer produced the others. The coarseness of transition between adjacent images in both figures is due to the small number of inbetweens requested.
When the user asks to see the computer-generated intermediate frames, each line segment of frame 1 is linearly interpolated into its associated segment in frame 2. The same processes are repeated for multi-frame animation using a sequence of key frames stored on disk. Just as frame 1 is smoothly transformed into frame 2, frame 2 can be transformed into frame 3, etc. This simple interpolation technique has been found extremely satisfactory both in terms of animation technique as well as computation time.
There are two modes of entry and display of the key frames: the first is illustrated in Figure 1, and the animation is performed from the frame specified on the left of the centerline to the one on the right. Each frame is defined relative to its center cross and the animation is performed in the center of the screen.
To maintain the desired correspondences between curves, the artist must draw the curves in the same order and in the same direction in each frame. To lessen the burden imposed by this requirement, the user is prompted automatically. For example, after completing the first frame, the user will move his pen across the centerline to begin the second frame. After the pen has crossed the boundary, the first curve of frame 1 will be repeatedly traced out in the direction it was originally drawn. When the user draws the first curve of frame 2, the second curve of frame 1 will be traced out, and so on.
The artist may draw curves in either frame in any sequence simply by moving his pen from one frame to the other and drawing. The program will always remind the user of the correspondences. There is also a convenient editing facility which permits the modification or erasure of any selected curve.
This mode was our initial experiment with the transformation techniques described. It shortly became obvious that this mode had two disadvantages: only half of the screen can be used for each frame, and registration of two frames is difficult.
The second mode is a tool for the working animator, and was used to prepare the animation for the film accompanying this paper. Two consecutive key frames labelled F (From) and T (To) are retained in core memory. The frames are shown superimposed, with the selected frame brightened. The user can select one by depressing the appropriate button on a function keyboard adjacent to the tablet, and add to the selected frame by drawing on the tablet. As an additional option, he can ask to see only one frame at a time. Review of the intermediate frames may be made at any time.
A sequence of key frames can be entered and reviewed from beginning to end. To allow this, frames are named with frame numbers and filed on a disk. After a pair of frames is filed, the system is ready to accept the next key frame in sequence. For example, if the pair of frames is filed as frame 1 and frame 2, then frame 2 is retained in core and relabelled F. If a frame 3 has previously been stored it is automatically retrieved, placed in core, and labelled T. If no frame 3 exists, the T frame is empty, and can now be defined.
Linear interpolation is a technique for deforming one curve into another through a number of intermediate stages. However, in many scenes a figure is not deformed, but merely stays stationary or is translated as a rigid object. KARMA has a facility for copying an object of one or more curves from the F frame to the T frame and choosing a position for the object. The intermediate frames will make the object appear to move between the two positions.
To implement this copy and translate function, a method for defining graphic objects, or entities, is provided. Each entity consists of one or more curves which are to be copied from the F frame to the T frame. When a particular entity has been copied the T frame is displayed, and the copy appears in the T frame in its original position. The copy can be translated to any desired position with the stylus. Each entity can be translated independently.
KARMA has three different methods for generating the intermediate positions of a translated object. The object can be moved along a straight line connecting the initial and final position. Alternatively, the user can move the copy over an arbitrary path specified by motion of the stylus; intermediate positions are then calculated along the path (Figures 4 and 5). A third method permits the user to touch the stylus to the tablet to define the precise location of each intermediate frame.
The animator will usually need to combine the two techniques of interpolation and translation. For example, a human figure may be translated while the hands and legs are interpolated. To do this, the entire figure is defined as an entity and copied into the T frame. Translation may be effected by any of the three methods just described. The curves that are to be transformed using the interpolation process are then erased from the copy and redrawn. During review, the translation and interpolation are combined as required.
In a typical sequence objects appear or disappear spontaneously from view, often without any gradual prior changes. However, interpolation between two consecutive key frames does not permit the appearance of a new curve, or the disappearance of one previously present. It can only transform the curves of the first frame into the corresponding curves of the second.
This capability is obtained by using a multiple frame sequence, where the frames are stored on the 1130 disk. For example, suppose frame 1, frame 2, and frame 3 have 6, 9, and 9 curves respectively. Interpolation between frames 1 and 2 will be followed by interpolation between frames 2 and 3. In the first interpolation the curves of frame 1 are transformed into the first six curves of frame 2. The final three curves in frame 2 are not displayed. The second interpolation will transform all nine curves of frame 2 into the corresponding curves of frame 3. The resulting sequence will show the appearance of three new curves.
Similarly, curves can be dropped from view. If frame 4 has seven curves corresponding to the first seven curves of frame 3, then during the interpolation between these frames the last two curves of frame 3 will not be displayed. The effect over the sequence is the disappearance of two curves.
KARMA also permits changes of scene, where the entire sequence shows a jump from one scene to another. For this purpose two store functions are provided; an initial store to identify the first frame of a new sequence, and a normal store for subsequent frames. For example, suppose frames 1 and 3 have been stored as initial frames and numbers 2 and 4 as sequels. A review of frames 1 to 4 will show an interpolation from frame 1 to 2, followed by an interpolation from frame 3 to 4. The effect is a cut from frame 2 to frame 3 rather than a smooth transition.
The earlier sections of this paper have outlined the capabilities of KARMA; this section indicates how control is exercised. The various functions are initiated by depressing a button on the function keyboard. In the outline that follows, the function associated with each button is itemized.
The current repertoire of KARMA seems to be a useful and practical set of tools for the creation of animation sequences. Since it was designed to exploit the techniques of linear interpolation, KARMA does not have the sophistication and versatility of conventional animation methods. However, it has been successful in achieving its primary aim: the elimination of a large amount of routine hand work. In addition, it is especially well adapted to the economical linear style exemplified by the artists Steinberg, Blechman, and Popko. Linear interpolation has been surprisingly effective as an intermediate frame generation technique. No more complex interpolation algorithm seems necessary at present, although incorporation of a non-linear technique would be straightforward.
Additional functions could be added to the system in order to enhance its power. Presently review proceeds between two key frames in a standard way: from the first key frame to the last, with repetition until the user stops it. For filming, this review is performed once. We are investigating a more complex review which would permit certain frames to be displayed cyclically, interpolating from first to last, then last to first, etc. With cyclic review superimposed on linear review a human figure could be translated while the arms and legs are cyclically interpolated to produce a walking cycle with a minimum of art work.
Rotation and scaling can presently be done reasonably well by interpolation. However, scaling and rotation functions similar to the copy and translate facility would be useful.
The ability to film in color directly from the CRT using color filters would also enhance the power of the program. Color could be added to KARMA with new functions enabling multiple exposures of different color overlays for each intermediate frame. The system would position the various filters in front of the lens and advance the film once a frame is complete.
The interest and intuitive understanding of computer graphics by the animator Stan Popko are gratefully acknowledged, as are the interest and support of E. J. Casazza of IBM World Trade Corporation. Mr. Popko prepared the artwork for the animation in the film accompanying this paper, and the photography is by D. H. Morehouse of IBM Kingston. Conversations with R. D. Tennison, A. Stein and others at T. J. Watson Research Center were very helpful in this effort. A conversation with N. Burtnyk of the National Research Council of Canada at an early stage of the project helped to stimulate our efforts. He seems to have followed a similar line of development, and his paper Computer Generated Key Frame Animation has been submitted to the SMPTE journal.
1. T. Miura, J. Iwata, and J. Tsuda, An application of hybrid curve generation - cartoon animation by electronic computers Proceedings 1967 SJCC, pp. 141-148.
2. R. M. Baecker, Picture driven animation, Proceedings 1969 SJCC, pp. 273-288.
To illustrate the algorithms used in this paper, consider the simple diagram below. The line segment on the left is an element of a curve in the F frame, and the segment on the right the corresponding element in the T frame. The interpolation algorithm in effect connects the endpoints with straight lines, marks those lines at equidistant points, and connects the appropriate points to generate the intermediate lines. The interpolation is performed for all corresponding segments of all pairs of curves.
If two curves have different numbers of segments the segment correspondence cannot be made. The curve with fewer segments is then redrawn preserving its original shape, with a segment count equal to the curve with more segments. For example, suppose the curve in the F frame consists of two segments and the corresponding curve in the T frame has five. The F curve is modified to contain five segments as follows: The first point is retained, the second and third are chosen at 2/5 and 4/5 along the first segment, the fourth and fifth points at 1/5 and 3/5 along the second segment, and the final point is retained as the sixth point. The line segments connecting the new points (marked by X in the figure below) form the modified curve. If the line segments are small and equally spaced, the shape of the curve is preserved.
The two algorithms can be formalized as follows: assume that the F frame contains a curve made up of m-1 line segments connecting the series of points (X(i),Y(i)) i=l,...,m, and the T frame contains a corresponding curve of n-1 segments connecting (U(j),V(j)) j=l,...,n. If m<n the F curve is replaced by the curve connecting (X'(i),Y'(i)) i=l,...n where
X'(1) = X(1)
X'(2) = (X(2)-X(1)) * (m/n) + X(1)
X'(3) = (X(2)-X(1)) * (2*m/n) + X(1) if 2*m/n < 1
= (X(3)-X(2)) * ((2*m/n)-1) + X(2) if 2*m/n ≥ 1
......
and similarly for the Y' values.
Once the segment counts are equalized, the interpolation can generate the intermediate frames. For each frame f a new curve is generated with the points (A(f,i),B(f,i)) where i is the point index, and f the frame index. If M intermediate frames are to be generated, then f=1,...M and i=1,...n. The new curves are defined by
A(f,i) = (U(i)-X'(i)) * (f/M) + X'(i) B(f,i) = (V(i)-Y'(i)) * (f/M) + Y'(i)