Using a Computer to make Animated Films

Colin Emmett

1972

Computer Arts Society

This paper gives a good description of the internal workings of Antics despite never mentioning the name of the system.

THE AIMS OF THIS PAPER

The overall problem with which I am concened is one of description, how to describe the idea to the machine. The first requirement is to be able to describe the idea to yourself, then to find a way of translating the description into a form the machine can understand. After that there is the problem of programming the machine to produce what you have described. In this paper I intend to deal with my solution to the first and the second description problems, and to deal with one or two of the principles of the third.

THE ROLE PLAYED BY THE COMPUTER

In an animated film, about 90% of the work involves the production of the pictures, very little of which work is creative or even interesting. So at the lowest level of computer usage, it is justifiable to use the machine to produce the pictures for you, provided you can describe what you want and program the machine to produce it. The computer makes possible types of animation which would be difficult or impossible by other methods. It makes it possible for an individual to produce work previously requiring a small army of animators. There are limitations to the images of course, but they are not as great as might be supposed. Solid areas in a wide range of colours are possible, and lines can be textured, though not to the sensitivity of a pencil line. In addition of course three dimensional effects are easily achieved, many sequences can be animated in parallel, and movements can be exactly controlled no matter how fast or slow they are. Add to this the possibility of you being able to design and produce films exploiting these possibilities without having to employ a small army of assistants, and you can see why the possibilities of computer animation are exciting.

FROM IDEA TO ANIMATED FILM

I am concerned here with the making of a film which tells a story. There are other types of film, but I am not concerned with them at the moment. The first task is to prepare a detailed description of the story in pictures and words, called the storyboard. This may include images or effects which are only possible to generate using the computer. Anyway, the making of a film generally starts with the production of the storyboard, which may be used to obtain the money to make the film. The next major task is the composing and recording of the soundtrack. It is a great deal easier to synchronise the pictures with the sound than the reverse.

The composer may desire to see examples of the animation in order to obtain a feel for the type of sound required, but he can generally work directly from the storyboard. The recording of the soundtrack is transfered to 16mm magnetic track, and using a Pic-Sync the sound can be analysed to single frame accuracy. This provides the basis for the timing of the animation. An example of such an analysis from the film Bean and Beane which I am still working on, is shown:-

Millimeter graph paper is used, each mm representing one frame, time running from right to left. The animation is broken down into keyframes, each key occuring at a particular frame number. A number of animation sequences may occur simultaneously, as in this example the eyes and the body of the bean. The image used and the position are roughly indicated, as is the basic action being performed. Having worked out what is to happen to this degree of detail, the action is sketched out on paper, over an area representing the projected screen. In this case camera movements are not being used, so the sketched size represents the actual size, if camera movements are used the paper represents the actions-surface, and the rectangle representing the screen boundaries may be any size and occurs anywhere over the surface. Of course there is no real limit to the size of the action surface, except the size of the DMAC table. The movements can be represented in shorthand form by sketching the size of the images' boundary and its position, or by drawing a line to indicate the height and position of an images boundary. The illustration shows the planned animation for the same piece of timing shown above.

The work so far is very similar to the plannjng of a normal animated film. The notation used is personal, but the techniques of expression play a significant part in development of ideas, since the better the techniques, the easier it is to describe and specify what you mean.

Now that the description of the animation has reached this degree of detail, it can be re-coded into the form required by the program. The animation programs are written in FORTRAN, and the animation description is prepared as data which is read and interpreted by these programs. A short description of the principles of their working is in order.

IMAGE DESCRIPTION

Areas are described by their outlines, and all lines are described by a series of connected points. Each point is described by the two numbers which give its coordinate position. Therefore, for our purposes all images can be represehted by one dimensional arrays of X and Y coordinates. These may be generated by a subroutine, encoded by hand from images drawn on graph paper, or digitised on the DMAC pencil follower or equivalent machine.

Functions may be used to alter the data and distort an image, producing similar effects to distorting mirrors:-

y = y0 + sine( (n*pi*x0)/max ) * m

for instance, where max is the largest coordinate and therefore x0/max varies between 0 and 1, and the sine value varies between 1.0 and -1.0 n times over the range of x0 values. The amplitude m controls the effect of the sinewave on the y coordinate. The overall effect being to add a sinewave distortion of amplitude m and frequency n to the y axis. A similar function could be applied to the x axis, or alternatively:-

x = x0 + sine( (n*pi*x0)/max ) * m

would have the effect of applying a compression and expansion effect of frequency n and amplitude m. Another function would be:-

x = x0 * f(x0)/f(max)

where the function f( ) could be the square, cube, square root, log or any other function of x0 and max. The ratio between them would vary between 0.0 and 1.0 (all x0 values are positive) and the effect, if the square was used, would be to pack all the lower coordinate values down further, squashing the image over towards the axis. The reverse effect would be achieved by:-

x = x0 * (f(max)-f(max-x0))/f(max)

ALTERING THE SIZE, POSITION AND ORIENTATION OF AN IMAGE

When an image is being used in a film, it is necessary to change its size, position and orientation from the basic position as digitised or generated, to the position required on the screen. The new position x, y is computed from the basic position x0, y0 by solving the equations:-

x = xt + x0 * s * cos (a) - y0 * s * sine(a) 
y = yt + x0 * s * sine(a) + y0 * s * cos (a) 

where xt, yt is the new position, a is the angle of rotation in radians, and s is the scale factor.

KEYFRAMES AND IN-BETWEENING

The animation of an image may be a sequence of repositionings, rescalings and rotations. Much animation also consists of transforming one image into another which is quite different, for instance two different views of a face, or two sets of lettering. The data-sets for the two images will normally differ in the number of lines or outlines, and in the number of points in each outline or line. An algorithm has been developed which alters the data-sets so that they correspond, without altering the appearance of the images. The algorithm is too complex to describe here however. After the two data-sets have been processed, each point in one image has a corresponding point in the other. If x0, y0 is a point in one image, and x1, y1 is the corresponding point in the other, the image at time t when the other two occur at times t0 and t1 respectively, can be expressed by:-

x = x0 + (x1-x0) * (t1-t) / (t1-t0)
y = y0 + (y1-y0) * (t1-t) / (t1-t0)

To avoid re-computing the difference between each pair of coordinates, it is stored as xi and yi, and the time ratio is stored as the variable tc. The way the image coordinates move from one position to the other can be varied in a number of ways.

TYPES OF MOVEMENT

The time ratio tc can be used to produce an indirect ratio to provide a range of movements besides the normal linear one:-

r = 0.5 *  (1.0 - cos(pi * tc))

for instance produces what is called a faired movement, accelerating at the start and deccelerating at the end. By having a choice of such functions, different types of movement can be specified, and the basic movement equations now look like:-

x = x0 + xi * f (tc)  
y = y0 + yi * f (tc) 

MOVEMENT PATHS

By applying different functions to the x and the y axes curved movement paths can be specified. Elliptical and parabolic paths can be chosen by selecting the appropriate fx(tc) and fy(tc) functions. The current image, at time t with movement type f(tc) and movement path fx(tc) and fy(tc) can be computed by:-

x = x0 + xi * f(tc) * fx(tc)  
y = y0 + yi * f(tc) * fy(tc) 

ANIMATED DISTORTIONS

Sine wave effects, tumbling, turning, spinning, zooming and so on are all applied by selecting the appropriate function, singly or in multiples, and specifying the starting and finishing values of the control parameters. The movement paths and types also apply to the in-between values for the parameters, the equations having the by now familiar look:-

scale for x:-     sx = sx0 + (sx1-sx0) * f(tc)      
scale for y:-     sy = sy0 + (sy1-sy0) * f(tc)   
angle      :-     a  = a0  + (a1-a0) * f(tc)   
x position:-      xt = xt0 + (xt1-xt0) * f(tc) * fx(tc)   
y position:-      yt = yto + (yt1-yt0) * f(tc) * fx(tc)   

and so on, appropriate to whatever function is being used. The processing of the data for the image is done function by function in the order specified.

DEPENDENT SEQUENCES

The position, scale and orientation of an image may depend on, or be related to the scale, position and orientation of another image, for instance the eyes in a face. To avoid having to specify all the same movements for the eyes as for the face, in addition to the eyes' own movements, the eye sequence may be made dependent on the face; the animation description is thereby considerably simplified, if at the expense of complicating the processing.

When the data is processed, the data for the eyes will be modified by the functions and parameters specified for the face sequence, before the data is further modified by those specified for the eyes.

SUMMARY OF PROCESSING

The images for a given frame are computed for a given sequence by firstly solving the equations for the intermediary values of the image transformations, then solving the equations to compute the scaling, rotation and translation and any other function specified firstly for the dominant sequence, if any, then for this sequence. The outlines are then scanned-in by the scan-shading algorithm, taking into account the hiding of one area by another, and the resultant image output at the specified intensity level, together with any line images. Each sequence is processed until all have been completed and the next frame can be started. The processing required is carefully optimised, but it is interesting to note that almost 99% of the processing involves simple arithmentic operations upon arrays of numbers. The main other time consuming operation is the output to magnetic tape of instructions to the plotter to draw a line.

DESCRIBING THE ANIMATION TO THE MACHINE

Some techniques for making detailed descriptions of the animation, and the computational approach to the production of animated film have now been discussed. All that now remains is to link the two, and translate the visual and verbal animation description into a numerical and verbal form which the programs can read in as data. Currently, a fixed format verbal and numeric system on data-cards is used; but a more man-oriented system is proposed which makes use of the digitiser to enter all the image and animation information. When the program reads in the data all the keywords are substituted by option numbers, and default numbers assigned to missing variables and options so that each data-sentence becomes standardised, with option numbers and data in fixed formats. The data is switched, by the effect of the option numbers on computed GOTO statements, to be processed by one subroutine after another; passing through successive generations of generality, from complex initial organisation to simple output data sequences.

The animation description falls into two distinct parts, the creation of the library of images, and the animation sequences using those images.

THE LIBRARY OF IMAGES

The coordinate data for the images can be produced on the digitiser, encoded from drawings on graph paper, or generated by a subroutine. In the first two cases the data is stored directly on cards, or in a card file, while subroutine parameters only are stored on the cards, or card file for the third type. Each entry to the library is given a number by which it can be reaccessed. Each separate element of a drawing, and each colour is entered individually as a separate library entry, and assembled into one image by specifying the group option, followed by the code numbers of each element. Once a basic image has been specified, other versions of it can be created and stored as separate library entries, it can be repositioned or distorted by specifying one or more of the functions discussed earlier, and including the appropriate parameters.

THE ANIMATION DESCRIPTION

The key images which appear on the action sheets can be translated directly into key image statements by stating the number of frames from this key to the next, the library image number to be used, the movement type, the movement path, and the type of repositioning or distortion function to be used; merely a matter of translating what is already shown on the action sheet into code numbers or words. The main difference is the ability to define a Cycle. This is any repeating sequence of key images, or the continuous animation of one key image; the keys to be used are specified first, and they are followed by a continuous stream of data for the chosen animation function. Each sequence is described separately, but the relationships between them in terms of cel level and dependence must be specified. Normally sequences are arranged to lie one behind the other, the ones in front hiding the ones behind; but a sequence can be specified as being transparent, in which case all images passing behind will be visible through it with colours combining additively.

CONCERNING AREAS OF COLOUR

The colour areas for an image are digitised as outlines and entered into the library one by one, as has been described. The scan-shading algorithm fills in an area by applying an off-on switching rule. It assumes that it starts outside a shape, and turns the line on when it crosses a boundary, and off when it crosses another boundary. The rule is that there are always an even number of boundaries. There are many possible cases when an uneven number may occur, but these are tested for by the algorithm so that incongruities do not occur. An interesting point is the effect of combining outlines, since the overlapping portion becomes an outside'. The effect of overlapping two comb shaped outlines would be to create a chequerboard look.

Colour areas can be grouped in two ways; the first being the former when they are of the same colour, or else default to the colour of the first outline in the list, and overlapping areas produce chequerboarding; the second being the combining of separate colour areas and elements to produce a complete picture or image, colours are arranged hierarchically in the list, areas with the same colour are blended if they overlap, colours which occur first in the list hide any overlapping portion of colour areas later in the list. This is useful when assembling a complex image like a figure, with independently articulated arms, legs, hands, face and so on, because the joints between the arms and the body will not show as a result. It is a matter of choice whether the figure is assembled at this stage in the library, the key positions of the walk action might be usefully stored in the library for instance; or the separate elements of the body might be handled as separate sequences, linked by dependences in the animation description.

COMPUTER TIME REQUIREMENTS

This work is batch-processing oriented; animation is normally non-interactive, therefore the lack of immediate visual feedback is not inhibiting to the animator; the chief aim of these techniques being to reduce the number of man-hours spent on the production. The programs can process fairly complex scanned images and do a useful amount of animation using five minutes of 1906A time. The bottleneck at present being the plotter, which is not designed to do work to the accuracy required, nor to handle the volume of data which these programs can produce. The product of a five minute computer run can keep the plotter busy for several hours. However, by viewing the data tapes on the PDP15, and taking advantage of slack off-peak usage of the machine, a time gap of less than half an hour between describing an animation sequence; the work for which may have taken one several hours; and seeing the result on the screen may elapse.

THE STATE OF THE SYSTEM

The work on developing these ideas has been done jointly by Alan Kitching and Colin Emmett, as a result of the Royal College of Art project on techniques and applications of computer animation, and of the Count Sawtooth film project supported by the British Film Institute. Use of the systems is not possible by the general public, but is an administrative problem which has to be individually considered for each applicant; the main problem being that it is under continuous development and any user would require the close support of one of us. However it is hoped that those people engaged in, or contemplating the development of their own system might find the ideas expressed here useful. The systems are currently implemented on the 1906A at the Atlas Computer Laboratory at Chilton, where they have an SD4020 microfilm plotter, a PDP15 with VT15 display and a DMAC digitiser for graphics work.

Appendix: THE TRANSFORMATION PROGRAM

Colin Emmett

This article appeared in the Computer Arts Society Page16 Bulletin, June 1971. It is attached here as a good description of the inbetweening procedure used in the Antics system.

I first used a graph plotter at Time Sharing Ltd., London, to draw the images generated by this program. Once the program was working properly I was able to output the images onto a Tektronix storage tube display; from which the illustrations printed here were photographed. I am still working on the Fortran version which will output the images onto magnetic tapes to be run on a Stromberg-Carlson 4020 microfilm plotter at the Atlas Company Laboratory at Chilton, from which an animated film of these transformations will be easily produced.

This program is a model of a world where everything is equivalent to everything else; it will transform any line drawing into any other. Time here is a series of discrete steps, each section of time having its own image, which is drawn onto a frame of 16mm film, so that it can be projected as a normal film. All objects in this world are collections of straight lines, or vectors, which are drawn sequentially by the graph-plotter, using two types of line, visible ones with the pen down, and invisible ones with the pen up. (In the case of the microfilm plotter, the invisible lines are just ignored, and it goes onto the beginning of the next line.)

Having defined this rather limited world, we see that each drawing is very similar, being a sequence of two types of vector. The only difference is that one drawing may have more or less of each type than the other. The program I have been involved in developing has three different strategies for equalising the number of points in the two drawings to be transformed. The idea is that the different solutions arrived at can be compared and judged according to whether they look right or not.

The first method implemented was called MAPPER. This method sees two strings of vectors, which are shape 1, the start, and shape 2, the goal. It takes the larger string and calls it BIG, and the other is SMALL. Now supposing BIG has 300 drawn vectors, and SMALL 230, in order to relate BIG to SMALL we need another 70 vectors in SMALL. So we divide the first 70 vectors in SMALL by 2. If BIG was 500 we would divide the first 30 vectors in SMALL by 3, and the remaining 200 by 2. But what about the undrawn vectors which we have ignored so far? These can be anywhere in the sequence, so each time one turns up, the point counters in the other shape are advanced so as to provide a line of zero length to correspond to it. We now have two new arrays which look exactly like the old objects when drawn, but which have an equal number of points.

The second method implemented is called GANGER. This differs from MAPPER in that instead of dividing up the vectors, it adds drawn vectors of zero length between the vectors of the smaller shape, in the same proportions as in MAPPER.

As can be appreciated, these two methods impose their own characteristics on the intermediate stages by dividing up the vectors in such an arbitrary and insensitive way, but it does provide some interesting solutions, when unexpected things happen. All the word transformations were done with MAPPER, hence the rather illogical correspondences introduced in some of the examples. There has not yet been time to do any good comparisons of these methods though. To avoid the tendency these methods have to dominate the transformation with their own character, the third method called SHADOW was implemented.

SHADOW works from a quite different point of view. Rather than taking the inequality in number of points into account, it takes the inequality of the drawings themselves. The physical length of the vector strings in the start and goal drawings are computed, and they are then projected onto each other to give two new collections of vectors with a total of shape 1 + shape 2 vectors in each. Each vector has an effect in proportion to its length relative to the overall size of the drawing. Undrawn vectors are ignored when computing the lengths, but are corresponded with zero length vectors in the opposite shape, as in the other methods. This method is much more sensitive to the qualities of the drawings themselves, but takes twice the amount of computer time.

Once the correspondence has been established, the difference between each point on one shape and on the other is calculated for the two co-ordinates (or three co-ordinates}. The generation of the imtermediary stages in the transformation is a trivial matter of taking the number of stages required and adding this fraction of the difference to the co-ordinates of each point on the start shape until the goal is reached.

Two things decided on in advance have a strong effect on the intermediary drawings: the first is the sequence of the vectors, and the second is the overall orientation of the start to the goal. If, for instance, the points in one are numbered from the bottom and going systematically anti-clockwise, and the other starting at the top and going clockwise, then the intermediary drawings will turn inside out as they progress from start to goal.

In the TELCOMP version of this program the operator sits at a teletype and is presented with simple sets of alternatives to help describe shapes and choose options, so it is usable by people unfamiliar with the program or programming. It can be used as a way of building up linear figures by starting with simple drawings and then using intermediary stages as original drawings, and so on. The drawings can be rotated relative to each other and the numbering of the points altered to provide the inside-out effect, or to avoid it. Subroutines for generating circles and polygons are also included.

The FORTRAN version is being re-designed to suit the more remote contact of punched-card input and SC4020 output, but will be able to operate on a larger scale due to the increased speed. The SCRIPT routine, which provided the letters, is being extended to provide a fairly naturalistic handwriting as well.

This program continues to expand very rapidly as new routines are thought of and old ones improved, so it is still very much alive. Only a few of the possibilities have been tried yet, due to lack of time up to present. This program has a lot to reveal.

AUTOBIOGRAPHY RE COMPUTERS

I was born in 1950, while my parents were stationed in Austria, my father being in the British Army. I spent the next seven years, until he retired, in four countries, moving from Austria to Hong Kong then to Norway and finally Germany, each move being interspersed with a short stay in England. I then went to boarding school, firstly in Otford, Kent, then in Woodbridge, Suffolk, and finally took my A levels at Kettering technical college, Northamptonshire. I went to Loughborough college of Art to do my Foundation Course, and was then accepted by St. Martin's College of Art to do a Diploma course in Fine Art, where I am now in my final year.

My interest in programming computers developed directly from the scripts I made, to notate the complex simultaneous movements occuring in the animated films, I was making, at the end of the first year at St. Martin's. These films were designed to simulate the types of change I wish to incorporate into Kinetic structures. Although three four-minute, 8mm, films were made, the scores themselves turned out to be far more valuable, because of the detailed frame by frame breakdown of movement, which was then easy to think of in terms of increments in DO loops and subroutines. My first contact with the computer was during a weekend programming course, organised by the C.A.S. at Time Sharing Ltd. in November '69. Since then various programs have been developed, some for animated film, a couple of text generators, and one for audience participation which I am still developing, as well as using the calculating power of the computer to help with three-dimensional geometry and the detailed dimensions of geodesic structures. The merge program is the most recent effort and is described here. Then there is the Learning Maze which was dreamed up during the C.A.S.'s brainstorming sessions for the Computer '70 theme exhibit; this was a labyrinth of cylindrical rooms, each about nine foot in diameter, entered via three, three-sided, doors pivoted about their centre points. Decisions on which of the two or three doors to enter or leave would be made from information displayed on the doors. The circular structure of the rooms give no direction clues, so it would be very easy to find ones self going in circles. I had hoped to be able to incorporate the computer into the structure so that it controlled the locking and unlocking of the doors, using response made by the participants in their rooms to guide it. A reel of thread wouldn't help you get out of this little maze.

The future looks favourable in that I have a place at the Royal College of Art, to do a two year course leading to a Master's Degree by Project, in the school of Graphic Design. Two or three students are taken, over and above the normal course intake each year, to do a project related to the work of one of the existing research groups within the school of Graphics, though the choice of subject need not be restricted to the areas covered by these groups. I intend to look into the potential for computer graphics as an aid to the imagination and creativity. I am interested in how our perceptions arrange things into classes of same and different, because we often do it in a rather illogical way, and it occurs to me that the computer could be used to make use of the way that two experts disagree in their interpretations of the same thing. To be able to reliably generate images which would cause a different reaction from each expert would provide a direct way of relating those experts to each other. Something like this would not require prior classification of individuals; as each person developed their ideas, so their reactions would tend to change, and this would be easily recognised.

I am going to the U.S.A. for three months this summer, during which time I hope to meet as many of the members of the C.A.S. over there as possible, with a view to exchanging ideas and getting a feeling as to where things might be going, before returning to start on the project at the Royal College of Art.

Computer Arts Society Page26, November 1972

I have been at the Royal College of Art for a year now, and I think that being in a department which is strongly design-orientated has had a directing effect on me. I have specialised in the problems of computer animation, and have been helped in this by having access to the Atlas computer at Chilton with its microfilm plotter. My first film was made at the end of my course at St. Martin's School of Art and was a demonstration of the MERGE program (see PAGE 16, June 1971). The pictures were drawn on the Tektronix storage tube display at Time Sharing Ltd and photographed frame by frame using a cine camera. It was an easy film to make since it only involved images and effects which the programs were designed to do.

I am now more interested in the problems of designing a film, and then using the computer to make it. In the last six months I have made a title sequence which is being used by the Open University for a series of programs on computing and a 2½ minute animated spectacular exaplaining a theory of how the ice ages happened, which is being used in a BBC Horizon program called When Polar Bears Swam in the Thames.

I am currently making a humorous 3 minute film featuring the meeting of two amorous bouncing beans (Bean and Beane), which is intended as a pilot study to develop the programs for a three dimensional 6 minute frolic called The Downfall of Count Sawtooth. I have been lucky to receive a grant from the British Film Institute to cover the film costs of this production. This will enable it to be in full colour and have a decent soundtrack. The result will be a good film which features some extraordinary animation, yet without being recognisably computer animated.

The Fortran programs can handle a large number of parallel sequences, using a key frame and inbetweening process. Sequences of key frames can be generated by a do loop expression in the data, making use of a library of characters set up in the beginning of the data. This data language will soon be usable as a version of PLAD, Roger Saunders' descriptive pictorial language. The images can be drawn to film in a wide range of colours using part of a system being developed by Alan Kitching. Though this is a problem-orientated system it is hoped that as more films are made using it fewer extensions will be needed and it will become a fairly general system.