The Moore School of Electrical Engineering, University of Pennsylvania, entered the field of computer animation in 1967 for the purpose of producing a 30-minute educational film on Electromagnetic Fields and Waves. The resulting three-color-and-sound movie represents the then state-of-the-art in computer animated film making. In his dissertation, Don Deily discussed the new techniques that were developed during the production of this film and suggested possible improvements in the animation process. Part II of Electromagnetic Fields and Waves is now nearly complete, and its production has embodied and extended many of these suggested improvements. The movie-making system now in use is significantly more advanced than that of two years ago.
In order to give a complete picture of how computer animated movies are made at the Moore School, we will first give a brief outline of the hardware involved, followed by an evolutionary overview of our software system. Next, we will give a more detailed look at the software extensions, discussing specific features that we feel make our system particularly effective. And, we will conclude with some remarks on the results of using this system and our goals of the future.
The interactive movie-making system operates on a computer complex consisting of:
The Spectra 70/46 is a virtual memory, paged, time-sharing system. Our installation has 2 million words of virtual memory, eight 590 disk drives (equivalent to IBM 2314), four 9-track tape drives, and a communications controller. The Spectra 70/46 has a (fullword) cycle time of 2.88 microseconds, a floating-point add time of 27.69 microseconds, and a floating-point multiply time of 186.55 microseconds.
Attached to the Spectra's communications controller are 6 teletypes, an RCA video data terminal, and the remote graphics terminal (DEC-338). The graphics terminal is interfaced via a 201B dataphone over a voice-grade, 2400 baud, synchronous, full-duplex communications channel.
The DEC-338 is located remotely from the Spectra 70. The configuration of the display terminal includes:
The large computer handles most of the computation and all of the user's programming; it also contains the communications control software for interface with the DEC-338 and produces the final output tapes for SC4020 production. The display computer contains a fixed program to handle communications, all user inputs (typed, hand-drawn, or function key), and all output from the Spectra 70.
From its beginning in 1967 the purpose of the MOVIES project at the Moore School has been twofold:
The basic software used in producing the first Moore School movie, Electromagnetic Fields and Waves: Part I, was the SCORS package. This was a minimal interface between a FORTRAN program and the SC4020 microfilm recorder. It included the ability to do scaling, text, and line drawing in FORTRAN, but the only form of output was a magnetic tape for the SC4020. With this system, the only way to check the correctness of images was to have the tape processed into film and then view the film.
This form of output was immediately found to be inadequate, since it was both expensive and time-consuming, often introducing more than a week's wait for the film. A quicker output mechanism was introduced in the form of a rough picture on the computer's on-line printer. Using asterisks and periods, a crude outline of images could be seen immediately. Also introduced at this time was the ability to get a CalComp plotter drawing of any image. Using these two intermediate outputs, the programmers on the first movie were able to check the accuracy of their images and frames fairly conveniently and quickly.
Other extensions to the basic SCORS package were also made. A better form of windowing and scaling was added specifically to facilitate movie production. The ability to generate an image once and save the SC4020 code for later use was added. This saving process applied to both static backgrounds and entire sequences. The concept of a two-dimensional virtual camera was introduced and general purpose routines for moving this virtual camera were added. A very fast tape-output buffering program was written to decrease I/O wait time. The first movie was made with the basic SCORS package and these extensions.
During the development of this first movie system, it was obvious that the above extensions to SCORS were insufficient for a truly general movie system and, indeed, were probably insufficient to produce Part II of the Fields and Waves movie. At least four things had to be added to this initial system:
Between the production of the first movie and the production of the second, all four of these objectives were met. Joel Katzen wrote a three-dimensional virtual camera programming system that allows a programmer to define and position bodies in three-space, and to independently define and position a virtual camera in three-space. Working in concert with this programming system is the hidden-line system written by Dan Callahan. This allows for the removal of hidden edges of solid objects in three-space. Finally, routines were written to replace the CalComp output functions with DEC-338 output. This new output consisted of frames displayed on the CRT of the DEC-338 as the Spectra 70 was computing them. Added at the same time was the ability to store these frames on a DEC-tape. This enabled the programmer to view his images again, off-line from the Spectra 70, and without the need for re-computing.
With this last addition, the programmer, for the first time, was able to debug entire scenes without having them made into.film. Now he could be certain not only that his images were correct, but also that the objects had the correct motions: individually, relative to one another, and aesthetically. This kind of debugging had been impossible with either line printer or plotter output. In addition, this debugging with the DEC-338 was faster and cheaper than the plotter output.
Having achieved a powerful computer-animated movie system within the Spectra 70 and successfully interfaced it to the DEC-338, the obvious next step was to convert the 338 from a mere passive output device to a truly interactive computer animation terminal. Several new routines were added to the software on the Spectra 70 to provide a flexible interface to the communications software that already existed. Then a fixed program was written for the DEC-338. This program handles all of the previous output functions of the 338 and how handles the various inputs from the user, too, providing him with a convenient method of interacting with his own program.
The resulting system for making computer-animated movies is very usable: the programmer is given a large and powerful array of software with which to define and manipulate his scenes, and he is given the ability to interact directly with these scenes as they are being produced. We think, that the quality of our second movie and the effective way that it was produced will further demonstrate the viability of this interactive, animation-terminal approach to movie making.
The first major addition to the extended SCORS software used for the first movie was the three-dimensional conceptual camera system. The objectives of this system were to provide a conceptual camera that was a reasonable approximation to its physical counterpart, and to provide a system for its use employing conventional terminology that would be easily learned.
The resulting conceptual camera system allows a user to define figures in three space, and to manipulate the conceptual camera around them. This is done by having one fixed reference coordinate system to which both bodies and camera relate. There are then ten independent body coordinate systems and one camera coordinate system. Any number of objects may be defined within one body system, but only ten completely independent body systems may be used. Objects are defined point-by-point with respect to a particular body system, and the body and camera systems are defined with respect to the fixed reference system.
Having defined his objects and their positions and the camera and its position, the user can now specify both body motions and camera motions. The basic body motions available are XTRANS, YTRANS, ZTRANS, ROTX2Z, ROTY2Z, and ROTY2X. These motions may also be combined to produce more complicated composite motions. The basic camera motions available are those associated with a physical camera: PAN, DOLLY, TILT, CRANE, ZOOM, TRUCK, and ROLL. Again, these motions may be combined to produce composite motions. All of these motion names are the names of FORTRAN subroutines in the conceptual camera system that will produce the desired effect on the bodies and camera.
In addition, several camera subroutines are available to perform often-used special effect motions. These subroutines (motions) are DOLLYF, CRANEF, and TRUCKF, and they are used to translate the camera while it is centered on a particular body system. For example, in an ordinary DOLLY, the camera is simply translated either left or right, while the camera orientation remains unchanged. If, during the DOLLY, the camera is to remain centered on a particular body, it is necessary to do a compensating right or left PAN. This compensating motion is done automatically by calling DOLLYF. A similar compensation is done for the other following motions.
Two other camera motion routines are available: PATH and PATHF. PATH is used to have the camera traverse a particular path through three-space without changing its orientation during the course of the motion. PATHF is the corresponding following motion: the camera travels on the specified path, but the camera orientation is adjusted frame by frame to remain centered on a particular body. There is an analogous routine available for specifying the motion of a body along a particular path: PATHS.
Finally, there is the ability to have all motions be faired. That is, the camera and bodies will not reach the constant velocities associated with PAN, TILT, etc., instantaneously. Rather, they will undergo an initial acceleration and a final deceleration over 16 frames at the start and end. This is an aesthetic consideration that has proven very useful.
The second addition to the extended SCORS package was the ability to do hidden line calculations on bodies in three space. This work is an adaptation of the algorithm of Philippe Loutrel of New York University for the removal of hidden lines of convex polyhedra. It allows a user of the conceptual camera system to specify that his final output frames are to have the hidden lines of individual objects, and the lines hidden by occluding objects, removed. This requires specifying additional information such as vertices and connecting edges for all bodies.
As with the conceptual camera system, the hidden line computations are done entirely in FORTRAN and the implementation is meant to be easy to use by the movie programmer. Early results indicate that it is effective but rather slow. Much of this slowness, however, is attributable to the multiply and access time of the Spectra 70/46 and to the fairly inefficient code produced by early versions of its FORTRAN compiler.
The most recent addition to the computer animation software in the Spectra 70 is the facility for interaction with the DEC-338. This interaction is enabled by a few basic communications-handling routines that are used by higher level subroutines that a movie programmer may call. All of these higher level routines are written in FORTRAN IV, with only the physical-level communications programs written in Spectra 70 Assembly language.
There are two basic types of communications from the Spectra 70 to the DEC-338: character strings and 12-bit binary strings. The character strings are usually messages from the programmer or the system to the operator at the terminal; the binary strings are usually display commands to produce individual movie frames. Either type of message can be initiated by a user's program.
There is also software for messages from the DEC-338 to the Spectra: for interrogating the status of the communications interface and for interpreting messages received. Again, the DEC-338 can send either binary information or character strings. The usual message, however, is character strings to be interpreted either as data or control instructions. Typically, data is entered into a user program through the FORTRAN NAMELIST processor. Standard subroutines convert the information sent from the DEC-338 into proper NAMELIST format and then initiate a NAMELIST READ. Program logic is usually modified by changing the values of control parameters with another NAMELIST input string. This kind of interpretation is just one of the possibilities: any user program may bypass the NAMELIST operations and interpret each character string directly. Thus a user is completely free to structure his input in any way he chooses.
The software system in. the DEC-338 consists of three interconnected programs for doing three separate tasks:
When a user wishes to use the DEC-338 as an interactive animation terminal in conjunction with the Spectra 70, the Interactive Movie Monitor Program (IMMP) is loaded and started. The IMMP is a fixed program designed to allow easy interaction between the movie programmer and his program running in the Spectra. It is capable of handling user inputs from light pen, function keys, and teletype and transmitting them to the Spectra. It will also receive and process all outputs from the Spectra, including messages and display processor instructions.
Control of the IMMP's functions is through the twelve pushbuttons provided with the DEC-338. When started, the IMMP waits for the first user command. This may be an interrogation of the Spectra 70 for a message, an indication that a message will be sent from the DEC-338, or a command to wait for the Spectra to send something. Once communications have been established, the course of interaction is jointly determined by how well the program is performing in the Spectra, how often that program requests an input from the user at the 338, and how often the user at the 338 wishes to interrupt his program to change something. Therefore, the IMMP is not programmed to expect a standard interactive sequence; but, rather, depends on the user's use of the pushbuttons to control its actions.
Several of the IMMP's actions merit additional comment. First, it can selectively store movie frames received from the Spectra onto a DEC-tape. In addition, these frames are stored in a compressed format that allows them to be viewed (via the PLAYBACK program) at speeds up to 24 frames/second and higher. At all times, the user has control over which (if any) of the frames the Spectra sends will be so recorded. The algorithm used in the compression process makes a frame-to-frame comparison of the information contained in successive frames and only stores on tape the differences between the two frames. It has been able to reduce the amount of data needed to reproduce a given frame to 10% of its original size. Typical compressions are from 50-20% of original size.
Another interesting IMMP action is its handling of teletype input and output. Any user messages typed into the IMMP are handled as completely free format: there are no restrictions on what may be typed or where. In addition, there are limited editing features available to modify what has been typed in before it is sent to the Spectra. Output messages from the Spectra can be either displayed on the display scope, or they can be printed on the on-line teletype for a hard copy record of the interaction.
Finally, the IMMP is able to call on the PLAYBACK program while still on-line to the Spectra 70. This enables the user to view an entire sequence of frames at projection rates (or slower) to be sure that the motions he has specified are aesthetically correct, in addition to being spatially correct. He can then return to the IMMP and resume his interaction with his program. This feature has been found to be very useful in debugging long sequences and scenes in a movie.
When the user wishes to utilize the DEC-338 as a movie playback system, the PLAYBACK Program is loaded and started. PLAYBACK is a program that transforms the DEC-338 into a special-purpose movie projector using DEC-tapes as input (film) and the CRT as output (screen). The program can show up to 8 movie scenes from one tape, at speeds from single frame to better than 24 frames a second. PLAYBACK runs stand-alone or may be called as a subprogram to the IMMP in conjunction with the Spectra (Spectra programs remain idle while waiting for the return to the IMMP). Like the IMMP, PLAYBACK depends on the use of pushbuttons to control its actions.
A maximum of 8 individual movie scenes may be stored on a single DEC-tape (stored in compressed form by the IMMP). Each scene may be accessed and played back individually or in sequence starting at a selected scene. Sixteen different frame rates are available; the rates are selected by pushbutton. These rates are (in frames per second): single frame, .05, .1, .2, .33, .5, 1, 2, 4, 5, 8, 10, 20, 24, 33, and tape speed. Tape speed is the upper limit on movie playback rates. At tape speed, the movie frames are projected on the CRT as fast as the tape can transfer them.
The PLAYBACK program was adapted from a scheme implemented by Noel Bernstein; the compression and coding scheme is easily adapted to any computer. Our current system compresses frames by using a word-by-word difference technique. When a sequence of words does not change from one frame to the next, it is not stored on tape. Instead, only sequences of changed words together with their locations and word count are stored on tape. PLAYBACK expands the data on tape by constructing the new display file from the preceding file until a word count and location are encountered. Then, the specific number of changed words are taken from the tape data and added to the new frame display file.
Other control words are also stored on tape. There is a control word and word counter used to repeat a frame several times. There are control words to indicate the end of a frame, the end of a scene, and the end of the movie film. The PLAYBACK algorithm uses these control words for positioning and timing information at run time.
Another interesting feature of PLAYBACK is its ability to automatically trigger a single frame camera by setting panel switches on the DEC-338- At the conclusion of each frame's display, a digital-to-analog converter can be triggered to cause a single frame movie camera to take from 1 to 7 shots of each frame. The frame rate setting of the pushbuttons is used to time out between frames to allow the persistence of the screen to die out.
Finally, PLAYBACK can call on (or return to) the IMMP. Once the user has finished checking his scenes on a movie tape, he can exit to the IMMP to resume or initiate interaction with the Spectra.
The ANIMATOR system, designed by Patti Talbot, is a collection of programs that enables anyone, not just programmers, to produce movies via computer. The system consists of six subprocessors that together allow a user to define and produce a complete movie segment. There is no formal programming language to learn in order to use ANIMATOR. Rather, control of the system and subprocessors is by pushbutton and light pen.
The first function of ANIMATOR is the drawing of pictures. This is done by sketching on the face of the CRT with a light pen and tracking square. Several constraints are available, such as horizontal and vertical line drawing, and lines may be entered through the teletype, if desired. Each drawing may be given a name and stored in a DEC-tape Image library. These library images may then be used in other pictures to form composite pictures, or they may at any time be recalled, changed, and stored again.
The second function of ANIMATOR is the definition of motions. Motions are defined independently of any picture, and may thus be bound later to any picture. The motions currently available are rotation, translation, and zooming. It is also possible to combine the motions in two ways: sequentially and in parallel. Sequential combination results in first one motion being applied for a certain number of frames, and then the next; parallel motions all occur in each frame. A typical parallel motion is that of a wheel rolling: the wheel is both translating and rolling every frame.
Defined motions and pictures may be combined to form movie scenes. A scene consists of one or more pictures and their associated motions. Many scenes may be further combined to form entire movie segments.
Both movie segments and scenes are produceable elements. That is, the user can request that any scene or movie segment he has defined be transmitted to the Spectra 70 for intermediate (debugging) production or final (SC4020) production. This is done by passing the output of the ANIMATOR system to a special IMMP program which transmits it to an acceptor program in the Spectra. This acceptor program then expands the scene and movie segment definitions into actual movie sequences by applying the motions to the pictures a frame at a time.
If the user wishes to see his scene immediately, he requests that the output be sent back to the DEC-338. Then he can use the PLAYBACK program to analyze his results just the way he would if he were running interactively with a FORTRAN program. In order to make changes, however, he returns to the ANIMATOR system instead of communicating with the program running in the Spectra. Thus, the Spectra is not used at all while he is thinking and making changes. After he has made all of the necessary corrections, he then re-initiates communications with the Spectra for another production run.
Several sequences in Part II of the Fields and Waves movie were produced using this system.
The production of Part II of Electromagnetic Fields and Waves has given us considerable experience in the use of the interactive movie making system. This experience makes us believe that the system we have is perhaps the best available for making educational films by computer. Independent of the interaction, the movies system represents an efficient, flexible, and powerful way to make a high-quality film. The addition of the interactive animation terminal greatly simplifies the debugging process and can be useful in the creative process also.
There is, however, room for improvement. First, there is a need for a language developed specifically for movie production. The use of FORTRAN for the specification of images and scenes is clumsy at best. A more natural method, incorporated within a higher level language, would be very desirable.
Second, there is the need for more analog inputs to image and scene specifications. Hand-drawn figures are one example of analog input (which we already have); a second is the description of a motion by an analog device such as a tablet stylus or joy stick. This would eliminate the need for many a torturous time-dependent-function specification in FORTRAN and should be very easy to use.
Finally, it should be possible to carry out every phase of a scene's production, from conception to output tape, from the remote terminal. Except in the limited case of the ANIMATOR system, this is not currently possible with our system: there must be a FORTRAN source program at some point.
The realization of these three improvements is our goal for producing our third movie.
1. Bernstein, Noel: A Compression and Real-Time Movie Playback Scheme for the DEC-338 Computer, Proceedings of the DECUS Fall 1969 Symposium.
2. Deily, Don: Principles for Producing Computer Animated Motion Pictures, Ph.D. Dissertation presented to The Moore School of Electrical Engineering, University of Pennsylvania, Philadelphia, December 1968.
3. Katzen, Joel: A Conceptual Three-Dimensional Camera for Computer Animation, Master's Thesis presented to The Moore School of Electrical Engineering, University of Pennsylvania, Philadelphia, May 1969.
4. Loutrel, Philippe P.: A Solution to the Hidden-Line Problem for Computer-Drawn Polyhedra, IEEE Transactions on Computers, Vol. C-19, No. 3> March 1970.
5. Talbot, Peggy Anne: ANIMATOR - A System for Using the DEC-338 as an Input Terminal for Movie Making, Master's Thesis presented to The Moore School of Electrical Engineering, University of Pennsylvania, Philadelphia, August 1969.