Some of us think that Computer Graphics is the wave of the future and that our electronic gadgets will become standard tools for commerical artists. This is not likely to happen soon or suddenly. Both buyers and sellers of computer graphics must change in significant ways before our prophecy can be fullfilled. The changes will not occur auomatically.
Let us approach the utilization of computer graphics with the thought that however hard we expect it to be it will turn out to be harder. This business is not for those who tire easily because we expect the whole undertaking to be depressingly difficult.
To begin with, people in the computer business always find it difficult to explain to their relatives and neighbors just what they do.
Every once in a while someone rudely asks us what we computer guys actually do with all the money we take from him in the form of taxes and phone bills. He doesn't necessarilv object to our having the money, but he would like to have some feel for what is done with it.
When we explain that computers are great for tasks like matrix inversion and the solution of polynomial equations he admits that he is impressed but he'd like to have something in lay terms - and never mind telling him about accounting and record-keeping. He knows about that because he has been corresponding with a computer at the book club of which he is a member and the impact of computers in that area is alreadv painfully familiar to him.
At this point we not only tell him what we do, we show him.
We make pictures with computers, see? (Figure 1). He looks at our so-called pictures and says that since he is neither an art critic nor a masochist he doesn't have the proper background to judge our work ... and by the way, how much would it cost to have a guy who can draw make a real picture of something?
There is a nagging feeling after one of these encounters that all is not right with the world. If we actually had to do something useful to justify our place in society, those of us who deal with computer graphics would be very hard pressed. If we wish to improve our standing instead of just defending our present position, we face uphill work.
After all, how do we measure the usefulness of something in our society? We ask if somebody will pay money or effort to have whatever it is.
Will people pay for computer generated pictures? Who? Why?
It's no fair counting other people in the computer business although this trade increasingly uses pictorial output. Computer people, like normal human beings, are especially designed to make use of graphic information.
According to somebody who counted: "38% of all fibers entering or leaving the central nervous system are in the optic nerve." It is estimated that as much as 75%, of information entering the brain is from the eyes. [1]
As Scientific American readers know, there is a great deal of evidence to suggest that information processing is done in the eye before data are passed to the brain. A recent estimate indicates that there are about a hundred million sensors in the retina and only five million channels to the brain from the retina. This twenty to one sort of data reduction is apparently not accomplished simply by using every fifth input on a sampling scheme.
For example, to recall McCullough's famous work at MIT, if a frog's eye is isolated from its brain in an experiment that measures output from the eye in response to visual stimuli it turns out that the eye is very selective. If something that looks like a fly is moved past the eye at about one frog's tongue distance, the eye puts out a substantial jolt in response. The eye wll not respond similarly to a list of numbers describing a fly.
Even our Indo-European languages equate the words see and understand. I see, Ya vizhu, je vois. Seeing something is tantamount to understanding it.
We often plot numbers on a graph so we can see what they mean. Programmers often surrender efficency and their native neatness to make their typewriters print out verv bad graphs because even a bad graph provides more meaning usually than a tidy list of numbers.
Some processes are too difficult to visualize mentally and we must turn our numbers into motion pictures to grasp the ideas. We've been involved with Sandia's ACCEL circuit card design program [3] and it's interesting to recall that the first motion picture sequences showing the placement and routing processes were not made to demonstrate the working program after its completion. Thev were used to let the programmers see what the programs were actuallv doing all along the way.
It has been suggested that prime numbers might be plotted graphically in such fashion that we could grasp the visual pattern of their occurrence and learn to predict their place in the number sequence.
Of course the computer business can use computer graphics, but what percentage of our society is in the computer business? A kid who sets out to be an insurance salesman can sell a few policies to his uncles and cousins but sooner or later he has to go out and tackle hostile strangers to justify the draw his company pays him. We manage to draw a good many million dollars from society and it seems we must figure out how to sell our work to hostile strangers.
A small number of people have been triyng to make a commercial venture of computer graphics. The survivors of the effort report vivid impressions of hostile strangers.
When we say computer graphics we tend to put the accent on computer and let the graphics take care of itself. Actually, hostile strangers are only moderately interested in computers. To them a computer is sort of like a head on a pole - nothing they'd deliberately seek out, but interesting as a novelty.
On the other hand our prospects do know a lot about commercial picture making. They are keenly aware of the mechanics and economics of their work. They want to know what sort of thing we can draw, how fast, for what price.
We can draw anything. (Figure. 2)
The actual drawing is done very rapidly and a thousand variations on a given piece of work can be produced with the speed of summer lightning, but the programming and especially the input of data are complex and tedious procedures. Further, the making of pictures is a creative undertaking subject to irrational and unavoidable disagreements. We are unable to predict with confidence the time required to do the work and live through all the arguments over taste.
Similarly we cannot predict the price. Any number we choose is really a blind stab.
We have tried to get around these sordid realities by deciding that we are actually fine artists as opposed to commercial artists. Fine art speaks for itself. Commercial art has to be good for something.
Like everyone else in this field we were equipped with a huge collection of randomly derived geometric forms. They are quite interesting the first few hundred times through. It isn't that we were so crazy about geometric forms, but the computer likes to draw them and it jolly well won't draw anything else without a large investment in time and money.
With op art at the height of its popularity we looked around for somebody who appeared to like randomly derived geometric forms. The big time art people who run the museums were interested in the material we offered and asked what artist had done the work. When we explained that no one who could quite be described as an artist had done the work, only a programmer who set some policies for the computer, there was a sense of coolness in the air.
It was regretful that an artist had not infused our work with artistic value. They named some artists who might lend grace to our poor efforts.
The recognized artists we talked to were hearty, pleasant chaps who were not fools. They could tell that we harbored a cynical attitude toward that sort of art of which op is a sub class. They didn't care to be used by us. We did not wish to use them.
We might whip up more enthusiasm if we thought there were money in fine art, but how many non-objective artists is the world willing to support? When we consult standard statistical references like the World Almanac we do not find any figures dealing with fine art, no indication of the annual yearly business of art galleries, or the number of people who list their profession as fine artist.
On the other hand, there are figures dealing with advertising expenditures, with television programming, with various sorts of things that depend upon the creation of pictures. One learns, for example that the annual billing of an ad agency like Leo Burnett Company is approximately $250 million. That is $37 million more than a published estimate of the annual budget of Sandia Corporation. [4] Burnett is not the largest agency. It's only about fifth. Sandia is certainly the largest atomic bomb manufacturer in this country.
Presumably one can find out about the economics of fine art but the lack of publicity attending the numbers does not suggest that we can win the hearts and the cash of our countrymen by automating Piet Mondrian ... even in three dimensions.
Fine artists will have an opportunity to use our instruments and their creative help will be stimulating and provocative. On the other hand we have not been able to break down any barriers by acting as if our randomly derived geometric forms are anything but interesting exercises. If they speak for themselves, they speak an unknown tongue.
We really have little to show, collectively, in this field. There is some variety and there are some spectacularly beautiful things. Some of the fine figure work at Boeing and some of Whitney's handsome work come to mind. The trouble is that we have a little man who couldn't draw better, faster, and cheaper.
On that basis it is very difficult to approach people who normally make good pictures that fit their purposes well. In that case we are inviting them to take a fling on research and development with our talent and their money. We can weave dreams for them, but we don't know how to perform to their specifications. Indeed, they can't imagine how to specify anything for us.
Obviously, we can use our techniques in the long run for things like package design.
A designer could feed the basic elements of a new package into the computer and get back five hundred variations on a basic theme overnight. He could paste up a few selected treatments right from the hard copy and deliver a virtually complete job to his client along with a bill.
Many such commercial applications will be practical once we have appropriate software, I/0 hardware, and the right attitude.
Meanwhile, there is one area in which we are just barely able to do something of value. We have in fact sold computer animation to producers of television commercials.
We started some years ago with jittery sequences of geometric nonsense and bit by bit - literally we have worked up to smoothly flowing, shimmering, full color sequences of geometric nonsense.
Television network executives look at it and say: Whatever would we do with anything like that?
Hollywood producers say: You show me an animated character emoting and I'll get you millions of dollars. By then, who needs it?
When we showed black and white material to ad agencies they said: It certainly would be better in color.
When we show them color footage they say: It certainly is hard to judge the basic art with all that distracting color.
One producer was putting together a series of psychedelic backgrounds for use behind musical groups on television.
He likes our colorful geometric nonsense and offered to pay a dollar a foot for specially prepared material. That's sixteen frames per 35mm foot. A three minute sequence would pay us $270. If that footage cost ten cents a frame, total, including postage for sending the bill, we'd lose $162 on the deal. Still, he was a live customer offering genuine cash money. It happens that he is just as happy with out-of-focus pictures of splattering ink as he is with our expensively wrought material. So are most people: Besides, ten cents a frame is not enough by a factor of twenty or fifty or eighty, depending.
We are left searching for wealthy customers who want to buy very carefully controlled, dramatically complex animation effects.
There is one such market.
When we worked in the television commercial business typical one minute spot cost seven or eight thousand dollars to produce - about five dollars a frame.
That seemed like rather expensive production even in Hollywood a few years ago.
Old friends in the business tell us now that a typical full blown spot for a national sponsor costs eighteen thousand or so dollars and it is not unusual to spend thirty or forty thousand dollars - twenty dollars a frame. That's enough to pay for some careful computer work ... not too much, because most of that money is carefully earmarked. Still, it beats trying to swindle an artist out of his fellowship money.
People argue the point, but we feel that the television commercial is the great American Art Form - native grown, governed by stringent disciplines of time and purpose - subject to fast feedback from a responsive audience of a hundred million people. There is more creative innovation in television commercial design, in commercial music, in the deft handling of language than there is in films that are designed for entertainment. We may quarrel about specific pieces of work, but notice that your kids will run to watch a commercial embedded in a program they have been ignoring.
Our highly stylized computer animation is acceptable and useful in television commercials.
Since we grasp the economics and the mechanics of both film making and computer animation we assumed that we could head off most of the problems and interpret adequately between the film people and the computer people in doing commercial work on contract.
We learned a lot that was not intuitively obvious. One thing was not a surprise. The man in the middle gets a lashing from both sides. The surprise was the number of subjects that bring on trouble.
There is a difference of viewpoints that makes it hard for either side to tolerate the obstinacy and irrationality of the other. Since we in the computer business are trying to sell something to the film people, it seems that we must flex, not they - at least in the short run.
Let us consider some conventions in the motion picture business and brood a bit about the ways in which our computer material varies from those conventions.
A professional motion picture is usually composed (Figure. 3) of rectangular images 3 × 4 in aspect ratio arranged at regular intervals along a strip of sprocketed 35mm film. Sixty years of working with international standards have established these specifications rather firmly.
Computers put out pictures by two standard means, through an oscilloscope tube, thence to either film or sensitized paper that is processed to produce hard copy, or through a stylus that draws an image directly on paper.
Since oscilloscope faces tend to be round, efficient practice suggests that the image area utilized on the scope face should be the largest practical square. The typical field is 1024 by 1024 points. This is not only practical, but 210 is a number of great power among mathematicians. The product of this mystical rationale is a square picture.
A chap at Eastman Kodak did a study almost forty years ago in which he measured 250 paintings by fifty famous artists. These were pictures that people were willing to pay good money for and to hang on their walls.
The plot of width to height of these pictures looked like this: (Figure 4). Square pictures were very scarce.
Another researcher wrote during the time when sound movies were just coming into use:
"We purposefully use the expression rebellion because for the first time in motion picture history a number of exhibitors and at least one of the greatest exhibiting and distributing organizations in America have taken matters into their own hands and have reduced the height of the projector aperture. They have considered it essential to maintain the rectangular form of the screen even at the risk of cutting off parts of the performers heads or some detail of the lower part of the general scheme of composition. Such procedure is rebellion and what is more important, it is seemingly justifiable. [6]
(Figure 5) The situation was this: the 3 × 4 aspect ratio had been established early in the movie game because people liked it that way. When sound came in it was clear that the track had to be put somewhere so it was printed inside the sprocket holes in what had been picture area and the picture area was chopped off to make room, leaving a square image. It was efficient and practical and people hated it.
(Figure 6) The present so-called Academy Aperture was a compromise arranged by the Motion Picture Academy to meet this problem. It is inefficient and reduces by an appreciable fraction the possible resolution of the picture on the screen, but people like it.
At least thirty years after this drama when we first needed some computer generated footage, the messenger brought back from the data center a roll of randomly spaced square images filling from side to side a piece of non-sprocketed 16 mm film. Run that through your projector! We had to shoot more nearly standard film footage from hard copy of this stuff.
This brings up another quaint custom of the computer business. It seems a shame to make an issue of the fact that this industry at the very forefront of technology is using antiquated methods, but the fact is that scrolls have been largely out of use in civilized society for some hundreds of years. Computers tend to print out scrolls.
Machines may handle scrolls comfortably but people don't. Further, machine operators aren't accustomed to really long runs of hard copy. During our two thousand frame runs the machines gradually ran out of developer and the last few hundred frames Were decorated with orange-brown streaks that wove strangely through the movies.
No matter, we can now put the images on regular movie film but it would never occur to a producer to specify sprocketed film and it does not necessarily occur to computer people that it matters.
We use the scrolls only in short test runs of some material and calm handling of the paper solves all problems. Still, there is little calm in the motion picture business and the customer always winds up with crushed and torn artwork somehow. It seems a pity.
Don't accept the simple thought that standard cameras taking pictures off the scope solve all the problems. There's still that square picture to contend with and the computer readout equipment is in general completely inflexible. The camera is locked into an electronics rack over the tube and it stares fixedly at the square image. If you're like us you don't own the equipment and you're in no position to modify what they have at the data center.
(Figure 7) Suppose you want the camera to see only that portion of the square that makes a 3 × 4 image and you can program so that all the action takes place inside that rectangle.
You call the man with the readout machine and say: Can I position the camera so that each standard rectangular frame of motion picture film is filled from side to side with the scope picture?
Yes he says, the image is normally 17 × 17mm but it can be expanded to fill the frame from side to side.
Good!
We can expand it to 19mm, says he.
Since a normal frame is about 25mm wide you can't fill it with 19mm of picture and you point this out.
(Figure 8) After an airplane ride he didn't want you to find out that his idea of side to side is your idea of top to bottom. He thinks of the film as running horizontally as it does in a 35mm still camera.
Crying relieves your feelings, but it doesn't modify the hardware for you.
You carve your 3 × 4 area out of the available square and photgraph it. This gives you a small rectangle of picture the proper shape somewhere in the normal film field.
The producer who is paying you will have to blow the image up to fill the frame. He resists the thought and expense and you tell him about 218 and how mathematicians are fond of it. He tells you to move the camera and you re-live the whole scene at the center.
Of course, the producer has to go to the optical house with his film anyway because you are supplying him with three separate strips of high contrast black and white microfilm. (Figure. 9)
These three strips must be printed in register, each with a different color filter, to produce a final color negative. It's just like color separation work in printing. Dreary technical concerns prevent production of fully saturated color pictures right from the scope onto color stock.
The producer is also horrified to hear that the picture we produce for him is centered on the film. The Academy aperture to which he must convert this stuff is not centered on the film. It is off to one side, leaving room for the sound track the way it is supposed to.
Not only that, the producer must rely on the computer people to generate designs that allow each of the three strips to be approximately uniform in density. That is to say: very fine lines on the negative can stand to have a lot of light pumped through them for reproduction on prints. Too little light will leave weak, broken lines with poor color saturation. On the other hand, large areas of solid color will bloom on the print if too much light is used in printing. If a given frame contains both fine lines and large areas the printer cannot balance for either and the product is bad.
Further, if the producer does not want the colors to blend at the points where lines intersect, he must make traveling mattes - a process too complex to cover here, but fussy in the extreme and especially difficult with multitudes of fine lines.
If our irregular film product is to be made useful to the buyer, we must anticipate a great many little problems. Some unexpected things crop up.
Normal motion picture film is numbered along the edge so that the original film can be matched foot for foot with prints made from it simply by matching numbers.
The microfilm stock normally used in computer systems is not edge-numbered. Nobody thinks about this until the editor bursts into tears after days of working with footage he cannot make head nor tail of.
We've been talking about purely mechanical matters. Consider creative problems.
A computer can do things a human animator cannot do. The computer can move each of ten thousand lines from frame to frame without fatigue. The human animator gags at the thought.
He not only can't do it practically. He can't imagine why he would want to. It is utterly outside his experience. If he designs action he does understand he wastes the capabilities of the computer. Anything he really understands can be done better and cheaper by hand.
He can't figure out what to do with hundreds of units of animation. He can't imagine what such stuff would look like, what to ask for. He does get a warm glow when we tell him that the computer will cheerfully draw action that is tough for human animators. It will move active figures toward and away from the camera and maintain the sort of perspective variations that occur when real objects are photographed.
We're not talking about regular forms that appear to move near or far simply by changing size. We're talking about the kind of distortion you get with a wide angle lens. (Figure 10) Such a lens enlarges closer parts of the figure more than distant parts. It is easy for an animator to show a figure walking left to right across the field at a given distance. It is much harder to make that figure look right when it walks toward the camera. The computer can handle that sort of thing well by formula.
The animator senses the power of the tool we offer when he thinks about this sort of thing, but he still doesn't know how to use it.
He's not about to learn for a given commercial job.
The producer suggests that he send his animation director out to work with us at the computer so he can see what actually happens to the rough ideas they eventually pull together.
He is disappointed when we explain that the computer is a smashingly uninteresting box that lives in a room of its own behind an iron fence somewhere. Even if we do have access to the computer it does nothing more dramatic than blow hot air on your shoe while it hums and soaks up money.
The computer output is invisible, just a lot of magnetic signals on tape that must be transported to the printout system somewhere else.
Fine, he'll have his man watch the printout machine.
No such luck, the thing is sealed tight and the outside of the box doesn't have the visual appeal of a Brownie Reflex.
He can see some test pictures, stills drawn on scrolls, that show him what the computer thinks it is supposed to be drawing, but he can't see test runs of animation unless he wants to pay a fortune for the privilege.
Normally his animators draw the material roughly in pencil and photograph that directly in what is logically known as a pencil test. It gives him a feel for the action and allows correction at an inexpensive level.
It's all or nothing with the computer. Onlv faith and hard liquor allow the customer to hold his peace while he waits to see the finished product.
In spite of all, the struggle has been worthwhile in practice. The effects are so striking, especially the effects of bugs, that customers do come back for more.
The point that seizes our attention is really that the computer is completely wasted if it is used to produce material that is already well understood and is under full control. We have a brand new medium and none of us quite knows what to do with it. This is not without historical parallel in the computer field or in the motion picture field.
Each of us has sometime been through a discussion with people who are convinced that a computer is simply the equivalent of ten thousand men with adding machines. They grant the idea is impractical but they feel we are begging the question when we insist that computers are really more than that. Well, maybe, but we were not able to predict from our knowledge of people with adding machines many of the things we now do with computers. Our viewpoint has changed with experience.
With respect to graphics, we suspect that the computer is more than the equivalent of ten thousand men at drawing boards. We must assume that the future holds surprises for us.
When motion pictures were first developed - if you'll pardon the pun - the mere technical fact of being able to record and reproduce moving images was stunning. It struck people that they could record great actors in great plays and carry the film off to places where people had never seen fine drama. Culture could be taken to the people.
Indeed, they put cameras down in the audience and recorded plays for the enrichment of mankind.
It turned out that motion pictures were no good for what they had in mind. If you plop a camera down and let it watch a play you find that the camera doesn't see a play the way a human being does. It is non-selective. When a human being watches what the camera has recorded he is bored to tears because he cannot escape the camera's viewpoint.
Every motion picture technique we now take for granted was at one time a startling innovation.
Everybody knows full well that one image following another on film follows in time as well as in position on the film. Still, if you see a burning house in one scene, then action at a firehouse, then a picture of a woman picking up a baby in a smoky room, you assume that these pieces of action are related and may be simultaneous in time. How do you know?
The first motion picture to include edited action of this sort was verv creative in the eyes of the people who saw it. Although roughly similar action in stage plays was common the effect was not quite the same. The camera could be moved and it was a surprise that story continuity could be maintained - even enhanced - in spite of abrupt changes in viewpoint. Motion picture technology was wasted if it was used only to record material that people already understood and could comfortably control.
Motion picture techniques developed rapidly because the technology was rather simple. Anybody could operate a camera and even process his own film with a bit of experience. Of course there was a big hassle about standards. Further, stage directors and actors did not all readily adapt to the limitations and new opportunities inherent in making movies. A whole new class of creative craftsmen had to grow up in the film business.
Now here we are with a new medium - computer generated pictures. Our technology is not simple. Our equipment is not designed so that anybody can handle it and innovate in the new medium. Even timesharing systems with light pens and memory drums and CRT displays allow laymen to do only such crude work at great expense that we do not see these systems as the key to general progress. Outsiders with fresh attitudes cannot easily join our club.
We are doing something about the input bottleneck. We're starting to beat the problem of teaching the computer what to draw. Till now the mere input chore was more costly than the programming and computer cost. It isn't that the programming is easy. It's just that after years of learning what to do to make things move the way we want, the big task in any given production job is to ram the picture data into the miserable machine. We are improving that and it will help a lot.
Does it matter to us how the computer graphics revolution will occur if we are so confident that it will apart from commercial considerations?
Well, there's that nagging question about what we have been doing with all that money society spends on our recommendation. Can we produce anything people want to buy?
So far, nobody can tell what we have to offer and as long as we speak gibberish and put out our work in forms nobody can use, hardly anybody will trouble himself to find out what we have to offer. Their enthusiasm for paying us may evaporate.
There is the happy possibility that a whole new class of creative craftsmen will grow up among us. If we actually follow standard practices in the graphics field and speak something resembling English or Russian or Chinese and we develop hardware that anybody can use without five years of intensive training - then the hostile strangers may actually buy things from us, whatever those things turn out to be.
There's an old story about a man who presented a new idea and was asked what it was good for. He floundered for a while and said: I don't know, but what good is a baby?
If we help our computer graphics baby to grow up we'll find out what it's good for.
(1) F R SIAS JR The eye as a coding mechanism. Medical Electronic News.
(2) G BIERNSON A feedback-control model of human vision. IEEE Proceedings June 1966.
(3) C J FISK D L CASKEY L E WEST ACCEL Automated circuit card etching layout. IEEE Proceedings November 1967.
(4) Article re Sandia Corporation. Business Week March 1968.
(5) L A JONES Rectangular proportions in pictorial composition. SMPTE Journal #14 Jan-Jun 1930.
(6) A S HOWELL J A DUBRAY Some practical aspects and recommendations on wide film standards. SMPTE Journal #14 Jan-Jun 1930.