Slashdot Mirror

Here are 10+ plus answers to Slashdot questions from motion picture digital effects expert Thad Beier. He chose the additional questions himself. (Yes, he's on Slashdot almost every day; we asked him to do the interview after reading many intelligent comments he's posted.) Anyway, there's some fine insight into the intersection of moviemaking, graphic arts, and computer science here, brought to you by an award-winning member of the film industry who just happens to be a fellow Slashdot reader.

Are 'FX programming' days numbered?
by Anonvmous Coward
Every year, 3D packages get more and more sophsticated. Not just in terms of rendering effects, but in their scripting capabilities as well. Do you see a day where the artist will be able to handle the rendering features and the scripting of a 3D prog so well that it'll no longer be necessary to have a dedicated programmer on board?

Is there a particular type of problem that will always need a programmer?

Thad:
First, I feel that the difference between 'scripting' and 'programming' is nonexistent; both are programming, albeit in different languages with different development environments. People can, and do, write thousand-line MEL scripts for Maya -- which are every bit as complex as anything written in C. With each new animation system, the scripting languages become more powerful, and subsume larger modules as primitives within the language -- this should allow non-programmers (or, more realistically, people who don't consider themselves programmers) to create significant custom systems with reasonable short scripts.

Secondly, though, I feel that there will always be a need in movies for people who are predominantly programmers. Films have to compete with each other and with the library of pre-existing films, and one way that is done is by continually pushing the state of the art. A consistent request from filmmakers is for 'something nobody has seen before'. Often that means creating custom tools; or building scripts and shaders far beyond the capabilities of non-programmers.

It is true that as time has gone on, the percentage of people on visual effects teams who consider themselves primarily programmers has fallen. One reason is that when doing 500 shots of a mouse for Stuart Little you only have to create the mouse once, but you still need hundreds of people to do the artistic tasks of animation, lighting, and compositing. That doesn't mean that the programmers aren't important, they are the key to ensuring that the artists can be productive.

Shaders
by f00Dave
How much overlap is there between the programable graphics processing units (AKA "shaders") found on modern game platforms and the software/hardware used in the special effects industry? Would programming skills for one translate to the other?

BTW, I realize that special effects are half artistry, half mathematics and half sweaty work: kudos from a 'GL hacker... [;-)]

Thad:
I note that some slashdotters have criticized your math, but you have hit upon a fundamental truth of visual effects, that the work takes far more than the available time.

While it is conceivable that there is overlap possible between programming of games hardware and writing shaders for visual effects, I haven't seen too many people making the move from games into FX; mostly it is the other way around. Certainly many people in the games business are clamoring for visual effects and other film artists to help bring cinematic ideas and qualities to the games world.

The interesting new wrinkle in this is the Cg language from Nvidia. It's a new, high-lvel language for writing shaders. Cg is then compiled down to microcode run on the graphics hardware in the machine. While I had been skeptical, now I think that this might dramatically change the way that rendering is done. The work of the visual effects and game shader-writers could be exactly the same. It wouldn't surprise me if future software renderers use graphics hardware to speed up the process.

Cost
by Fembot
When films are labled as "100$ Million on special effects" where does most of that money go? On rendering hardware or what?

Thad:
I don't think that any movies have had $100 Million in special effects, yet -- unless you count Dinosaur or Final Fantasy -- which are animated (as opposed to FX) films. That said, the overwhelming cost on any films for effects at this point goes to the creative people. Especially today, the hardware is virtually free. (In some cases, the hardware is literally free as a company will donate machines in return for good PR.)

A reasonable estimate for the cost is 75% for artists, and 25% for everything else. This has changed dramatically over the digital visual effects era which started around 1990 -- back then it was probably exactly the opposite. But machines have gotten much cheaper and animators have become more expensive, and that trend will probably continue. It's interesting that people talk about how much cheaper Linux PCs are compared to SGI machines (say), but truly both machines have almost the same cost (zero) compared to the cost of the animator who is using the machine. The choice of workstation should be entirely based on what makes the artist most productive.

Directors approach?
by FurryFeet
I'm guessing you get to work pretty closely to directors. If so, can you tell us what is their approach to the new tools technology has given them? Are they still "thinking celluloid" made cheaper by rendering it digitally, or do they really seek to break the mold and make shots that were previously impossible?

Thad:
The job of a movie director is to harness the skills of hundreds of talented, unique, possibly difficult people to create his vision and tell his story. In our experience, directors always request the ideas and proposals from his creative team; and they listen to that advice. The FX team is hired to help make the movie, and are trusted to help make the decisions. In most cases, the director will work very closely with the FX supervisor when shooting the shots that will have effects, asking for help and comment on all aspects of the shot. After the ability to get the most out of his team, though, the most important quality of a director is decisiveness -- once all of the input has been gathered, everybody has to march in the same direction.

Every director we have worked with has been extremely interested in any ideas we could contribute to making shots cheaper, better, easier to shoot, or cheaper. They want to get the best images on film, and any resources saved on one shot can make the next one better.

best way to get into the industry?
by josepha48
What is the best way to get into the computer generated special effects industry? Is it who you know or what you know? If it is what you know what should one know? (Programming, graphics tools, etc...).

Thad:
Well, my first sincere, if unhelpful answer is "Are you sure that you want to?" It isn't really an industry in the traditional sense -- there is little or no job security, there are long hours typically with no overtime paid, the stress can be extreme and the rewards are not great. There are almost no rational reasons to choose CG visual effects as a career. So think about it before making that choice. If it really is the most important thing in the world for you, then read on.

Every person is different, and every position to be filled is different, so any advice given will either be too specific to be generally useful, or so broad as to be a platitude, but I'll do my best. Over the last few years it is my impression that there have been far more applicants trying to get into the field than available jobs; that might just be a cyclical problem or it might be persistent.

A solid undergraduate education is always a good thing. Some basic art experience is helpful, to learn color theory, layout -- basically learn what makes a good image good. Knowledge of mathematics and elementary physics is useful, to know how the world works. General computer experience is helpful, for example the ability to write and understand shell scripts. To get a job at a large facility a familiarity with the most commonly used tools is helpful.

Clearly you would like to have some animation experience. Computer animation is useful, but 2D hand-drawn animation is also an exceptionally good way to learn how to bring images to life.

When preparing a reel of your work, a few great shots is better than a large volume of mediocre work. You want something to make your reel stand out from the rest of them. Play to your strengths; concentrate on what you do best. If the work on your reel includes shots done by a team of people, be certain to call out your particular contribution. A demonstrated ability to work on teams with other creative people is a definite plus.

The Siggraph show every year is a good place to meet recruiters from many companies in a few days. It takes place in late July or early August. This year's conference took place last week, and all of the big companies demonstrated vigorous recruiting efforts. A few companies have great pages to assist people in planning their careers. Here is the employment FAQ from Pixar and the one from PDI. While they are animation companies as opposed to visual effects companies, their advice is still appropriate, by and large.

What movies have impressed you?
by Anonvmous Coward
When somebody has intimate knowledge about how a movie is made, it gets really hard to make their eyes jump out of their head.

For example, there's a scene in the Director's Cut of Robocop where Alex Murphy is just about to be shot in the head by the lead bad dude. The camera is pointing right at Alex's face, then swings around behind him. As soon as the camera is behind him the bad guy fires a gun, the back of Alex's head explodes and you can see a hole clean through it. This whole scene was one smooth camera movement, no edits.

I was *stunned* to find out that Alex was a puppet. They were able to make a puppet that totally convinced me that Peter Weller was sitting in front of this guy about to get his head blown off. I could not believe that they were able to do one that convincing.

I'm curious, what movies have had that affect on you? "OMG! I had no idea that was an effect!"

Thad:
Your example is a classic of FX misdirection. Another one is in 'Spiderman'. We see Peter Parker with his shirt off pretty early in the movie, and he's the scrawny little twerp that he's supposed to be, and you accept it without a second thought. Later, after he's been bitten, he takes off his shirt and he's totally ripped. Not until that point do you say "hey, wait a minute! How did they do that effect!" When, of course, the effect happened in the first shot with a body replacement that you never expected. I was blown away, it was just so cool, and so easy. The best effects are those that you would never expect, and that by the time you realize that they must have been effects they are over.

These days almost every film has FX shots that nobody could possibly see. Our first film was 'Showgirls', and I defy anybody to find the dozen shots we did in that movie -- they are not in-your-face effects. Two of our more recent films, 'The Fast and The Furious', and 'For Love of The Game' were praised in the Los Angeles Times and Variety as films with a refreshing lack of special effects. It's not that they're missing obvious things; it's just that FX can be undetectable.

So, when you say if there's anything where I'd say "I had no idea that was an effect", well, it's certainly true -- but for most of those shots I still don't know that it was an effect.

Project you'd like to tackle?
by seldolivaw
Although recently a lot of the big names in science fiction and fantasy are finally making it onto the screen in a plausible way (e.g. Tolkein) there are still plenty of great books out there that haven't even been optioned. If you could turn any science-fiction/fantasy book or series into a movie, which would it be?

[My personal choice: the Foundation saga by Asimov. So huge! Such a great plot! So eminently filmable! Somebody make this movie, dammit! :-)]

Thad:
Surprisingly, and contrary to your question, classic SF books like The Foundation Trilogy and Ender's Game are always in play; we get scripts or proposals for these every couple of years. You're not the only one that wants to see these books filmed; it's very difficult to do, though. While the tremendous success of The Lord of The Rings is on everybody's mind, don't forget that people have been trying for years to make those books into films with limited success. A good book has such scope and detail that it's hard to distill it into a reasonable-length movie. While I'd love to see a movie made from Stephenson's 'Snowcrash', any reasonable length movie would have to leave out at least half of the stuff that makes the book great.

Short stories are a better bet. The astonishing success of movie versions of Philip K. Dick's short stories would have completely bewildered him, but they are great source material. I'd love to see a John Varley short story -- say, 'The Phantom of Kansas' -- although I admit that 'Millenium', based on the book-length version of his short story 'Air Raid', was perhaps the worst movie I've ever seen.

Reduction in man-hours for CG?
by ceswiedler
At one point, as a film student, I was interested in computer animation as a way for a single person or small group to produce a film, without the expense of locations, casting, cameras, etc. I thought that soon, as hardware and software improved, it would be possible for me to create a film on my own computer at home.

But my experience in animation in college taught me that increasing hardware capacity doesn't reduce the time it takes to produce a film or demo reel; it simply increases the quality of the final output. I imagine that the modelling, animation, and rendering of the scenes in Tron took as much human time as comparable scenes in Fellowship of the Ring. It's possible to render Tron-quality CG in realtime on a modern PC, but nobody wants to watch it.

My question is this: do you think it will ever be possible to produce a full-length CG film in about a man-year or less, with effects which are reasonbly "modern" for the time? Will the technology curve eventually flatten out, once we get to a certain point where the human eye can't really tell the difference? Or is it implausible to think that a single person or small group could provide all of the artistic input (scriptwriting, directing, modelling, animation, acting, etc) to produce a full film, even ignoring all technological constraints?

Thad:
There are movies created by small teams of people; and some of these will be CG generated films. They won't be "Toy Story", though, they'll be motion-capture or cg-puppet films with relatively simple lighting setups; I don't think that you can do high-quality animation quickly, except through some kind of performance capture. There's a sort of Moore's Law at work with state-of-the-art animation where the complexity of scenes doubles every couple of years. Animators always will wait a certain amount of time for their frames to render, on the order of 15 minutes to an hour -- and that time hasn't changed even though computers are 1000 times the speed they were 10 years ago.

Your question has been answered in the affirmative last year by 'Jimmy Neutron, Boy Genius'. That was a relatively small team of people working for just a few man years; and they created an incredibly successful film. Compared to 'Monsters Inc.' it wasn't state-of-the-art, but compared to say 'Rugrats in Paris', you'd have to say that it was.

I think your question about the technology curve flattening out means that you're asking whether at some point the most elaborately specified scene might render in real-time. That it's not inconceivable but it is unlikely. It's possible that computer speed will finally outstrip the ability of an animator to create complexity, so that frames will render that fast; but I think it's more likely that database-amplification techniques will allow the specification of arbitrarily complex scenes.

To some extent what makes movies interesting is that a single two-hour movie can contain the distilled essence of a thousand man-years of work, and if it was a well managed process you can see each hour of effort up on the screen. You can see these movies over and over, and always see something new. It's like a tidal wave of information flooding over you. A small team of people won't be able to do that; but they can make perfectly good smaller movies.

Killing the Classics
by Skyshadow
Several directors have recently released "special editions" of their classic movies which subtly change the films by using computers effects to either clean up the old effects or (far worse) alter the original film.

The problem that I have with this is twofold: First, these "special editions" seem to be the ones that show up on TV and on video rental shelves, so that they and not the original become the pervasive copy.

Second, I can foresee a day when older movies are edited in this fashion so they can be remarketed to audiences with more "modern" attitudes (think similar to Speilburg taking the guns out of the hands of the pursuing authorities in the ET rerelease).

Do you believe that, as a creative professional, you have any sort of ethical duty to resist these sorts of changes? Is there a line to be drawn between merely cleaning up the original effects and replacing them entirely (as in the Star Wars special edition), or between effects-patchup and all-out content alteration (aka, the wussification of Han Solo by having Greedo shoot first)? Do you feel that old films should be left alone, or do you consider them more as ongoing acts of creation?

Thad:
I do not like the changing of movies. A movie, to me, reflects the time that it was created and becomes a kind of historical document. On the other, dominant hand, it is completely the choice of the owner of a film to do with it what he pleases.

I can understand the feeling that a movie is somehow owned by society at some point, but my point of view is different. Making a film is tremendously hard, making a good one far harder still -- and with that effort comes the right to muck it up down the road if that's what one decides to do. So, I don't see an ethical dilemma at all. I think that the place to make your protest felt is as a critic and as a movie patron; vote with your wallet. A related problem is that movies have a relatively short lifetime. The first Star Wars film reportedly had deteriorated quite a bit before the Special Edition was created; as the dyes in the film don't have good long-term stability. There will be a fifty-year period of movies that will be lost unless extraordinary (and unlikely) efforts are made. In the near future, though, all movies will be digital at some point in the process, and they have at least a fighting chance of being around for a long time. There are several digital-post facilities being set up now, which scan the whole film to allow better color correction and editing -- the most striking use of this was on O Brother, Where Art Thou?, where the final movie was dramatically color-corrected throughout in a way not possible with optical means.

question for thad
by Jucius Maximus
Thad: When designing tools for making 3D scenes or characters, how much does real world physics play into what is generated? Do you use fluid mechanical models to generate the flow of water over a waterfall or the movement of a large tree affected by a mass of air? Do you use vibro aoustical and biomechanical models to determine they way a CG mechanised character will walk?

In essence, how much do you take real physics into account when designing something a CG item to emulate a 'real' item on screen? What is the balance between physical limits and creative freedoms?

Thad:
Our charter is to create the sequence that the director of the film wants for his movie; that usually means building things that look and move like things do in real life. Often we would use real-world physics to do this. Typically, though, we take extremely simplified views of the real world to make the computations more simple, and to make them run faster.

As an example of physics in action, Nick Foster at PDI created a simplified fluid dynamics model to be used for animation; this was used to create several shots in ANTZ and Shrek. One of the big problems with simulation, as opposed to animation, is that it is difficult to control. Typically one sets initial conditions and then lets the simulation run. Having a system that runs very quickly enables the artist using the tool to try many different initial conditions, to try to create the desired final result. A slow, but more physically accurate solution would have been worthless if the animator couldn't get to a reasonable result.

Often what is done is an absurdly simplified model of reality is chosen, then it is made more accurate (and slow) until it looks good enough for the film. On our recent movie 'Showtime', we had to do a waterfall bursting out of a building, and we simulated the motion of water with air-drag, then simulated the water dragging the air with it, to get the characteristic motion of a waterfall; but we didn't have to go any further than that and simulate viscosity of the water or evaporation.

ILM has done some wonderful work simulating the dynamics of creatures, creating models of bones, muscles, fat, and skin. These give a character like a dinosaur a 'weight' that just can't be animated by hand. These dynamics are a great cue to they audience for how big and heavy these creatures are.

One curious reality of the FX world is that often reality is not what is wanted. A classic example of this is starfields. In any real-world photograph, the stars are invisible, they are far far darker than anything else in the scene. Directors often want stars in the sky to go with their actors, though; so that is what they get.

Finally, there are times when straightforward animation is the best approach. For the movie Red Planet, we had to create zero-g fire. I spent a few weeks trying to simulate the flow of smoke and fire in a zero-g environment, when my colleague Jamie Dixon thought that he could just animate all the shots by hand in a couple of days -- which he proceeded to do. When CalTech's physics department reviewed the movie for the Los Angeles Times, they panned every bit of science in the movie, except for the zero-g fire; which they thought looked "pretty cool."

CGI alternatives
by Strange Ranger
Do you think CGI can too often be seen as a "suppressor" of other art forms? The specific example in my head right now is Old Puppet Yoda vs. New CGI Yoda, we haven't seen (AFAIK) any major puppeteering work in cinema in a long time. Other possibly "suppressed" art forms might be makeup art, the art of the stunt man, set construction, backdrop painting, cinematograghy, heck even acting could be listed here. Will CGI be escorting some or all of these art forms down the same path as Silent Films, blacksmithing, and totem-pole carving?

Do you ever want to say "Hey this would be a lot better if it were done with [not CGI] instead"?

Thad:
There are many times as many people working in the FX field today as there were ten years ago. Now, it's true that some of the techniques are not as much in demand as they were, but it's not as bad as you might think. A company that we do quite a bit of collaborative work with is Illusion Arts, in Van Nuys California. The two founders, Bill Taylor and Syd Dutton, started doing practical camera effects and matte paintings, and built a very successful company around these kinds of classical techniques. Today, they are now doing synthetic 3D camera moves and painting on Macs; but 90% of the talents and skills they used before are still applicable today; just the medium is different. What makes a good artist is foremost their eye; their ability to see what is right and to see how to fix what is wrong. Illusion Arts was the lead shop in The Fast and The Furious, and along with us and Digiscope made a very modern movie.

In your example of puppetry, I too was a little disappointed to see the CG Yoda; especially in the closeups it just wasn't exactly the same. Of course, there's no way that a puppet could have done the lightsaber battle. Also, a growing area of FX is performance capture; recording data in real-time and applying it to CG characters. In Episode One and Two of the Star Wars movies, there is a tremendous amount of motion capture, used to animate robots and creatures. Performance capture is just puppetry with one's whole body, really.

Back in 1989, Graham Walters and I build the CG puppet Waldo C. Graphic for The Jim Henson Hour. The puppet was animated by putting one's hand into a 'waldo' (a mechanical tracking device reminiscent of a Luxo Lamp), and moving it around; and watching the results on a TV screen. This was so similar to the way that the Henson puppeteers usually work that it took no time at all to get comfortable with the puppet; I don't think it took Henson himself more than about 5 seconds to get totally up to speed on it.

Speaking about stunt work, one of the very first things that people realized with digital techniques is that 'wire-removal' is fairly straightforward. One can identify a moving wire in a scene and use several techniques to get rid of it. This meant that whereas stunt people used to use the thinnest possible safety wires, or none at all; now they could use systems with significant margins of safety. Also, face-replacement techniques coming to the fore means that stunt people can play far closer to the camera than they used to, opening up new opportunities for stunts.

When it comes to acting, though, I don't think that digital graphics will ever replace traditional techniques. There's no good reason to attempt it, and it's unbelievably hard. The subtlety and complexity of motion of skilled human actors is astonishing, and a ridiculous portion of the human brain is dedicated to interpreting those expressions and motions.

So, I would say that for every job lost, many are created -- and the people whose jobs are lost can often put those same skills to use in this new digital world.

Little studios vs Big Studios
by Milinar
I've followed your company's work over the past few years with great interest. It seems to me that the effects you do are pretty much on par with big studios like digital domain, etc. Have you purposefully stayed a small studio, with a few dedicated individuals? And what advantages has that given you?

Thad:
When we started Hammerhead, we made a deliberate commitment to stay small. We didn't see significant economies of scale in the field, and it seemed like we'd have much more fun in a smaller company. There is a strong culture in American business that you have to "grow or die", but it was our experience that growth was extremely difficult to manage and that companies that grew quickly found themselves dying quickly, too. Once you get past a couple of dozen people, there seems to be a phase change in company culture, and productivity declines.

We do find that our small ('boutique' is the term of art) studio can compete against companies one or two orders of magnitude larger than us on many jobs that don't require a huge volume of shots. While our staff is small, we are extremely experienced, having been doing digital visual effects since we helped create the field at the beginning of the 90's. We tend to hire very capable, experienced artists -- one way that we keep it interesting for them is that they are given a huge amount of creative control over their shots.

As a small company we can be very flexible, too. We can reconfigure ourselves for whatever project is at hand; and become the Deep Blue Sea company when that is what is going on. There is very little overhead not contributing to getting a particular job done. Paradoxically, in a small company you can do more different things. We've done FX for films, wrote and produced a big Hollywood film, made our own low-budget horror film, and wrote and sold software. We will very likely be making a couple of TV pilots next year of shows with substantial visual effects content. Bigger FX companies have to be more focused, they can't afford to be experimental and possibly make mistakes, because they would be much larger mistakes.

Our biggest weakness is that we cannot even begin to take on huge jobs. Movies like Pearl Harbor or Spiderman require hundreds of people; and we have to leave these jobs to the ILMs and Sonys of the world. Still, there are hundreds of movies a year with a few dozen to a hundred and fifty shots, with reasonable time schedules, where we can compete well. I think that we have found a 'sweet spot', where many features combine to make a pleasant, profitable, successful company -- and the small size is an important part of that.

Dropped crusade against Pixar patent?
by Anonymous Coward
I heard a rumor that you dropped your "crusade" against Pixar's software patent on deep-shadow technology? The rumor implied you were "bought-out"? Care to comment/share your thoughts on software patents in the VFX industry?

Thad:
While this was not moderated up, I do feel it needs an answer. The patent that is referred to is for the obvious enhancement of Lance Williams' 1978 z-buffer shadow scheme [pdf link] given that today's computers have more than 64 Kb of memory. In the Williams algorithm the scene is rendered from the point of view of the light, and the depth to the first surface is stored. Then, when rendering the image from the camera's point of view, you can easily tell whether a surface should be in shadow or not. The Deep Shadow Map idea was to store a function of depth vs. opacity at each pixel in the image rendered from the light POV, to allow partially transparent surfaces and subpixel shadow coverage.

Unfortunately, Pixar has decided to patent this. They presented the idea at Siggraph '00 but didn't mention in the paper the fact that they'd filed a patent; although word got out pretty soon. As the patent has not been granted yet, and they filed the patent before the Patent Office's policy change that now publishes patent applications, it's unknown what their claims are. What I am fairly sure of, though, is that Pixar didn't invent this technology, and that people at Pixar know this. So, it's not only really nasty to try to build on somebody else's technique and wrest it for yourself, but there may be legal problems as well.

I've discussed this with lawyers, and they say that the time to fight a patent is after it grants. While that seem weird and suboptimal, there's nothing about patent law that isn't weird and suboptimal. So, I'm going to wait and see what happens. There are other possibilities for fighting the patent that don't make sense to reveal at this time, for obvious reasons. Clearly this comment reveals that there is no agreement between Pixar and me to remain quiet on this issue.

It wouldn't surprise me if patents destroy the visual effects industry as we know it today. Pixar already has one notch in its belt, last week forcing the company ExLuna to withdraw its Entropy renderer that competed with Pixar's Renderman (and the shareware BMRT program that preceded Entropy, as well). A rational, cold-blooded analysis of the software patent situation would reveal that almost every complex program today could be attacked on patent grounds, as we've seen recently with the JPEG fiasco. Back when I worked at PDI, we were attacked a couple of times for patent violations, only escaping a devastating patent by NYIT on the thinnest of technicalities. In irony not lost on anybody, Ed Catmull of Pixar (with Disney's lawyers help) led the fight against NYIT's patent.

Interestingly, this has happened before in visual effects. Back in the bad old days, every single analog visual effects technology was patented and owned by the studios. Rear Projection, Front Projection, Blue Screens, Sodium Screens -- everything. The studios would in effect pool the patents between themselves; but if you wanted to make a visual effects film you had to do it completely within the studio system. It might happen again.