Improv Animation as an Art Form?

Dean Siren asks: "When will mainstream moviemakers, such as Lucasfilm, finally replace their render farms and Renderman with a GPU (Geforce or Radeon) and Cg based renderer? Would the savings in equipment cost and rendering time be worth the learning curve? Is anyone developing such an app? We've had the tech for years with video games, but the art form hasn't really been tried. Is anyone working on this now?" An interesting thought, and it puts an interesting spin on the old computers-will-replace-actors argument. It also means good planning ahead of time, since there will be no "post-production" stage where you can clean up the mistakes, and perform the minute adjustments needed to make things just right. Do you think such an art form will ever catch on in Hollywood, or will small shops have to be the ones to pioneer this before others follow suit?

"There's a forum called Machinima whose main idea is that not only should the final rendering of a movie be generated in real time, but so should the animation, implying that computer animation should be performed, maybe even improvised, live by motion captured voice actors. Accomplishing this goal would require replacing not only Renderman but Maya and Softimage as well. A developer named Strange Company took the challenge and started writing an app in this direction called Lithtech Film Producer (interview here). They even made easy porting a high priority. But they soon realized that they were tiny and the project was huge so they quit. But the idea of improv animation is full of potential."

Comments

HUH?

[furiousgeorge]

Is it a slow news day or what???

In a nutshell, this topic makes zero sense.

Nobody is going to drop PRman for Cg anytime soon. Why? Because they have two different target markets and address two different needs.

Talk to somebody like ILM or PIXAR thats doing renderings that take 70 hours a frame (like some of the frames for Toy Story II did) and talk about real time cards. They have a good laugh and say "go away kid".

Can these cards handle anti-aliasing like RM can? No.
Can these cards handle DOF like RM can? No.
Can these cards to programmable shading like RM can? No.

These cards are designed to do graphics real time with the best quality they can squeeze out while still hitting their timing targets. RM is meant to get the best possible quality - and who cares about time?

This is a silly pointless discussion. Yes, in 10 or 20 years maybe the hardware will be there, but it isn't now and you sound silly making speculations like these.

They have, where it makes sense.

[Bruce Perens]

Studios like Disney, Pixar, and Dreamworks are installing PCI 3D cards in the animator and technical director's workstations. These were previously much more expensive SGI workstations, and are now IA-32 PCs running Linux and OpenGL.

It's not at all clear to me that Cg provides any advantage over OpenGL used from C/C++ for the sort of work that the high-end studios do.

The vanilla CPUs in render farms and the software renderers that run on them could be replaced by hardware rendering for the lower-quality work, but never for the highest. First, the render farm doesn't need the real-time facility of the GPU - the part the GPU does best, and the part that contributes most of the cost to the GPU. The render farm just needs to render a frame to disk, and can do this more cost-effectively with a software renderer and a general-purpose computer. Second, the GPU isn't as extensible as the software renderer, because it's cast in silicon. There will always be an effect you want that the GPU can't handle. And then, the GPU is built to render video fast, and trades off many aspects of the rendering algorithm that we really want when we render to film.

You will, however, see all of the studios buy arrays of GPUs for making rushes. These are less-than-full-quality playbacks that they use to review the animator's work-in-progress before final rendering. If we got some really fast programmable gate-arrays, or GPUs with documented and programmable microcode, we could use them as a GPU is used, but in a way that might support the highest-quality rendering.

Pixar tried to make high-speed hardware for years, and we always found it to be a losing game. I wrote microcode for one of these beasts, a parallel bitslice engine that inspired today's MMX instructions. We could not keep up with the development of vanilla CPUs, and the CPUs ended up being more cost effective.

Thanks

Bruce

An Odd Lack of Vision Here

[FreeUser]

I've seen a plethora of posts that basically argue "today's tech can't do it, so this is a stupid discussion."

Remarkable.

Technically savvy poeple, of all people, should realize that simply because Farscape-style special effects cannot be done in realtime today with today's low end consumer graphics GPUs doesn't mean the concept of 'live performance animation' as such is flawed at all.

First, much lower quality 'live performance' animation is possible with today's consumer hardware, and the improv aspect alone makes it an art form worth persuing in and of itself. The possiblity for algorithmic and technical enhancements that could be driven, or at least explored, by such an art form make it a worthwhile endeavor as well.

Second, in another 5 or 10 years (at most) it will almost certainly be possible to do live performance, farscape quality digital animations (assuming the technological development of the computer hasn't been brought to a standstill through stupid legal 'innovations' like DRM and Palladium). While movie makers would likely simply add this to their set of tools and not replace post-production entirely, the ability to create 'live theatre' digital productions and interactive, perhaps even submersive, two or multi-way environments if not completely synthetic realities is an intriguing one, to say the least, and certainly a worthwhile endeavor whether or not Hollywood can make use of the technque in their movie productions. Indeed, such systems could well render the movie as obsolete as the live stage play is today: in other words, no longer the main popular attraction, but a continuing artform valid in its own right, if no longer the center of public attention.

8 years ago I was at the U of Illinois' virtual reality lab and had an opportunity to play around with some of simulations they run, including one which allows the viewer to explore a three dimensional (submsersive) grey-scale view of the mega-structure of galaxies in the universe (to study large scale structures such as strings of galaxies, etc.).

8 years later I can explore the universe in living color on my GNU/Linux box running Celestia, in 1920x1200 24-bit color, in realtime. While it isn't submersive 3-d VR just yet, it is much higher resolution and full color, and while I can't explore the farthest reaches of the universe, I can explore the immediate galactic neighborhood in incredible detail (much greater than the old simulation allowed). All of this on a $400 Nvidia card, running a free operating system on commodity hardware.

So, in other words, dismissing this possibility simply because you can't do it with perfect, photo-realistic effects today shows a remarkable lack of vision, and a blindness to similiar leaps in technology that we've all beeen taking for granted for the last decade or two. We will be able to do this sort of thing, photorealistically, much sooner than most people probably realize, and the art form can be persued long before the final polish is available.

Realtime and offline rendering ARE converging

[John Carmack]

There are some colorful comments here about how studios will never-ever-ever replace tools like renderman on render farms with hardware accelerated rendering. These comments are wrong.

The current generation of cards do not have the necessary flexibility, but cards released before the end of the year will be able to do floating point calculations, which is the last gating factor. Peercy's (IMHO seminal) paper showed that given dependent texture reads and floating point pixels, you can implement renderman shaders on real time rendering hardware by decomposing it into lots of passes. It may take hundreds of rendering passes in some cases, meaning that it won't be real time, but it can be done, and will be vastly faster than doing it all in software. It doesn't get you absolutely every last picky detail, but most users will take a couple orders of magnitude improvement in price performance and cycle time over getting to specify, say, the exact filter kernel jitter points.

There will always be some market for the finest possible rendering, using ray tracing, global illumination, etc in a software renderer. This is analogous to the remaining market for vector supercomputers. For some applications, it is still the right thing if you can afford it. The bulk of the frames will migrate to the cheaper platforms.

Note that this doesn't mean that technical directors at the film studios will have to learn a new language -- there will be translators that will go from existing langauges. Instead of sending their RIB code to the renderfarm, you will send it to a program that decomposes it for hardware acceleration. They will return image files just like everyone is used to.

Multi chip and multi card solutions are also coming, meaning that you will be able to fit more frame rendering power in a single tower case than Pixar's entire rendering farm. Next year.

I had originally estimated that it would take a few years for the tools to mature to the point that they would actually be used in production work, but some companies have done some very smart things, and I expect that production frames will be rendered on PC graphics cards before the end of next year. It will be for TV first, but it will show up in film eventually.

John Carmack