Slashdot Mirror

They re-wrote an entire renderer? Granted, Shrek is still behind some of Pixar's work but i've got to ask... Why not use some of the other renderer tools out there?

Because PDI is a mostly propietary place. They wrote their own renderer years before there was anything commercially available. As such they have an R&D team continually updating their infrastructure. Interestingly enough I saw a couple of PDI guys at the SIGGRAPH photon mapping course by Henrik Wann Jensen a few years ago in San Antonio.

The upside is you don't have to wait for a commercial vendor to get those new features. They control their own destiny rendering wise. Witness for example how long it took Pixar to make Depp Shadow maps available in PRMan (something like 2 years) even though they had published a SIGGRAPH paper and were using it internally (for Monsters Inc.). Some clients were a bit upset about that.

Dan Wexler used to write their renderer (he is now at Nvidia with Larry Gritz and those crazy Entropy guys). He has some interesting statistics on the first film:

Renderfarm Statistics
Shrek Rendering Statistics

They re-wrote an entire renderer? Granted, Shrek is still behind some of Pixar's work but i've got to ask... Why not use some of the other renderer tools out there?

Because PDI is a mostly propietary place. They wrote their own renderer years before there was anything commercially available. As such they have an R&D team continually updating their infrastructure. Interestingly enough I saw a couple of PDI guys at the SIGGRAPH photon mapping course by Henrik Wann Jensen a few years ago in San Antonio.

The upside is you don't have to wait for a commercial vendor to get those new features. They control their own destiny rendering wise. Witness for example how long it took Pixar to make Depp Shadow maps available in PRMan (something like 2 years) even though they had published a SIGGRAPH paper and were using it internally (for Monsters Inc.). Some clients were a bit upset about that.

Dan Wexler used to write their renderer (he is now at Nvidia with Larry Gritz and those crazy Entropy guys). He has some interesting statistics on the first film:

Renderfarm Statistics
Shrek Rendering Statistics

There seems to be a bit of confusion as there are two animation companies at Dreaworks. There is Dreamworks/PDI (makers of Shrek and Antz) and then there is Dreamworks Animation (makers of El Dorado and Prince of Egypt). They are separate entities though both use Linux. You can get a pretty good description of their use at Animation in the Linux Journal arrticle from about a year ago. They seem to be turning both workstation and servers to Linux. PDI uses more a mix of Linux and SGI. For some interesting stats check Dan Wexler's site:

Shrek Rendering statistics

But it sounds to me like they're saying that they're actually doing the work on software running under Linux? I'm just sort of wondering.. where are they getting this software?

There are a number of critical factors in this process that you haven't told us. The issues of display devices, gamma, and implementation details all play an important role in your ability to visibly distinguish between two colors.

What sort of monitor are you using? Have you correctly callibrated the display? What software are you using to display the colors? How does this software deal with display gamma? Other important details include the brightness of the surrounding environment, other windows and such on the screen which can distract the eye and interfere with your visual processing of the colors.

If you haven't already read the books and web pages by Charles Poynton, they cover all the details. Color issues always seem simple, but actually this is an extremely complex and subtle issue. Also, people's ability to visually distinguish color varies quite a bit. A surprisingly large percentage of the population is color blind to at least some portion of the spectrum. Also, display devices vary widely in their ability to correctly display different colors.

Anyway, to sum it all up, I'd be really surprised if you can use any sort of theory to predict whether you can visually distinguish between different colors. Even with correctly callibrated equipment, and experienced researchers, I doubt that your problem is easily answered!

Best of luck,

Daniel Wexler
www.flarg.com

Here is the article, it mainly deals with Dreamworks Animation, but also touches on PDI. There is also Daniel Wexler's page, who is the main rendering guy at PDI:

Well you might be right, but as far as I know no major movie FX or animation have been done over the Net. Not only because techical considerations but also dealings with the studios. Studios want to protect their investement and market the film according to their plans. You can't just transmit stuff over the net without concerns that someone might snoop in. Besides probably doing the transfers to Exabyte and sending them via courier are faster and more secure.

There might be a few cases though. For example DD has a commercials divisions (also does music videos) and used to have an interactive division. Maybe they worked sending stuff over the net for these smaller projects. ILM Commercial Productions seems to be doing this also (but not the main film FX part):

So maybe you are right.

The stuff your are mentioninmg is referred to 4:3 pullup or pulldown (depending on direction) about transfering from film at 24 frames per second to video at 30 frames per second and viceversa.

But the other thing about FX is not that the plates are scanned typically at 2K resolution but all the data files involved in the FX elements: models, shaders, texture maps, etc. In the renderMan newsgroup there was a thread some time ago that had some very interesting stats. While a single frame might be around, say 10MB, RIBs and textures alone for models might take dozens to mabybe even hundreds of MBs. I couldn't find the reference but it was probably posted by Tom Duff or Larry Gritz. This past SIGGRAPH some people discussed some info about their projects. There is some info about Shrek out there, stating that on average they had 2 to 3 GB per frame (though FX would be a bit lower since not all the image is 3D). It's usually a lot less number of frames, but the amount of data per frame is huge. Take Pearl Harbor that only had around 200 shots but they occupy like 50 min of screen time, or long seuqnces like the Contact oull back or the frozen NY flyby in AI.

You might want to check the RenderMan course notes from this past SIGGRAPH, also a lot of info. For the Gungan battle of episode 1, all the baked RIBs accounted for 64 GB of data. The smallest RIB for one cycle was 209 MB.

About Major Damage, besides SIGGRAPH I think it was discussed in the CH_Char mailing list, now web forum. Jeff Lew used (or maybe still) posts around there.

I work in the R&D department at PDI/DreamWorks. My website has some real renderfarm statistics and some specific Shrek statistics. This article was fairly accurate, which is great. You can believe the hype. The entire production industry is behind Linux and is pushing the hardware and software vendors to firm up their offerings. We just had an industry wide meeting on Linux, and the movement toward Linux and away from Windows is clear and strong. There is even talk that some of the high end studios may work together to release some OpenSource tools, but right now that's still pie-in-the-sky. However, the fact that these studios, which previously guarded their trade secrets jealously, are even talking about this possiblity is exciting.

Also, PDI/DreamWorks is trying to release some code under OpenSource. I just released my frame buffer library under GPL. It is a small, but important, gesture, as it represents the first source code that we have ever released. We hope to have more soon.

Someone mentioned TIFF playback, which, these days is really easy, since it the latest rounds of cards are able to support 30fps playback using standard OpenGL calls. We have had our internal flipbook and quicktime programs working for over a year now. I'm sure the public tools will soon cover this gap. Also, audio is starting to work well. Our tools have nice sync'ed audio playback, which was one of the last things we got in place. We are now placing Linux workstations on animator desktops as opposed to batch use on the renderfarm. It has been a long road, but we are finally there.

I work in the R&D department at PDI/DreamWorks. My website has some real renderfarm statistics and some specific Shrek statistics. This article was fairly accurate, which is great. You can believe the hype. The entire production industry is behind Linux and is pushing the hardware and software vendors to firm up their offerings. We just had an industry wide meeting on Linux, and the movement toward Linux and away from Windows is clear and strong. There is even talk that some of the high end studios may work together to release some OpenSource tools, but right now that's still pie-in-the-sky. However, the fact that these studios, which previously guarded their trade secrets jealously, are even talking about this possiblity is exciting.

Also, PDI/DreamWorks is trying to release some code under OpenSource. I just released my frame buffer library under GPL. It is a small, but important, gesture, as it represents the first source code that we have ever released. We hope to have more soon.

Someone mentioned TIFF playback, which, these days is really easy, since it the latest rounds of cards are able to support 30fps playback using standard OpenGL calls. We have had our internal flipbook and quicktime programs working for over a year now. I'm sure the public tools will soon cover this gap. Also, audio is starting to work well. Our tools have nice sync'ed audio playback, which was one of the last things we got in place. We are now placing Linux workstations on animator desktops as opposed to batch use on the renderfarm. It has been a long road, but we are finally there.

I work in the R&D department at PDI/DreamWorks. My website has some real renderfarm statistics and some specific Shrek statistics. This article was fairly accurate, which is great. You can believe the hype. The entire production industry is behind Linux and is pushing the hardware and software vendors to firm up their offerings. We just had an industry wide meeting on Linux, and the movement toward Linux and away from Windows is clear and strong. There is even talk that some of the high end studios may work together to release some OpenSource tools, but right now that's still pie-in-the-sky. However, the fact that these studios, which previously guarded their trade secrets jealously, are even talking about this possiblity is exciting.

Also, PDI/DreamWorks is trying to release some code under OpenSource. I just released my frame buffer library under GPL. It is a small, but important, gesture, as it represents the first source code that we have ever released. We hope to have more soon.

Someone mentioned TIFF playback, which, these days is really easy, since it the latest rounds of cards are able to support 30fps playback using standard OpenGL calls. We have had our internal flipbook and quicktime programs working for over a year now. I'm sure the public tools will soon cover this gap. Also, audio is starting to work well. Our tools have nice sync'ed audio playback, which was one of the last things we got in place. We are now placing Linux workstations on animator desktops as opposed to batch use on the renderfarm. It has been a long road, but we are finally there.

Well for one thing PDI uses almost propietary in house built software. They did use Maya for modeling some of the stuff and certain fire FXbut as far as rendering, animating, etc. they used their own stuff. CGW did a report on Shrek. What they have done is that they have po0rted most of their software to Linux. Daniel Wexler, the guru behind their renderer has posted very interesting statistics on their rendering and Linux use. Last years SIGGRAPH had a Sketches section devoted to it and this year there will be a course. VES 2001 also had a talk about some of their technology used. And the current issue of Cinefex, THE magazine of the FX industry covers Final Fantasy in detail:

So it isn't exactly like no info was out before.

You are right, we didn't use Linux boxes on the desktop for Shrek. However, the first desktop Linux boxes are now in production use at PDI.

I have some slightly out-of-date information about the actual composition of our renderfarm and some rendering statistics for Shrek on my website at www.flarg.com.

I work in the R&D department at PDI. We do use some SGI Linux boxes in our renderfarm, but not on the desktop. The SGI's probably comprise about 10% of the renderfarm and less during Shrek production. Despite what you may hear reported in the press, we only used Linux boxes in our renderfarm during Shrek. We are just starting to deploy desktop Linux boxes now.

About 80% of our pipeline is proprietary software. The other 20% is largely made up of Maya, which works on our Linux boxes (so to speak) and things like PhotoShop where we generally use Macs. We are lucky that we use proprietary software since it made it feasible to get the renderfarm ported to Linux quickly. The GUI based tools were also relatively easy, but it took us about a year to get everything totally ported and ready for desktop production. Of course our staff of 16 programmers was also doing production support at the same time.

For the details on our Renderfarm setup, check out my webpage at www.flarg.com

Also, during Shrek, our animators used desktop SGI O2s (yes, O2s, not Octanes) running IRIX. About half to three-quarters of the renderfarm was made up of Linux boxes. Of that, only about 10% were HP boxes. We also had a spattering of V/A Linux, SGI and Atipa boxes. HP just gets the press.

For specific statistics regarding our Renderfarm and how it was used on Shrek, check my website www.flarg.com. I work in R&D at PDI, and these are live stats on rendering performance along with some other interesting tidbits.

Well not quite a surprise considering recent happenings in the last few SIGGRAPHs. Of course there is that recent article in Millimeter magazine, and for specific you can check the rendering statistics from Daniel Wexler, PDI's renderer architect:

I work in the R&D group at PDI/DreamWorks and I've put up a rendering statistics page with a description of our renderfarm and some usage graphs. We're rolling out Linux desktop platforms running our proprietary animation software right now. Our animators are getting a 12x performance and 8x memory boost over the SGI O2 machines used on Shrek, our next feature, due out in May.

I work in the R&D group at PDI/DreamWorks and I've put up a rendering statistics page with a description of our renderfarm and some usage graphs. We're rolling out Linux desktop platforms running our proprietary animation software right now. Our animators are getting a 12x performance and 8x memory boost over the SGI O2 machines used on Shrek, our next feature, due out in May.

I work in the R&D group at PDI/DreamWorks and I've put up a rendering statistics page with a description of our renderfarm and some usage graphs. We're rolling out Linux desktop platforms running our proprietary animation software right now. Our animators are getting a 12x performance and 8x memory boost over the SGI O2 machines used on Shrek, our next feature, due out in May.