Slashdot Mirror
Rendering Shrek@Home?
JimCricket writes "There's an interesting piece at Download Aborted about using distributed computing (a la SETI@Home, Grid.org, etc.) in the film industry. With the recent release of Shrek 2, which required a massive amount of CPU time to complete, one must wonder why the film industry doesn't solicit help from their fans. I'd gladly trade some spare CPU time in exchange for the coolness of seeing a few frames of Shrek 3 rendered on my screensaver!"
Easy: Pixar and Dreamworks have both developped highly proprietary rendering technology. They're not about to just give copies to everyone who wants one. Even if the renderer itself weren't reverse-engineered, which isn't beyond the realm of possibility, it would likely be far easier to decipher the protocol used and voila, a functioning copy of [Pixar|Dreamworks]'s renderer.
Lobotomizing it to the point where this wouldn't be useful would probably make it useless for distributing the workload as well.
There'd be no sound.
I'm sure people would sit through it anyway, though.
Only the most high end of machines could even consider attempting to render even one layer of a frame for this kind of animation. We're talking systems with 2-4GB of RAM as a minimum (preferably 4+) and the scene files/textures would weigh in the tens to thousands of megabytes that must be downloaded for each scene. Think uncompressed TIFF or TARGA texture files that might be 5000x5000 at 40 bits/pixel.
:)
Even on high end machines they often do not render a full frame, but a layer of a frame which is then composited with other layers into the full frame. Why? Many reasons but one of them is that even the high end machines don't have enough RAM and the render would take too long (the machine would need to swap).
So aside from the issues of fans returning bogus data, or extracting highly proprietary information out of the client as other threads have mentioned, this would be a real show stopper. Breaking the problem into small enough pieces to be handled by joe-blow's computer would be prohibitive and require tons of calculations to figure out which pieces of textures are actually required for a given piece of rendering etc. It would probably require a compute farm just to manage it!
Rendering is also a lot more complex than you might think, there are render wranglers who manage the rendering queues and look at the outputs... many renders may require specific versions of the rendering software, so a frame that rendered with 2.7.2.1 won't render anymore without errors with 2.7.2.2... so many copies of the software are managed in parallel with the wranglers helping to clean up the errors. How would you manage this in a distributed client environment?
Furthermore most of the proprietary rendering apps are certified against VERY specific platforms, eg. one specific kernel version and build level, specific versions of shared libraries etc.
Long and short is there's a reason why movies cost millions.
"what's proposed here is just does not seem possible under low bandwidth conditions. it's not like you can just run off to computer #2,398 and say "go render frame 1,503" -- there are textures and models and state information that probably total somewhere on the order of gigabytes (give or take a factor of ten) in order to render that frame."
I can give you a little data here. Take a look at this image I made. The scene is roughly 1.5 million polygons, and virtually everything is textured. The folder containing the bare bones version of this scene is roughly 600 megabytes. I could probably cut that size in half via JPEG etc, but we're still looking at a massive amount of data to send to one person to render a frame. I know this because I seriously discussed sharing the rendering with a friend of mine on the east coast. We both felt it'd take longer to ship than it would to render.
I doubt this scene is anything close to what they were doing in Shrek 2, let alone whatever will happen with 3.
"Derp de derp."
How would you ever reproduce this on a distributed network of limited bandwidth home PC's ? Here's some LOTR rendering stats from Wired.com - [http://www.wired.com/wired/archive/11.12/play.htm l?pg=2]
... The Return of the King, which opens in theaters December 17, will feature almost 50 percent more f/x shots than The Two Towers and will be composed of more data than the first two movies combined. Churning out scenes like the destruction of Barad-dûr and the Battle of Pelennor Fields (with thousands of bloodthirsty CG Orcs) took 3,200 processors running at teraflop speeds through 10-gig pipes - that's one epic renderwall. What else went into making Frodo's quest look so good? By Weta's account, more than you might think.
WETA BY THE NUMBERS
HUMANPOWER
IT staff: 35
Visual f/x staff: 420
HARDWARE
Equipment rooms: 5
Desktop computers: 600
Servers in renderwall: 1,600
Processors (total): 3,200
Processors added 10 weeks before movie wrapped: 1,000
Time it took to get additional processors up and running: 2 weeks
Network switches: 10
Speed of network: 10 gigabits (100 times faster than most)
Temperature of equipment rooms: 76 degrees
Fahrenheit Weight of air conditioners needed to maintain that temperature: 1/2 ton
STORAGE
Disk: 60 terabytes
Near online: 72 terabytes
Digital backup tape: 0.5 petabyte (equal to 50,000 DVDs)
OUTPUT
Number of f/x shots: 1,400
Minimum number of frames per shot: 240
Average time to render one frame: 2 hours
Longest time: 2 days
Total screen time of f/x shots: 2 hours
Total length of film: Rumored to be 3.5 hours
Production time: 9 months
My big question is why would you rather donate to a large commercial organization well funded from it's previous Shreck flick -- rather than donate the cycles to a project like the IMP works themselves?
Renderman is not a renderer it is a specification for interoperability between modeling tools and renderer (like XMI for software engineering tools except that it works.)
Pixar's renderer is actually PRMan.
From Renderman.org:
There are a lot of people when you hear them talking about RenderMan and how great the images are from it, etc. They are most likely really talking about Pixar's PhotoRealistic RenderMan® (PRMan).
RenderMan is actually a technical specification for interfacing between modeling and rendering programs. From 1998 until 2000 the published RenderMan Interface Specification was known as Version 3.1. In 2000 Pixar published a new specification, Version 3.2. Coming soon Version 3.3
English is not this
I believe I saw someone earlier mention how there can be terabytes of data go into a single frame of CGI film, and these days that can be pretty correct.
.rib file or similar type file for PDI's renderer will probably contain a few million polygons and/or a few hundred thousand control verticies for implicit surfaces such as nurbs and sub-Ds, which can be a lot of data (my scene files at work average 4-5 million polygons and are about 150 megs on average, saved in a binary file format). And, that doesn't include particles, procedurals, all the motion data so that proper motion blur can be calculated...
A
And then the textures... They do use lots of procedurals, but they also use lots of 16 bit per channel textures of 4000x4000 for face textures, or even higher. Some people are using tiles if 16 bit tiffs for displacement maps now that equate to like a 100,000x100,000 image for displacement maps, because the accuracy requirements for close up renders are so bloody high. That can be many many gigs of data there.
And, if you're raytracing like in Shrek 2, then you need to have as much of that data in RAM at once, or else render time spirals out of sensibility, unlike scanline renderman where swapping is easier, because the rays bouncing throughout the scene make scene divisions more difficult (but still possible).
I work with 4 gigs of RAM and we can just barely render 6 million polygons + a few 4k displacement maps all raytraced at once (in windows unfortunately). And, when we render sequences and stuff, we often almost kill our network because distributing all this data to just 20-30 rendernodes is pretty tough (and how would that scale to a big renderfarm with thousands of rendernodes...)
So, yeah, like everyone else is saying, bandwidth limitations and that people running the screen saver probably don't have the hardware and OS to really run 4+ gigs of RAM, this Shrek@home idea seems rather unlikely. It would be cool though, if it worked...
Hooray for my totally unoriginal post!
you cannot dodge the quad laser. jumping is useless.