Slashdot Mirror

← Back to Stories (view on slashdot.org)

Rendering Shrek@Home?

Posted by timothy on from the render-it-like-pork dept.

JimCricket writes "There's an interesting piece at Download Aborted about using distributed computing (a la SETI@Home, Grid.org, etc.) in the film industry. With the recent release of Shrek 2, which required a massive amount of CPU time to complete, one must wonder why the film industry doesn't solicit help from their fans. I'd gladly trade some spare CPU time in exchange for the coolness of seeing a few frames of Shrek 3 rendered on my screensaver!"

21 of 345 comments (clear)

  1. Doubt it'll happen... by Zweistein_42 · · Score: 5, Insightful

    Security issues would be a concern I'm sure. There's plenty of hackers who'd see no harm in, for example, extracting a number of images from around the world and sticthing a trailer, etc. And of course, rendering is a "trial-and-error" process - would they want people to have access to broken scenes? Or deleted scenes? Speculation would seriously dampen their ability to control marketing and release info. On the technical side, farms are reliable and predictable. Who can figure out how many fans will keep their computers up tonight for the critical preview tomorrow? What about the decline of interest after first little while? Distributed computing of this sort isn't well suited for commercial projects with fixed schedules. Not that I don't think it'd be COOL... I just don't think it'll happen :-/

    --
    - To err is human; but to really screw up, you need a computer
    1. Re:Doubt it'll happen... by frenetic3 · · Score: 5, Interesting

      I'm not a film tech -- but besides abuse and security issues, what's proposed here is just does not seem possible under low bandwidth conditions. it's not like you can just run off to computer #2,398 and say "go render frame 1,503" -- there are textures and models and state information that probably total somewhere on the order of gigabytes (give or take a factor of ten) in order to render that frame. Joe Dialup isn't going to be able to handle that; the film studios I'm sure have crazy fiber/multi-gigabit interconnects within their rendering farms.

      If they could find a way to offload some intermediate calculations (like deformations of hair or fabric or something that can be used as an intermediate result in a scene) then that might be a clever use for a distributed.net style technique.

      -fren

      --
      "Where are we going, and why am I in this handbasket?"
    2. Re:Doubt it'll happen... by vasqzr · · Score: 5, Informative


      There'd be no sound.

      I'm sure people would sit through it anyway, though.

    3. Re:Doubt it'll happen... by joib · · Score: 5, Insightful


      the film studios I'm sure have crazy fiber/multi-gigabit interconnects within their rendering farms.

      While the amount of data to move around probably is too much for dialup, gigabit ethernet is certainly fast enough, and dirt cheap as it's integrated on motherboards. If you look at the top500 list, you see that weta digital (the company which did the CG for lord of the rings IIRC) has a couple of clusters on the list, and they have gig ethernet.

      Basically, while rendering is cpu-intensive it's not latency sensitive, so there's no point in blowing a huge amount of cash on a high end cluster interconnect.

    4. Re:Doubt it'll happen... by b1t+r0t · · Score: 5, Interesting
      Why on earth would they want to let the frames from Shrek get out onto the net where rampant piracy and trading would ensue, just so your computer could turn in a frame a week?

      The heck with that, why would they want the 3D wireframe models to get out on the net? What do people think the frames are rendered from, anyhow? I predict it would be less than one week between someone figuring out how to extract the models, and someone else making a low-res animation of those models doing the nasty with each other.

      --

      --
      "Open source is good." - Steve Jobs
      "Open source is evil." - Microsoft
    5. Re:Doubt it'll happen... by NanoGator · · Score: 5, Informative

      "what's proposed here is just does not seem possible under low bandwidth conditions. it's not like you can just run off to computer #2,398 and say "go render frame 1,503" -- there are textures and models and state information that probably total somewhere on the order of gigabytes (give or take a factor of ten) in order to render that frame."

      I can give you a little data here. Take a look at this image I made. The scene is roughly 1.5 million polygons, and virtually everything is textured. The folder containing the bare bones version of this scene is roughly 600 megabytes. I could probably cut that size in half via JPEG etc, but we're still looking at a massive amount of data to send to one person to render a frame. I know this because I seriously discussed sharing the rendering with a friend of mine on the east coast. We both felt it'd take longer to ship than it would to render.

      I doubt this scene is anything close to what they were doing in Shrek 2, let alone whatever will happen with 3.

      --
      "Derp de derp."
    6. Re:Doubt it'll happen... by Rothron+the+Wise · · Score: 5, Insightful

      I've been unable to dig up the reference, but I read in an article about Pixar's "Monster's Inc." that for some frames it took longer to load the geometry than actually rendering the frame.

      SETI and Folding@Home work because of the massive asymmetry between the amount of data and the CPU power required, and although you _perhaps_ could find subtasks that could easilly be "offsourced" so to speak, that made sense performance wise, I very much doubt that it would interface very nicely with the way the artists work, or make any sort of economic sense.

      --
      A witty .sig proves nothing
  2. Making things worse by Baron_Yam · · Score: 5, Funny

    Don't animators already insert single-frame porn, etc into these things?

    Can you imagine how quickly the client software would get hacked, and how crappy the movie resulting from nothing but single-frame porn shots would be, especially to photosensitive epileptics?

  3. This made the front page? by jbellis · · Score: 5, Informative

    Easy: Pixar and Dreamworks have both developped highly proprietary rendering technology. They're not about to just give copies to everyone who wants one. Even if the renderer itself weren't reverse-engineered, which isn't beyond the realm of possibility, it would likely be far easier to decipher the protocol used and voila, a functioning copy of [Pixar|Dreamworks]'s renderer.

    Lobotomizing it to the point where this wouldn't be useful would probably make it useless for distributing the workload as well.

  4. Slashdot BLOG advertising... by Anonymous Coward · · Score: 5, Insightful

    Nice to see you can advertise your NEW BLOG on slashdot...

    how much did it cost?

  5. Data by Skarz · · Score: 5, Interesting

    The problem with trying to help render frames is that your system needs to have the data to do it (3D objects, textures, etc.)- not to mention the renderer. Companies wouldn't take kindly to sending off their IP data (esp. custom 3D models/textures/shaders) to the masses to be hacked. Having people get a hold of the "official" Shrek models and textures for example would be a bad thing.

  6. do you really want this? by jfroebe · · Score: 5, Insightful

    Do you really want the MPAA to run programs on your computer?

    --
    No one has seen what you have seen, and until that happens, we're all going to think that you're nuts. - Jack O'Neil
  7. I doubt it... by millahtime · · Score: 5, Funny

    I wonder if you could get a section of the frame(s) you (helped) to render...

    /.ers would combine their powers and probubally have a lot of the movie weeks before it was released.

    --
    Evolution or ID?
  8. The reason why..... by reality-bytes · · Score: 5, Insightful

    The main reason they don't employ this technique is that their own 'render-farms' are a known quantity; they can, with reasonable accuracy, calculate how long a given scene will take to render, whereas with public distributed computing this calculation is not possible.

    There are many variables in distributed public computing such as:

    *Different CPU capabilities.
    *Different OS capabilities
    *High/Low use Systems
    *People's 'uptime'
    *Users leaving the project before its completion etc.

    Another risk is that another movie-house could start a production which everyone sees as 'cooler' and your entire userbase decides to up-sticks and render for them instead.

    --
    Ripping an new rectum in the fabric of spacetime.
  9. How cool would the other way be? by jmpresto_78 · · Score: 5, Insightful

    How cool would it be to see them allocate THEIR distributed system to projects like SETI, etc. Even though I'm sure there are other projects being worked on, one would imagine the system is pretty dormant after a release.

  10. You don't have the machine for it... by Obasan · · Score: 5, Informative

    Only the most high end of machines could even consider attempting to render even one layer of a frame for this kind of animation. We're talking systems with 2-4GB of RAM as a minimum (preferably 4+) and the scene files/textures would weigh in the tens to thousands of megabytes that must be downloaded for each scene. Think uncompressed TIFF or TARGA texture files that might be 5000x5000 at 40 bits/pixel.

    Even on high end machines they often do not render a full frame, but a layer of a frame which is then composited with other layers into the full frame. Why? Many reasons but one of them is that even the high end machines don't have enough RAM and the render would take too long (the machine would need to swap).

    So aside from the issues of fans returning bogus data, or extracting highly proprietary information out of the client as other threads have mentioned, this would be a real show stopper. Breaking the problem into small enough pieces to be handled by joe-blow's computer would be prohibitive and require tons of calculations to figure out which pieces of textures are actually required for a given piece of rendering etc. It would probably require a compute farm just to manage it!

    Rendering is also a lot more complex than you might think, there are render wranglers who manage the rendering queues and look at the outputs... many renders may require specific versions of the rendering software, so a frame that rendered with 2.7.2.1 won't render anymore without errors with 2.7.2.2... so many copies of the software are managed in parallel with the wranglers helping to clean up the errors. How would you manage this in a distributed client environment?

    Furthermore most of the proprietary rendering apps are certified against VERY specific platforms, eg. one specific kernel version and build level, specific versions of shared libraries etc.

    Long and short is there's a reason why movies cost millions. :)

  11. ...a whole new world by steveha · · Score: 5, Interesting

    Shrek2 just shattered all kinds of records [...] And there are no real actors.

    You do still need voice actors. With an animated feature, a really good voice actor can really add to the experience.

    And you still need to make the character models move in realistic ways. So you need motion capture actors, or else truly skilled "puppeteers" to animate the models.

    All that said, I actually agree with you. Take a look at Killer Bean 2: The Party by Jeff Lew. One guy made this, using his computer at his home. I think it's really cool that people can just make movies now with only a tiny budget.

    steveha

    --
    lf(1): it's like ls(1) but sorts filenames by extension, tersely
  12. Rendering time by Gitcho · · Score: 5, Informative

    How would you ever reproduce this on a distributed network of limited bandwidth home PC's ? Here's some LOTR rendering stats from Wired.com - [http://www.wired.com/wired/archive/11.12/play.htm l?pg=2]

    ... The Return of the King, which opens in theaters December 17, will feature almost 50 percent more f/x shots than The Two Towers and will be composed of more data than the first two movies combined. Churning out scenes like the destruction of Barad-dûr and the Battle of Pelennor Fields (with thousands of bloodthirsty CG Orcs) took 3,200 processors running at teraflop speeds through 10-gig pipes - that's one epic renderwall. What else went into making Frodo's quest look so good? By Weta's account, more than you might think.

    WETA BY THE NUMBERS

    HUMANPOWER
    IT staff: 35
    Visual f/x staff: 420

    HARDWARE
    Equipment rooms: 5
    Desktop computers: 600
    Servers in renderwall: 1,600
    Processors (total): 3,200
    Processors added 10 weeks before movie wrapped: 1,000
    Time it took to get additional processors up and running: 2 weeks
    Network switches: 10
    Speed of network: 10 gigabits (100 times faster than most)
    Temperature of equipment rooms: 76 degrees
    Fahrenheit Weight of air conditioners needed to maintain that temperature: 1/2 ton

    STORAGE
    Disk: 60 terabytes
    Near online: 72 terabytes
    Digital backup tape: 0.5 petabyte (equal to 50,000 DVDs)

    OUTPUT
    Number of f/x shots: 1,400
    Minimum number of frames per shot: 240
    Average time to render one frame: 2 hours
    Longest time: 2 days
    Total screen time of f/x shots: 2 hours
    Total length of film: Rumored to be 3.5 hours
    Production time: 9 months

  13. Sounds like imp.org by ron_ivi · · Score: 5, Informative
    The Internet Movie Project has its renderfarm software on sourceforge

    My big question is why would you rather donate to a large commercial organization well funded from it's previous Shreck flick -- rather than donate the cycles to a project like the IMP works themselves?

  14. Re:NOT proprietary rendering technology by Whalou · · Score: 5, Informative

    Renderman is not a renderer it is a specification for interoperability between modeling tools and renderer (like XMI for software engineering tools except that it works.)

    Pixar's renderer is actually PRMan.

    From Renderman.org:
    There are a lot of people when you hear them talking about RenderMan and how great the images are from it, etc. They are most likely really talking about Pixar's PhotoRealistic RenderMan® (PRMan).
    RenderMan is actually a technical specification for interfacing between modeling and rendering programs. From 1998 until 2000 the published RenderMan Interface Specification was known as Version 3.1. In 2000 Pixar published a new specification, Version 3.2. Coming soon Version 3.3

    --
    English is not this .sig mother tongue...
  15. Further elaboration on the impossibleness of this by anti_analog · · Score: 5, Informative

    I believe I saw someone earlier mention how there can be terabytes of data go into a single frame of CGI film, and these days that can be pretty correct.

    A .rib file or similar type file for PDI's renderer will probably contain a few million polygons and/or a few hundred thousand control verticies for implicit surfaces such as nurbs and sub-Ds, which can be a lot of data (my scene files at work average 4-5 million polygons and are about 150 megs on average, saved in a binary file format). And, that doesn't include particles, procedurals, all the motion data so that proper motion blur can be calculated...

    And then the textures... They do use lots of procedurals, but they also use lots of 16 bit per channel textures of 4000x4000 for face textures, or even higher. Some people are using tiles if 16 bit tiffs for displacement maps now that equate to like a 100,000x100,000 image for displacement maps, because the accuracy requirements for close up renders are so bloody high. That can be many many gigs of data there.

    And, if you're raytracing like in Shrek 2, then you need to have as much of that data in RAM at once, or else render time spirals out of sensibility, unlike scanline renderman where swapping is easier, because the rays bouncing throughout the scene make scene divisions more difficult (but still possible).
    I work with 4 gigs of RAM and we can just barely render 6 million polygons + a few 4k displacement maps all raytraced at once (in windows unfortunately). And, when we render sequences and stuff, we often almost kill our network because distributing all this data to just 20-30 rendernodes is pretty tough (and how would that scale to a big renderfarm with thousands of rendernodes...)

    So, yeah, like everyone else is saying, bandwidth limitations and that people running the screen saver probably don't have the hardware and OS to really run 4+ gigs of RAM, this Shrek@home idea seems rather unlikely. It would be cool though, if it worked...

    Hooray for my totally unoriginal post!

    --
    you cannot dodge the quad laser. jumping is useless.