Slashdot Mirror
Rendering Shrek@Home?
JimCricket writes "There's an interesting piece at Download Aborted about using distributed computing (a la SETI@Home, Grid.org, etc.) in the film industry. With the recent release of Shrek 2, which required a massive amount of CPU time to complete, one must wonder why the film industry doesn't solicit help from their fans. I'd gladly trade some spare CPU time in exchange for the coolness of seeing a few frames of Shrek 3 rendered on my screensaver!"
I wonder if that would be considered pirating by the MPAA. Smart people out there would figure out a way to "download" the movie from the frame generated. Then there would be no reason to see it in the theater. Just playing the devils advocate. Personally I think it would be REALLY cool :)
Also, there is bound to be a fan site created that would allow users to upload their rendered images and somebody would manage to piece it together into a halfway coherent movie. Then some nerd would mystery science theatre 3000 it and it would become an internet phenomenon. Hmmm, maybe that's not a bad thing...
01100111 01100101 01110100 00100000 01101111 01110101 01110100 00100000 01101101 01101111 01110010 01100101 00101110
The problem with trying to help render frames is that your system needs to have the data to do it (3D objects, textures, etc.)- not to mention the renderer. Companies wouldn't take kindly to sending off their IP data (esp. custom 3D models/textures/shaders) to the masses to be hacked. Having people get a hold of the "official" Shrek models and textures for example would be a bad thing.
We could have entire CVS trees of movies!
Don't like how Matrix Revolutions ended? Just load up the "Smith kills us all" branch and choose your own adventure!
<insert witty linux comment here>
I'm not a film tech -- but besides abuse and security issues, what's proposed here is just does not seem possible under low bandwidth conditions. it's not like you can just run off to computer #2,398 and say "go render frame 1,503" -- there are textures and models and state information that probably total somewhere on the order of gigabytes (give or take a factor of ten) in order to render that frame. Joe Dialup isn't going to be able to handle that; the film studios I'm sure have crazy fiber/multi-gigabit interconnects within their rendering farms.
If they could find a way to offload some intermediate calculations (like deformations of hair or fabric or something that can be used as an intermediate result in a scene) then that might be a clever use for a distributed.net style technique.
-fren
"Where are we going, and why am I in this handbasket?"
You make valid points although most (maybe all) of your points could be eliminated by having multiple hosts render the same frame (a la SETI's response to people uploading false data).
If you send the same input to three different IP addresses (extra-paranoid: use three different top-level IP blocks) and get the same result back, you can be reasonably certain that the result is valid. If there are -any- discrepancies in the images, assume that one (or more) was improperly rendered, discard all three, and try again with three new addresses.
Even should you manage to hit three different IP addresses that return the exact same 'hacked' image, it's not exactly hard for an editor to step through the movie frame-by-frame, looking for discrepancies...
Obliteracy: Words with explosions
In theory this is a good application for Distributed Computing. I doubt it would ever happen in practice. Maybe for an indie but not for Pixar. The funny part would be someone hacking one of the scene and throwing something else in in it's place. Kinda like the part from Fight club.
500 dollar reward for tip(s) leading to the arrest of the person(s) who stole my sig.
Users gladly contribute their spare CPU cycles to fold proteins for a non-commercial purpose, or help a non-profit organization seek out alien life. These are tasks affecting all of mankind.
Giving away CPU cycles so that a multi-million dollar company can improve its product is a wholly different thing.
Great, maybe for saving some effects company from shelling out for a few more $10K graphics servers with which they will make the next $150M movie, perhaps I can loan them a few CPU cycles and they'll cut down my move ticket cost from $10 to $9.75.
The last film I worked on, we had anywhere from 800MB to 12GB of data per frame that the renderer had to have. I am talking about compressed renderman rib archives, textures, normal maps, displacements, shadow and other maps.
The data was mostly generated at render time for things like hair and shadow maps, but if it was being distributed, there is no way to do that - they would be transferred beforehand.
Also, there are always many terabytes of data generated by the renderers for each render layer, for diffuse color, specular color, etc.
It is just not feasible to transfer all that data around, and its not like bittorrent or other p2p systems will help much with that since each frame would most likely only be rendered by a few people (for verification).
Also, the model geometry and shaders (and somtimes textures) are closely guarded secrets... In short, if a major film were ever to do somthing like this, everyone participating would need huge (> 100mbit) bandwidth and a LOT of disk space and also be under very tight NDAs.
But this opens up a whole new world to the independents. Shrek2 just shattered all kinds of records in terms of cash. And there are no real actors.
So what happens when a few talented indies get their paws on the processing power required to blow the doors off of convetional actors? It won't be goodbye to Hollywood just yet but I can't wait for the first CG/Anime crossover. I can't imagine how Cowboy Bebop would fare if it didn't have the cartoon stigma.
Life is the leading cause of death in America.
You might be the unfortunate soul that renders a movie spoiler as you watch the images coalesce on your screen ...
The heck with that, why would they want the 3D wireframe models to get out on the net? What do people think the frames are rendered from, anyhow? I predict it would be less than one week between someone figuring out how to extract the models, and someone else making a low-res animation of those models doing the nasty with each other.
--
"Open source is good." - Steve Jobs
"Open source is evil." - Microsoft
I wouldnt do it for the same reason I refuse to buy clothes with logos on them that advertise the maker of those clothes, or some other form of advertisement.
If someone wants me to wear such advertisement-enhanced clothes, they should pay me for the priviledge.
Same with computer cycles. I pay the electricity. If they plan on making money from the product of the cycles I give them, they should pay me.
However, I have no problem giving away free computer cycles to non-profit scientific endeavors.
Why have one distributing computing CLIENT for gene folding, one extraterrestrial searching and yet another for video rendering?
It might be a good idea to have a distributed computing SERVER (DCS) that can take and serve number crunching tasks to those that request them.
Some good ideas might be the following:
Build a mesh of DCS' by using current P2P technologies so that one DCS can know about a good number of others.
DCS' initially (might) not provide services to every user by default; it is up to the owner to choose who to allow to use CPU power and how much.
--
After all, the CPU power you don't use goes wasted.
(If you're not a power saving aficionado.)
- "They misunderestimated me."
I think people have hit on a number of reasons this idea just wouldn't make sense. I'll add one more:
The cost of a server farm is a small fraction of the cost of making a movie.
A close friend of mine is in charge of the server farms for a feature film animation company. For their next film they sat down and considered the hardware architecture they wanted, bought a few test machines, then said okay send us a few hundred PCs.
They didn't even care (much) how much the PCes costed, or that they might want to reuse hardware from the previous movie. For them, new movie = a bunch of new hardware - no big deal.
The only way I see for this to be feasible is if each user was only rendering a tiny segment of each frame. I don't know if this is technically possible, but, it would reduce the massive bandwidth needs to a more SETI like level.
Secondly: Users cannot see what they have rendered. This is a given, as has been pointed out a thousand times already, this is insane from a security and PR standpoint. INSTEAD, simply let users who participate on a regular basis have access to a private forum, developer blogs, and grant them access to the official PR material slightly before it gets published. It's less cool, sure, but it could work.
If the resulting product from the distributed computing were public property, or for the public good-- then I could see it.
But to offload all the costly/time-consuming work and THEN let them sell it back to you 4 or 5 times (Theater, Pay-Per-View, DVD Rental/purchase) along with all the marketing tie-ins doesn't seem to make sense (to the consumer, that is. For them it's great, less equipment overhead and quicker turn-around.)
If you think that's a good idea, I've got some swampland I can sell you.
This type of thing makes absolute sense for things that might not otherwise get funding and don't derive a profit (SETI@home, Folding@home, etc.).
If you want to get paid next to nothing animating scenes for a giant studios next big movie, get a job at some asian animation house.
Shrek2 just shattered all kinds of records [...] And there are no real actors.
You do still need voice actors. With an animated feature, a really good voice actor can really add to the experience.
And you still need to make the character models move in realistic ways. So you need motion capture actors, or else truly skilled "puppeteers" to animate the models.
All that said, I actually agree with you. Take a look at Killer Bean 2: The Party by Jeff Lew. One guy made this, using his computer at his home. I think it's really cool that people can just make movies now with only a tiny budget.
steveha
lf(1): it's like ls(1) but sorts filenames by extension, tersely
I can't wait for the first CG/Anime crossover
Uhh, there have already been some. The first one I've been looking for (Appleseed) is coming out in Japan soon, and it looks quite badass. I've heard of a few crossovers before now, but can't think of any off the top of my head.
I've got more mod points and GMail invi
From: http://www.aspenleaf.com/distributed/ap-art.html#i mp
The Internet Movie Project renders images for computer-animated movies. The project is an open-source collaboration of volunteers and is just for fun. It is still in the development phase, but you can volunteer to be a "render-farmer," to render images for test animation sequences. Anyone who can run the free POV-Ray ray-tracing program can join this project, although the supporting scripts and software needed for the project only work on the Windows and Linux platforms for now.
Pixar renders at insanely high resolutions. pretty close to the theoretical limit of 35mm film.... then their secret and house designed laser film printing process creates a film master that makes everyone elses prints look like utter garbage.
on your 1400X1200 screen you might get an Eye or fingernail.
think 10,000 pixels by 40,000 pixles in 32 bit color.
Sony Imageworks definitely didn't think gigabit was fast enough, and that was 6-7 years ago when I talked to them. They were deploying some sort of customized super-HIPPI setup for shipping their digital assets around.
Now, it might make sense to go to some rental service that can set up the licenses and servers and just charges per CPU hour, license hour, and MB on the server. It wouldn't be a small thing to set up, though, and it would be a professional service with strict NDAs.
For Shrek, and according to Dan Wexler's statistics at http://www.flarg.com/Graphics/ShrekRenderfarm.html , on average each frame took 2GB of data. I'm not sure and won't bother to find out how much we used on Shrek 2, but of course chances are the figures are even bigger.
This alone would make it impractical, without even taking into account all the IP and copyright issues.
j
If you send the same input to three different IP addresses (extra-paranoid: use three different top-level IP blocks) and get the same result back, you can be reasonably certain that the result is valid.
Actually, just double. First use "Comparison Mode". If the two come back different, resolve it by switching to "voting mode", doing a third frame at a third site and seeing which it agrees with. (If all three disagree you've got a systematic problem and you need to debug the whole project.)
If there are -any- discrepancies in the images, assume that one (or more) was improperly rendered, discard all three, and try again with three new addresses.
A disagreement proves that at least one of your sites had a problem. But (unless there's some non-randomness to your selection of the three sites) the fact that ONE of your first set had a problem does NOT mean that the OTHER sites in your first set are any more likely to have misrendered than the new sites. So there's no reason to throw away the results from your first set. Just use the additional site(s) to select which one of your first set was wrong.
If a significant fraction of your sites are compromised to make identical misrenderings, you still don't want to throw away your original set. Keeping them all and going to 5, 7, 9, etc gets you closer to the likelyhood of the right answer, while discarding them and repeating a 3 site run with new sites gives you a greater probability of accepting the wrong answer.
= = = =
But all redundant systems fail if you have a systematic problem that may affect more than half of your potential samples. For instance: A virus infection that causes a systematic corruption of the rendering process (i.e. changing the texture map so the pants are transparent or carry an image of genitilia). If more than half render wrong than right, more redundancy makes you accept more wrong frames.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
More likely than that, I can totally see hackers sending in images that don't belong, right smack in the middle of valid rendered images. Think along the lines of Fight Club...
This is not the greatest sig in the world, no. This is a tribute.
It already has for Matrix and LOTR 1 and 2
I've seen LOTR 1 and 2 and they're hillarious. You have to know Russian pretty well though to understand all the humor.
Can't wait for LOTR 3 to come out
what would be distributed is the RAW calculations. Instead of sending the calc to the local CPU, it would get passed to the 'cluster' of web machines. the rendering would send all calcs out, and then get them back and assemble the results.
this way no actual creative content would leave the building.
will deal with that. You may seen a generic screensaver, but I doubt that a company would risk leaking their movie. However, it is possible to perform certain calculations (such as addition and multiplication) in a way such that the clients in this scenario would work on encrypted values.
The calculations are done in encrypted values and returned as such. The host can then decrypt the result.
This sounds pretty amazing but consider addition as a starter: The host uses a one-time-pad for each number and XORs them. The client adds the encrypted numbers. When you add the numbers, the host only needs to XOR all the keys on the result and gets the true result! The client, though, knows NOTHING about the true values (the protocol is information theoretically secure), as the XOR turns them into "signal noise".
I imagine, though, that the effort of implementing this probably outweighs the benefits for a project like rendering a movie. But for truly mission-critical data, it may be worth it...
Allow movie companies access to the virtual farm at a rate of 1 cent per frame or something like that. Then the person who renders the frame or who has contributed X number of render units has the option to have the micropayments sent to a charity, open source project or site of their choosing.
more than likely it will be limited to using all the PC's on the employee's desks once they log off and go home for the night.