Slashdot Mirror
Cloud Computing Democratizes Digital Animation
kenekaplan writes "John McNeil is the chief creative officer and founder of a digital arts and communication company based in Berkeley, CA. After turning to Amazon's Elastic Cloud Computing service for the first time to finish animation under tight deadline, he was impressed by how it would let him compete with bigger studios. He said, 'Cloud computing is the first truly democratic, accessible technology that potentially gives everyone a supercomputer...it's a game changer. I could never compete or be able to deliver something at the level of a Pixar or a Disney, given what I have at my disposal inside the walls of the studio,' McNeil said. 'But if I factor in the cloud, all of a sudden I can go there. And then the limitations of whether or not I can deliver something great will be on my own talent and the talent of the people that are part of the studio.'"
"And then the limitations of whether or not I can deliver something great will be on my own talent and the talent of the people that are part of the studio." ... and also how much money I can put. Using a massive computing power on a cloud requires a lot of money.
Compute resources don't come for free, you pay per use. You'll only be able to harness the Cloud if your business is sustainable. But if it is, then you could afford to buy compute resources anyway -- albeit in a smaller fashion. The only real difference is that with cloud services you can save some money if you don't run jobs 24/7.
Computer simulation made easy -- LibGeoDecomp
Rendering projects tend to be quite small in their models form. When utilizing a render farm you really don't have that much I/O after the initial project upload. granted a big project (a movie) might have gigabytes of texture data but still nothing that can't be uploaded in half a day or so. After that it's all internal I/O (which in ec2 is very fast), the control server telling the nodes what to render and providing render assets and the nodes just returning rendered products and requesting new tasks. After the task is complete you just have to download the rendered products and you are good for post processing :-)
Also, I don't think your argument stands. It is very affordable if you are in that line of busyness. Hell if you are a hobbyist and have a couple hundred € to burn, you can run a mini jaguar for the weekend, just to get a feel for it.
-- no sig today
Let's not forget, you can render a really nice smooth teapot for peanuts, but you can't tell a good story, choreograph a chaotic rescue scene, or draw the expression on the protagonist's face when a little old lady kicks him in the nuts. You also have no chance of getting Jack Black or Angelina Jolie as voice actors.
Pixar is not just a render farm. It's also a studio. The South Park guys showed you can still beat a studio if you are willing to target a new demographic (people who liked The Simpsons, and want sex jokes as well as fart jokes) and have talented actors and writers, but it's not easy.
I doubt there is much difference between running your own computer or sharing a supercomputer that is roughly ten thousand times as powerful with at least ten thousand people. Not to mention the bottlenecks caused by the internet connection you need to tell it what to do, and get what it has done back from it.
Err. Not sure you grasp this.
Computer A and Computer B
Computer B is 10,000 times more powerful but is shared with 10,000 people
The key difference is timing and time. If all 10,000 people want to do the same thing that they do on their A computers on computer B at the same time then I agree with you. However that is not real life. If the distribution in time of tasks from the 10,000 users is spread out then my task will run up to 10,000 times faster on computer B than computer A.
If I could use computer B all the time there would be no point in sharing it. If I don't (and I assume all the others sharing computer B can't use it all the time either) then there is a benefit to me using computer B since my task will complete faster, maybe much faster.
As far as internet traffic is concerned I assume that the task I want to do on Computer B is compute intensive on local data, like ray tracing / animation.
Does the CEO realizes that he's trading CPU's limitation against bandwidth's limitation ?
Generating a picture in full HD requires 1920 x 1080 = 2,073,600 pixels.
But for a movie, the resolution is 4096 x 3112 = 12,746,752 pixels
http://en.wikipedia.org/wiki/4K_resolution
This gives 36 megabytes per picture.
Now, you have to create 25 pictures for one second.
You get 5400 megabytes for every minute of movie.
It may be faster to compute digital animation, but you still have a large IO problem, both in storage and in preserving the data (you may lose pixels when downloading the files) !
It's similar to outsourcing tasks: it's a short-term solution for larger problems.
I think the SOPA blackout is useful because it helps inform the general public who is unaware of the problems and ramifications of SOPA and it's three vile siblings.
I doubt there is anyone on slashdot who is unaware of the ramifications, even those of use who support some copyright stuff, and who are against piracy, are aware that SOPA is a dangerous piece of legislation that will only harm society, and not help any artist.
Self proclaimed typo king, and inventor of the bear destroying coffee table (patent not pending).
Yeah, but they don't need to maintain the systems for all the time they don't use them, which the big boys can afford to do (usually because, with multiple projects, they'll be able to have much less downtime).
Ex: You need about "100 units" of CPU time per year. A computer that does this costs $500. Now, lets say, you have about a month from sufficient data to start, to the deadline. Now, you need a computer that can provide "1200 units" per year. This will probably be closer to $6000. And part of that money goes towards having it for 11 months where you don't need it. You might pay more ($1000 maybe?) to get the job done in a month, than with a computer that, given a year, could do it, but you get it done on time, which is probably worth that extra $500.
It may not be ideal for everyone (or even most), but for a smaller player to move up in the world, when they can't afford to have enough projects simultaneously running, that would make maintaining their own desired system financially viable, then this kind of CPU timesharing, is not a bad idea.
Self proclaimed typo king, and inventor of the bear destroying coffee table (patent not pending).
Studio Pyxis (www.studiopyxis.com) is a Burbank based production company that is taking full advantage of cloud computing, both for our own productions and for sharing with clients. The company is a new model of production company leveraging years of development in real time technology such as the Virtual Production process used on Avatar which involve real time graphics and visualization so that directors can shoot visual effects interactively as if they are really happening in front of you, and cloud computing that can complete the photorealistic renders on the back end in record time.
One feature of cloud computing that is often over looked though in these production discussions is the breadth vs depth computing model. It's obvious that it's a value to have a massive cold room that you don't have to buy up front, but the real advantage comes when it costs exactly the same thing to run 1000 cpus for 1 hour as it does to run 10 cpus for 100 hours.
Visual effects and animation production is all about revisions. It's a huge win to have your full renders back sooner. Being able to run every frame of a shot at once regardless of how many frames you have means that you have the entire shot in the time it takes the longest frame to render. This has never been possible before. Production has always wanted a dynamically scalable solution but as always had to contend with some fixed capacity. Granted EC2 has a fixed capacity as well, but it is so much more massive then a typical production facility as to be a non issue.
As for what some commenters are saying about bandwidth issues it is true it's a factor, and this is why it's not a turn key solution for the average small company. We've spend a fair amount of time creating an infrastructure that mirrors assets in the cloud, renders and composite locally to the cloud, then generates compressed images and movie files for download at review. Only when we approach the completion of a shot do we download actual exr, or dpx data. But we do make our infrastructure available to other companies to help them be more turn key.
Another aspect that more then democratizing cg production actually gives an advantage to the smaller facility are the limitations that larger facilities working on mainstream studio pictures have such as MPAA rules about keeping film assets off the internet and/or on physically disconnected machines. Whereas small facilities like ours can be satisfied with a VPN connection to Amazon, larger facilities are often legally obliged not to.
The one area that still needs to be solved to truly make this work for everyone is for the software companies to start offering the same type of pay as you go licensing so that we can more easily use the professional tools. It would be relatively easy for a company like Pixar to offer a RenderMan license server that one could connect to over the internet or even EC2 based that would monitor your hourly usage. Are you listening Pixar?
If I'm afraid you're going to "compete or be able to deliver something at the level of a Pixar or a Disney", couldn't I just get your ISP to block you off from the cloud because I don't feel you're doing enough to prevent Pixar/Disney intellectual property from being incorporated into your work?
My turnips listen for the soft cry of your love
I actually did the same thing for some projects I was working on... used Amazon's EC2 as a Maya/3DS Max Backburner renderfarm. I posted some tutorials on how I set it up on my website: http://www.judpratt.com/tutorials/ec2-renderfarm/
EC2 let me render in 5 hours what would've taken my own computer about 110 hours to render. The cost came out at about $.06 per core hour. The commercial cloud renderfarms charge about $0.75 core hour for comparison.
Web hosting that doesn't suck!Dreamhost
Granted a big project (a movie) might have gigabytes of texture data but still nothing that can't be uploaded in half a day or so.
I measured this a couple of years ago. At the time, you could ship data to S3 or EC2 at about 10GB/hour provided you're not saturating your own network or doing it at a very busy time. For a truly large amount of data (1TB up) I suppose you'd just FedEx a hard disk...
"Little does he know, but there is no 'I' in 'Idiot'!"
Commercial render farms have been around for years, long before "cloud" computing. Search for "render farm" to find some of them. They compete on price, so the pricing is good. The concurrency generally consists of running one frame on each machine, so the intercommunication during rendering is zero. Pixar was doing this by 1995, using a set of shell scripts called "Ringmaster" to push the data around. If the CEO of an animation house just discovered this, he's way behind.
Rendering isn't the bottleneck on cinema projects. People are. Look at the full credits at the end of any modern animated production, and watch as a thousand names scroll by. There's an army of people drawing background objects, landscapes, crowds, and fine detail. That's where the cost goes.
It hasn't improved much in the last decade, either. A decade ago I knew a director who'd done some feature films with mixed animation/live action, and he was hoping the technology would get the cost down, so he could do a feature for $20 million instead of $80 million. A decade later, budgets for A pictures are up, not down. "Tangled" came in at $260 million. Which is why what gets green-lighted is usually a known franchise.
If you're willing to drop to video game levels of quality, animation can be really cheap. See Next Media Animation. Fastest production house in the world.
These days it's even bigger. In 2007, in Transformers, rendering a single frame with one bot in it took hours. Rendering the whole gathering at the observatory took around 25 hours per frame. The total models only took around 2TB or so of data.
The 2011 final Transformer film though exceeded that data amount - I think it was aorund 200TB or so, and despite newer equipment, render times stayed the same - 8+ hours for a normal frame, days for a more complex one.
The only good part is it's a stupidly parallel problem since each cluster can render a frame independently of each other.
These days it's even bigger. In 2007, in Transformers, rendering a single frame with one bot in it took hours. Rendering the whole gathering at the observatory took around 25 hours per frame. The total models only took around 2TB or so of data.
These numbers would mean that even if you leased just one rendering node (capable of rendering a frame per day - not very useful), you would still need to be pushing 200 Mb/s of scene data into it on average - over the Internet. Now perhaps your animation studio has such an external backbone access, but multiply it by two thousands if you're doing a movie and not just a couple of stills and we're talking about +100Gb/s lines from your company to the Amazon service. Yummy...
Qa'pla!
Ezekiel 23:20
We used spot instances for most of the project, averaging a bit more than 50 cents per hour. Spots are great for rendering because we didn't mind getting outbid - we'd just do that frame again. The hard part was writing the script that tied it all together.