Slashdot Mirror
HP & Commodity Computing
Handpaper writes "The BBC has a story about HPs SE3D lab's pilot scheme to provide raw rendering power for smaller studios and amateurs. A sample movie is available.. " Yes, the long fabled "grid computing" may arrive soon on a massive scale.
Is this considered grid computing (which enables the sharing, selection, and aggregation of a wide variety of geographically distributed computational resources)? All processing seems to come from one source, more like lease-computing?
One of the articles mentioned "All the animators are independently funded to make their films.", but none mentioned the cost to use 1 unit (however it's calculated) of the processing power. How do animators justify the expenses to sponsors?
Rock that crushes, Paper & Scissors that don't matter.
Since they have the hardware in place, I wonder what they do when they don't have films to make and/or work - i.e. would they consider contributing those idle CPU times to something like the Folding@HOME project ... the powder2glass team would love the work units! ;-)
Hulk SMASH Celiac Disease
I know that data security is a top concern in today's animated film business. This is why render farms for animated films are in secure office buildings rather than data centers. Additionally, the render farms are not networked to the internet.
This makes me skeptical that a 'lease farm' model can work for anything more secure than things like TV Commercials.
Guys, many schools already have the PC power for grid computing, my school (university of minnesota) has over 20,000 PC's! Now how would you like to tap into that type of power? It is just a matter of PR, trends, and software. Most Pc's are just sitting idle anyway, why not use them? I just cannot speak for what type of power bill the school will be footing then ;-).
I just hope it is TRUE grid computing - as in following a standard communcation so that any application that is grid compliant can take advantage of the farm.
More often than not renderfarms have a few formats that you can use (usually expensive ones) and that's it. PR-Renderman, Maya, 3dsMax, Lightwave, and a few other big ones are guaranteed to run about everywhere. Blender? Nice try, but "little" software projects like Blender don't have much of a chance at a renderfarm. Tell a renderfarm you have a Blender file to render, and that you have money in hand - they will tell you to go home.
Grid computing is slightly less efficient than a direct rendering program since it has a little extra overhead. But the ability to run ANY program on a farm is quite enticing. That's what we should be aiming for. That is a good goal.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
We have things to do to keep the facility busy; its a flexible fabric and it can do interesting stuff. To be honest, pure CPU cycles is not its strength -many home computers can deliver more mips.
What the farm(s) have is large amounts of storage near those mips, which is what you need for data-intensive computation. Large animation models is one use. data processing from things like the Large Hadron Collider another, though we wont have real data from the LHC yet.
Now, if you have projects to run on such a fabric come talk your nearest HP sales rep.
The stuff we do in HPLabs is focused on 'research on how to use these systems'; things like resource allocation, load balancing for thermal management, etc, etc. I'm working on distributed deployment and testing, using the datacentres and perhaps soon the PlanetLab facility, which has more distribution for interesting problems.
If you want to play with the deployment tools, to http://smartfrog.org/ and download it. The technology is designed to make it possible to install and configure complex systems over a utility computing infrastructure.
I have yet to see a service where you can just simply buy time anonymously, upload your program, crunch the data, download the results.
I know some might say that this might be invitation for someone to unleash zombies, but the grid does not need a connection to the internet for this kind of work, and besides that outgoing traffic can be monitored.
I did not notice what the process was for buying computing time on the SE3D website, nor sepcifications of the grid. Am I missing something? Is this a fluff/vaporware story?
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
All the professional 3D apps e.g. Maya, Shake etc. are built to operate in this kind of 'decoupled rendering' environment, and render farms are probably the simplest type of 'distributed computing' - generally theyre just a bunch of independent computers on a LAN which see a shared disk on which the source materials (e.g. textures, models, motion data, imagery) is kept.
Accounting is pretty simple too as a central dispatcher keeps track of which frame is submitted to which node, and how long it takes - it is uncommon to have 1 machine working on more than 1 frame simultaneously.
Sun also offer this type of service, as well as others. I think it will be interesting to see if this model is adopted by hollywood etc. but I can see a lot of smaller shops taking advantage of it to acheive what would otherwise be impossible.
I remember using a Compaq Test-Drive-Programme 4-CPU Alpha 21264 box to render a 4096x4096 cornell box using BMRT that would have taken my 128MB P3-500 an age to do.
The model definitely works from my point of view, however I don't really think that Sun or HP will be in much of a position to actually make money out of this type of service since hardware depreciates so fast and the price people will be willing to pay would be pretty low I imagine - I guess they could simply use their unsold inventories to extract some kind of revenue from them instead of writing them off, but getting the 'volume' up to profitable levels will be a challenge.
I gots ta ding a ding dang my dang a long ling long
That is, CPU & storage is something you pay for on demand. It is distributed (two facilties, two continents), and the users are all scattered round the UK. One of the facilities is downstairs from me in Bristol; a large chilled datacentre that I dont have access to for either security or competence reasons :)
Projects aren't charged real money for render time, do have virtual monety; cost of rendering varies depending on demand. Projects have their own balances. This lets money optimise the resource allocation. Get your design done early, render before the rest and you get more CPU time for your cash, hence better rendering.
There are lots of other sponsors, as this is giving regional and up and coming artists/animators chances to do serious renderings on facilities they wouldn't normally get.
"Could someone not write something like this but open source and distributed."
d ex .html
l
n tr o.html
Darwin:
http://www.opensource.apple.com/darwinsource/in
Xgrid for UNIX:
This article introduces the first working Xgrid agent for Linux and other Unix systems that can be integrated in any XGrid cluster (managed by OS X).
http://unu.novajo.ca/simple/archives/000026.htm
http://developer.apple.com/hardware/hpc/xgrid_i
~hylas