Slashdot Mirror

← Back to Stories (view on slashdot.org)

HP & Commodity Computing

Posted by Hemos on from the the-grid-network-cometh dept.

Handpaper writes "The BBC has a story about HPs SE3D lab's pilot scheme to provide raw rendering power for smaller studios and amateurs. A sample movie is available.. " Yes, the long fabled "grid computing" may arrive soon on a massive scale.

17 of 64 comments (clear)

  1. Grid-Computing? by fembots · · Score: 4, Insightful

    Is this considered grid computing (which enables the sharing, selection, and aggregation of a wide variety of geographically distributed computational resources)? All processing seems to come from one source, more like lease-computing?

    One of the articles mentioned "All the animators are independently funded to make their films.", but none mentioned the cost to use 1 unit (however it's calculated) of the processing power. How do animators justify the expenses to sponsors?

    --
    Rock that crushes, Paper & Scissors that don't matter.
  2. Wonder what they do with their "idle" CPU cycles by xmas2003 · · Score: 4, Informative

    Since they have the hardware in place, I wonder what they do when they don't have films to make and/or work - i.e. would they consider contributing those idle CPU times to something like the Folding@HOME project ... the powder2glass team would love the work units! ;-)

    --
    Hulk SMASH Celiac Disease
  3. How about security by wheelbarrow · · Score: 4, Interesting

    I know that data security is a top concern in today's animated film business. This is why render farms for animated films are in secure office buildings rather than data centers. Additionally, the render farms are not networked to the internet.

    This makes me skeptical that a 'lease farm' model can work for anything more secure than things like TV Commercials.

  4. They have the power scotty by Ambient_Developer · · Score: 3, Insightful

    Guys, many schools already have the PC power for grid computing, my school (university of minnesota) has over 20,000 PC's! Now how would you like to tap into that type of power? It is just a matter of PR, trends, and software. Most Pc's are just sitting idle anyway, why not use them? I just cannot speak for what type of power bill the school will be footing then ;-).

  5. Finally.... by Viceice · · Score: 2, Funny

    The thing that is going to save (slacking) animation students who finished animating their piece 1 week before deadline but need 1 month to render on their workstation.

    --
    Sometimes I wish I was a plumber, then I'd know how to deal with other people's shit.
  6. True Grid Computing by Doesn't_Comment_Code · · Score: 4, Interesting

    I just hope it is TRUE grid computing - as in following a standard communcation so that any application that is grid compliant can take advantage of the farm.

    More often than not renderfarms have a few formats that you can use (usually expensive ones) and that's it. PR-Renderman, Maya, 3dsMax, Lightwave, and a few other big ones are guaranteed to run about everywhere. Blender? Nice try, but "little" software projects like Blender don't have much of a chance at a renderfarm. Tell a renderfarm you have a Blender file to render, and that you have money in hand - they will tell you to go home.

    Grid computing is slightly less efficient than a direct rendering program since it has a little extra overhead. But the ability to run ANY program on a farm is quite enticing. That's what we should be aiming for. That is a good goal.

    --

    Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
    1. Re:True Grid Computing by Dracolytch · · Score: 2, Interesting

      The ability to run any program on a farm doesn't make a lot of sense.

      I've studied and worked with distributed/parallel programming. There are a lot of things you need to take into account. Your primary troubles are how to divide up the work evenly and well, especially when you're working with multiple computers at different specifications.

      So, yeah, you have Blender, and you want to distribute it... But how do you tell blender to divide up the work? Do you have another program that manages multiple installations of blender?

      Other parts of the program, usually synchronization and organization, need to be run on one (or a few) places to prevent chaos.

      That's, honestly, the easy part. If you want a good system (versus getting by on the skin of your teeth), then you also need to deal with fault-tolerance (As the number of machines increases, so does the chance of failure).

      All of this assumes that the grid is designed for speed in mind (instead of fault tolerance).

      Writing for a distributed environment requires that you re-think the way that you approach the application at hand. You litterally have to program it differently.

      If you're into software, then I highly recommend that you do some research into distributed computing. It's a really interesting field.

      ~D

      --
      This sig has been enciphered with a one-time pad. It could say almost anything.
  7. We keep it busy by steve_l · · Score: 5, Informative

    We have things to do to keep the facility busy; its a flexible fabric and it can do interesting stuff. To be honest, pure CPU cycles is not its strength -many home computers can deliver more mips.

    What the farm(s) have is large amounts of storage near those mips, which is what you need for data-intensive computation. Large animation models is one use. data processing from things like the Large Hadron Collider another, though we wont have real data from the LHC yet.

    Now, if you have projects to run on such a fabric come talk your nearest HP sales rep.

    The stuff we do in HPLabs is focused on 'research on how to use these systems'; things like resource allocation, load balancing for thermal management, etc, etc. I'm working on distributed deployment and testing, using the datacentres and perhaps soon the PlanetLab facility, which has more distribution for interesting problems.

    If you want to play with the deployment tools, to http://smartfrog.org/ and download it. The technology is designed to make it possible to install and configure complex systems over a utility computing infrastructure.

  8. Alternatives?? by eclectro · · Score: 3, Interesting

    I have yet to see a service where you can just simply buy time anonymously, upload your program, crunch the data, download the results.

    I know some might say that this might be invitation for someone to unleash zombies, but the grid does not need a connection to the internet for this kind of work, and besides that outgoing traffic can be monitored.

    I did not notice what the process was for buying computing time on the SE3D website, nor sepcifications of the grid. Am I missing something? Is this a fluff/vaporware story?

    --
    Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
  9. The deployment tools are OSS by steve_l · · Score: 2, Interesting
    If you point your web browser at
    SmartFrog you can read about and download the framework used to dynamically deploy the fabric management tools and the rendering apps.


    It's LGPL, Java based.


    What you are thinking about, community rendering, needs community rendering tools. Bandwidth is an issue with all these apps; the filesets are huge.

  10. Re:OSS? by Doesn't_Comment_Code · · Score: 2, Informative

    Of course, security would be another issue. There would need to be some way to encrypt data, even when being processed on a machine.

    Renderman uses the concept of "buckets" to limit memory usage. It essentially renders on small part of the screen at a time. If you split the scene into many buckets (already done for you) and send each bucket to a different computer (like many renderfarms already do) you would minimize the security issue. You could "view" the image you rendered by hacking the software - but even then you'd only have a small fraction of the image.

    But before we even start talking about all this, we'd need to know HOW the software was going to communicate. Is it just sending raw scene information to be processed? Or is it sending computations to be processed at an abstract level like grid computing is supposed to? If you really are doing grid computing, your render nodes shouldn't even know they are rendering a picture, or be able to assemble their computations into a picture.

    Your biggest concern would be making the rendering software grid compliant in an efficient manner. It's all downhill from there.

    --

    Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
  11. Resource Allocation by steve_l · · Score: 2, Interesting

    Everyone in the project has virtual $ to buy render time; the cost of render time varies depending on demand. So, you get more rendering for your money if you time it right.

    Advanced resource allocation is one of the research projects that are keeping some people -AI and mathematicians- busy. That's things like options, auctions, etc, etc.

  12. Could work pretty well by ikekrull · · Score: 3, Interesting

    All the professional 3D apps e.g. Maya, Shake etc. are built to operate in this kind of 'decoupled rendering' environment, and render farms are probably the simplest type of 'distributed computing' - generally theyre just a bunch of independent computers on a LAN which see a shared disk on which the source materials (e.g. textures, models, motion data, imagery) is kept.

    Accounting is pretty simple too as a central dispatcher keeps track of which frame is submitted to which node, and how long it takes - it is uncommon to have 1 machine working on more than 1 frame simultaneously.

    Sun also offer this type of service, as well as others. I think it will be interesting to see if this model is adopted by hollywood etc. but I can see a lot of smaller shops taking advantage of it to acheive what would otherwise be impossible.

    I remember using a Compaq Test-Drive-Programme 4-CPU Alpha 21264 box to render a 4096x4096 cornell box using BMRT that would have taken my 128MB P3-500 an age to do.

    The model definitely works from my point of view, however I don't really think that Sun or HP will be in much of a position to actually make money out of this type of service since hardware depreciates so fast and the price people will be willing to pay would be pretty low I imagine - I guess they could simply use their unsold inventories to extract some kind of revenue from them instead of writing them off, but getting the 'volume' up to profitable levels will be a challenge.

    --
    I gots ta ding a ding dang my dang a long ling long
  13. It's utility computing by steve_l · · Score: 3, Interesting

    That is, CPU & storage is something you pay for on demand. It is distributed (two facilties, two continents), and the users are all scattered round the UK. One of the facilities is downstairs from me in Bristol; a large chilled datacentre that I dont have access to for either security or competence reasons :)

    Projects aren't charged real money for render time, do have virtual monety; cost of rendering varies depending on demand. Projects have their own balances. This lets money optimise the resource allocation. Get your design done early, render before the rest and you get more CPU time for your cash, hence better rendering.

    There are lots of other sponsors, as this is giving regional and up and coming artists/animators chances to do serious renderings on facilities they wouldn't normally get.

  14. Utility, not Grid by steve_l · · Score: 2, Informative

    'grid' computing is such a vague term. The fabric here can be reconfigured to boot different virtual disks, with different stuff running on it. It's not scavenging spare CPU or anything fancy.

    But it could host Globus2.4, even Condor. Or a very large three tier Apache 2.0 HTTPD cluster. If you have the money, you get to choose.

  15. Did anybody actually pay for this? by Animats · · Score: 2, Insightful
    It sounds like HP ran a free demo. But has anybody actually paid for their service?

    There are commercial render farm services running right now. Over 400 machines. 440 frames are rendering right now. Over 6 million frames sold. On line. Self service. VISA/MC accepted. The going rate is about $1/GHz/hour, before discounts.

    And they never mention "grid computing".

  16. Xgrid for UNIX by not_hylas(+) · · Score: 4, Informative

    "Could someone not write something like this but open source and distributed."

    Darwin:

    http://www.opensource.apple.com/darwinsource/ind ex .html

    Xgrid for UNIX:
    This article introduces the first working Xgrid agent for Linux and other Unix systems that can be integrated in any XGrid cluster (managed by OS X).

    http://unu.novajo.ca/simple/archives/000026.html

    http://developer.apple.com/hardware/hpc/xgrid_in tr o.html

    --
    ~hylas