Slashdot Mirror


Weta Digital Supercomputer For Hire

sushi writes "NZ's Stuff news site is reporting: 'Peter Jackson's special effects shop Weta Digital has teamed ... to establish a world-class supercomputing facility in Wellington which will be rented out to clients worldwide.' Currently comprising 504 IBM blade servers, each of which contains two 2.8 Gigahertz Intel Xeon processors, 6 Gigabytes of memory and 40 Gigabytes of storage, and ranked 80th in the top 500 supercomputers, they are intending to upgrade into the top 10. Also covered at the Australian Financial Review."

49 of 184 comments (clear)

  1. So... by iamdrscience · · Score: 3, Funny

    Are we going to see a "Meet the Feebles part 2" or what?

  2. Dupe? by Anonymous Coward · · Score: 5, Informative
    1. Re:Dupe? by wrinkledshirt · · Score: 5, Funny

      Maybe Slashdot should rent the machine to keep track of stories with similar content...

      --

      --------
      Bleah! Heh heh heh... BLEAH BLEAH!!! Ha ha ha ha...

  3. Awesome by iamdrscience · · Score: 3, Funny

    So now instead of curing diseases, finding large mersenne primes or discovering inteligent life forms we can get hyper realisitc renderings of Gollum's eyelash crust. Spectacular.

    1. Re:Awesome by Jerry+Talton · · Score: 5, Insightful

      It really cracks me up that you rate finding large mersenne primes in the same category as curing disease or discovering other intelligent life forms. Talk about something with no practical applications...

    2. Re:Awesome by mod_parent_down · · Score: 2, Insightful
      or discovering inteligent life

      Or, you know, a spell-cheeker...

    3. Re:Awesome by ColaMan · · Score: 3, Funny

      A good toast :
      "To Pure Mathematics - may it never have any practical use!"

      --

      You are in a twisty maze of processor lines, all alike.
      There is a lot of hype here.
    4. Re:Awesome by LoudMusic · · Score: 2, Insightful

      You mention using it for the standard distributed projects, but what about the computers that are already running these projects? Are they not part of a super computer? A type of cluster? Surely the fellas (and ladies?) at distributed.net and / or the SETI@Home crew could write up a simple distributed RMAX app to test how much CPU time is available on the internet. Submit those numbers to the top500.org and see where you end up. I bet it'd put the Earth Simulator to shame.

      Now, for 'real time', it'd be shat. But the computational power is there. It's just a high latency cluster.

      --
      No sig for you. YOU GET NO SIG!
  4. top 500 ! by phreakv6 · · Score: 5, Informative

    This is the top 500

    --
    fifteen jugglers, five believers
  5. Re:6gig of memory? by Omega+Hacker · · Score: 3, Informative

    Only 6gig of memory? Is it me or that that seem a stupidly low amount when dealing with a amachine of this power?

    That's 6GB for each of the 504 (not 512??) nodes, or 3GB per processor, almost 3TB total. Trust me, that's plenty for all but the most extreme uses.

    --
    GStreamer - The only way to stream!
  6. Correction? by iamdrscience · · Score: 4, Insightful

    Hmm... if you look at the top 500 it appears that 80th was their previous place and that they have since upgraded their cluster to become 77th.

    1. Re:Correction? by Anonymous Coward · · Score: 3, Informative
      Not quite, Weta have two clusters. The cluster for rent (that the article is about) was built only to supplement Weta's primary cluster (and get the films out on time).

      It has 1080 processors (540 blade servers) and is at rank 80.

      Weta's primary cluster (not for rent!) has 1176 processors and is at rank 77.

  7. Doomsday scenario by ciurana · · Score: 5, Interesting

    Hrm...

    How soon before the bad guys set up a dummy corporation and start running nuclear bomb or protein folding simulations on this cluster? I'd be very interested (probably along with some governments) in Weta's and Gen-i screening processes. Will anyone who can foot the bill get access?

    I know, this is tinfoil hat stuff, but it's late and I get this "glass half full" visions when I'm sleep deprived.

    Cheers,

    E

    --
    http://eugeneciurana.com | http://ciurana.eu
    1. Re:Doomsday scenario by Vellmont · · Score: 5, Insightful


      How soon before the bad guys set up a dummy corporation and start running nuclear bomb or protein folding simulations on this cluster?


      The hardest part by far in making a nuclear weapon is getting the fissile material. If you are able to get highly enriched uranium you don't even need to do any simulations, the design is fairly simple and no testing is needed. Plutonium is a bit harder.

      The point though is that computer simulations of nuclear weapons is the least of your problems, and is by no means required. Computers aren't secrets, and getting a few hundred of them together in a cluster is a task anyone with $100,000 can easily accomplish. Compared to getting the required fissile material, any required computations are easy.

      I'm not sure what you're getting at with protein folding. Is their some doomsday weapon you can create by knowing how proteins fold? Even if it is, it's not a big concern. No one has gotten even close to completely simulating a protein folding. There's simply not enough computing power yet. What's been done to date are just small scale simulations.

      --
      AccountKiller
    2. Re:Doomsday scenario by jesterzog · · Score: 2, Interesting

      How soon before the bad guys set up a dummy corporation and start running nuclear bomb or protein folding simulations on this cluster? I'd be very interested (probably along with some governments) in Weta's and Gen-i screening processes. Will anyone who can foot the bill get access?

      I think it would depend on how open it is. The New Zealand government is strongly anti-nuclear (however rational or irrational that may be). eg. US nuclear vessels aren't allowed within the NZ economic zone. This type of stance is mostly on principle based on popular opinion (rational or otherwise), much to the country's disadvantage in things like trade deals. New Zealand is small enough that popular opinion can still have quite a knee-jerk effect on most government policy.

      If you're referring to businesses or other governments when you say "bad guys", it wouldn't be unexpected for the NZ government to step in and say "you can't do that", irrespective of what WETA might want to do.

      If there's some way to run such simulations without making it clear what's being done, then it might be possible. With something that's so high profile, though, it'd be very difficult to get away with it, without at least some people having an idea of what's going on, or at least being suspicious enough to enquire further.

  8. my precious..... by Splezunk · · Score: 3, Funny

    my precious.....

  9. Finally! by xsupergr0verx · · Score: 4, Funny

    Thanks to WETA, now I can run Doom 3!

    --

    Click here for a free picture of an iPod!
  10. rent it out by OneArmedMan · · Score: 4, Insightful

    to other ppl that need to render stuff. Im sure they could figure out some reasonable pricing vs CPU time etc.

    1. Re:rent it out by michajoe · · Score: 2, Funny

      C'mon, if you're not going to read the article, at least read the friggin' Slashdot story. If even that is too much for you, please consider at least reading the headline.

      But yeah, renting it out seems like a smart idea.

  11. Digital Supercomputer? by Anonymous Coward · · Score: 2, Funny

    Where can I rent an analog one?

  12. An idea... by Veridium · · Score: 4, Interesting

    You know, that only has ~500 nodes right? How many people out there are either out of work, or sick of doing what they're doing? Maybe we ought to get about 1000 of us nerds together in some kind of co-op, cluster our machines then rent it out? My main box is dual opterons and I already have 6 dual P-pro 200s clustered...

    Yeah, I know, the logistics of it, the devil would be in every detail... Neat to think about though.

    --
    Think for yourself, destroy your television.
    1. Re:An idea... by mefus · · Score: 2, Informative

      Computational Grid is working out the logistics, and has a software package that'll get you on the grid.

      --
      mefus
      In Open Society, GPL Software frees YOU!
    2. Re:An idea... by Vellmont · · Score: 4, Insightful


      You know, that only has ~500 nodes right?


      It has 500 nodes that are highly and quickly interconnected. It's like the difference between 500 people working on a problem in the same room, and 500 people spread across the country communicating by postal mail. Most interesting problems require a lot of inter-communication, so 500 slowly connect nodes isn't too usefull.

      --
      AccountKiller
  13. Imagine by Malfourmed · · Score: 3, Funny

    Imagine Peter Jackson making a cluster of Beowulf movies using those...

  14. Re:This is great by nz_mincemeat · · Score: 3, Funny

    Missle trajectories: supercomputers can help design accurate missle systems, and missle defense systems. See 1 for why this should be restricted.

    Missile trajectories? All you need to calculate that stuff is a Playstation2... Quick! We need to keep these "supercomputers" inside honest nations too ;)

  15. Re:Image a... by nz_mincemeat · · Score: 2, Interesting

    Another movie will probably take too long to render before that renderfarm is needed again for one of Weta's own jobs. Hence hiring out CPU hours while it's sitting still doing very little (and helps pay the rent too, I suppose)

  16. Re:6gig of memory? by Gopal.V · · Score: 4, Insightful

    > almost 3TB total

    Do remember that 90% of the time, it's not the size that matters , but the organization.

    I've worked on a relatively small cluster processing experiment in college with 12 boxes on a 10 Mbps LAN with a combined memory of 1.5 Gb RAM . It might not look much , but with 32 MB of RAM on each box (each had 128 MB ram) being held by the home cooked shared memory daemon (this was waaay before memcached was born, Ok) , the boxes ran the number crunching beautifully .

    The operation needed was simple, to sort and process an amazingly HUGE chunk of data in almost realtime (in this case some wierd algorithm some Mech teacher wanted and did up in C).

    Anyway in about 7 weeks and reusing a dozen of the college's vanilla PCs we did a LOT of interesting things .

    So my question is , how's the server connected memory wise (most of these tasks are highly memory bound or at least that's the major bottleneck to optimise).

  17. distcc? by SKPhoton · · Score: 2, Funny

    How about using all those cpus as Distcc nodes!

    (Go Gentoo!)

  18. Re:This is great by Anonymous Coward · · Score: 4, Funny

    If someone paid to rent a supercomputer for a few hours to break your SHA1 SSL keys, you've got bigger problems than just your keys being broken :)

  19. Re:6gig of memory? by zcat_NZ · · Score: 4, Funny

    Yep. Enough for Doom3, or Longhorn. Possibly even enough to run doom3 IN longhorn!!

    --
    455fe10422ca29c4933f95052b792ab2
  20. for one second by phsdv · · Score: 2, Funny

    with your budget you probably could play for excatly 1 second...

  21. Re:Nah... by Anonymous Coward · · Score: 3, Funny

    Well it's called 'Brain Dead' down here, so I'd suggest either 'Brain Deader' or 'Dubya'.

  22. Re:Nah... by grolschie · · Score: 2, Funny

    "Headshot Barry! It's the only true stopper!"

  23. Re:6gig of memory? by Syzar · · Score: 3, Informative

    Remember that those 6 gigs of ram are per node. Total combined ram is 6gigs * 504nodes = 3024 gigs ram. Grandparent propably meant to say that 6 gigs are enough per node.

  24. Weta's old cluster by SlashdotMeNow · · Score: 5, Interesting

    We actually bought the old cluster Jackson used (for next to nothing I might add!) It's 62 PCs (no they run Windows) totalling 124 processors (2 racks full)

    That's a total of 124GHz and 124GB of RAM. We're using it to render architectural fly-through movies.

    Hmm... I'm all hot now... Need a cold shower!

    1. Re:Weta's old cluster by Aussie · · Score: 2, Funny

      by SlashdotMeNow (799901) Alter Relationship
      We're using it to render architectural fly-through movies

      Give us a link to one of these movies and we would be happy to SlashdotYouNow

  25. Re:This is great by Old+Telco+Guy · · Score: 2, Informative

    Dude, Eniac computed artillery trajectories. A PS2 could probably do the onboard realtime guidance, nav and detonation sequence :-)

  26. Re:This is great by PCM2 · · Score: 2, Interesting

    Are you kidding? You don't need a Playstation2 or any other kind of supercomputer to plot missile trajectories. Terrorists had no trouble doing that with archaic 6502 processors back in the early 1980s.

    --
    Breakfast served all day!
  27. Re:40 Gigabytes of storage.. by Anonymous Coward · · Score: 2, Informative

    That's 40GB *each*... > 20TB total.

  28. Re:Nah... by empaler · · Score: 2, Insightful

    Yeah, let's start making jokes that are about things we can *all* agree on are funny.

    No point in dragging down a person who has proclaimed himself to be the most devout Christian since the Apostles.

  29. Re:King Kong by lewp · · Score: 3, Funny

    Actually they've decided not to use any digital effects for King Kong. Peter Jackson will be wearing a gorilla suit.

    Why do you think they're renting out the cluster?

    --
    Game... blouses.
  30. Re:Once and for all... by I+confirm+I'm+not+a · · Score: 3, Funny

    We will now truly be able to figure out how many New Zealanders it takes to change a light bulb.

    Just one, but he won't change the bulb - he'll fix it using bind-a-twine and 8-guage fencing wire.

    (Disclaimer: exiled Kiwi)

    --
    This is where the serious fun begins.
  31. Noooooooo.... laptop hard drives... by ricky-road-flats · · Score: 5, Interesting
    We've had big problems using IBM blades, not least because by default they come with crappy 5400rpm laptop hard drives - and the 40 GB mentioned in these blades imply that's what they've got (which is what my predecessor ordered).

    I guess with 6 GB RAM each they shouldn't have to do much (or ANY, if I was running this) swapping, and if the jobs are tweaked to not use the hard drives too intensively, they might be OK. If what you do uses the hard drives for much, they are sh*t, to put it mildly. If you could plug these into the blades, they's be very useful, quick machines. But you can't yet.

    The really crap thing is, if you do want SCSI drives in the IBM blades, you connect a module ot the side of the blade which gives you a couple of proper SCSI drive bays. Which halves the number of blades-per-bladecenter to 7.

    Given the bladecenter is 7U tall, you'd be better off with 7 1U servers with SCSI bays already in and better NIC options. The internal networking of the bladecenter is awful for everything but the simplest low-requirement setups - it's hideously expensive to give each blade a couple of gigabit connections.

    Even these cheap little things are 1U, take 2 U320 SCSI drives, and have dual Gigabit connections built-in.

    And I *still* can't get USB dongles to work with thes fscking blades, grumble grumble.

    Having said all that, when can I play on this thing? My Folding@Home could do with a bit of a boost, and with Hyperthreading I could have 2016 units running simultaneously.... although it might get a little warm behind the racks, 1008 2.8 GHz Xeons pump out a good bit of heat!

  32. I need to get more sleep... by Jugalator · · Score: 2, Funny

    Read this

    "Weta Digital Supercomputer For Hire"

    as

    "Weta Digital Supercomputer On Fire"

    Thought, whoa, finally some big news on Slashdot!

    But no... Anyone willing to go with me to put them on fire for some hot Beowulf cluster action? :-P

    --
    Beware: In C++, your friends can see your privates!
  33. Small potatoes by rumblin'rabbit · · Score: 2, Interesting
    I'm surprized this is ranked as 80'th, because it's not that large by todays standards. Even some mid-sized geophysical processing companies, for example, can beat it. Large ones might have 5 or 10 times this capacity.

    If they're going to market this capacity, they had better do it quick. The shelf life of computational power is not much greater than milk.

  34. Connectivity Question by dosun88888 · · Score: 5, Funny

    Do they use Tolkien Ring?

    ~D

  35. Weta Digital please read this! by natefanaro · · Score: 2, Interesting

    I think you should run SETI for one day. I would love to see how many work units can be computed. It would probably set a record.

  36. What about #77 by beaverbrother · · Score: 2, Interesting

    Theres no reference to WETA DIGITAL's second cluster, #77 on the list. It contains 588 computers as opposed to 504.
    top 500 page for the cluser here
    Why don't they just combine the two. That would surely grant them a top 10 spot...

  37. They tried ... by gstoddart · · Score: 2, Funny
    Do they use Tolkien Ring?


    They tried, but they discovered an error in the protocol whereby the one who held the token wouldn't let it go.

    =)

    --
    Lost at C:>. Found at C.