Slashdot Mirror


Weta Digital Grows Cluster

Korgan writes "A little over 3 years after their last upgrade, Weta Digital has just added another 250 more blade servers to their render farm to help with the final renderings of King Kong. From the article: "The IBM Xeon blade servers, each with two 3.4 gigahertz processors and 8 gigabytes of memory, are housed at the New Zealand Supercomputing Centre in central Wellington. They have been added to the centre's existing bank of 1144 Intel 2.8GHz processors, boosting its power by 50 per cent to create a supercomputer with the equivalent power of nearly 15,000 PCs. The servers run the Red Hat version of the open-source Linux operating system. The purchase means the centre is back among the 100 largest supercomputing clusters in the world." And all that computing power is still available for hire when Peter Jackson isn't using it."

22 of 209 comments (clear)

  1. Export restrictions? by BWJones · · Score: 4, Interesting

    So, out of curiosity. What happened to the export restrictions of the US government on CPU's beyond a certain MIPS range? I remember that the old PowerMac 9600/300 eclipsed this federally mandated figure and now we have home game consoles that easily eclipse that performance range. Certainly the advent of cluster computing with commodity hardware made many of these issues moot, but what is the status of the law? Was it repealed or is it just commonly ignored?

    I know that historically, NeXT did quite a bit of work for TLA agencies and that Richard Crandall's program, zilla.app grabbed some attention from interested parties. Because of this work, NeXT had some cash infusion for their hardware even after shutting the line down for general commercial consumption. More recently, Apple has been selling Xserves to some of those same agencies, and contractors for work, but I do not know if they are selling any clusters outside the US?

    The history of course behind this law was that the CIA and NSA were concerned that foreign governments could use compute time to help design nuclear weapons as well as defeat cryptography that might compromise US secrets.

    --
    Visit Jonesblog and say hello.
    1. Re:Export restrictions? by 14erCleaner · · Score: 5, Informative
      They've been eased over the years, e.g. here is a list of articles.

      In 2002 it was upped to 195,000 million theoretical operations per second, and the limit goes up automatically every six months. A typical PC in 2002 was 2000 MTOPs, so this allows export of some rather big honking systems.

      --
      Have you read my blog lately?
    2. Re:Export restrictions? by Hey+Pope+Felcher+.+. · · Score: 3, Interesting

      . . . export restrictions are generally overcome by the all mighty dollar, and besides hasn't New Zealand merged with Hollywood now?

      More interestingly, can anyone see digital actors quickly surpassing their organic cousins, no matter what Peter Jackson says?

      And slightly more interestingly, when will New Zealand surpass California in flim making, it is the ideal location, with better light, more interesting geography, and (at the moment) far cheaper to work in. There are of course the problems with the remoteness of the location, but with the rapidly shrinking world cliché, this is surely no longer such a problem, especially with the work Mr. Jackson is putting in regarding the logistics.

  2. Clearly... by TooMuchEspressoGuy · · Score: 5, Funny

    Clearly we need a Beowulf cluster to slay this gigantic King Kong cluster!

    --
    Many Bothans died to bring you this sig.
    1. Re:Clearly... by Mercano · · Score: 4, Funny

      What about a MechaMozilla, the open source alternative?

      --
      #include <signature.h>
  3. Blah by Anonymous Coward · · Score: 4, Funny

    That's nothing. The people arrested in the story earlier today had the computing power of 100,000 personal computers. Beat that!

    1. Re:Blah by sleeper0 · · Score: 3, Interesting

      How do you combine 1644 server class ~3ghz CPUss and end up with the power of 15,000 PCs? Only in the marketing department...

  4. power draw by dreadlocks · · Score: 5, Funny

    that's why the lights dim in Wellington when the cluster is rendering

    1. Re:power draw by Farmer+Tim · · Score: 3, Funny

      You have no idea how hard it is for an Australian to resist making a joke about dim lights in Wellington having nothing to do with the power grid...oh, whoops...

      --
      Blank until /. makes another boneheaded UI decision.
    2. Re:power draw by shokk · · Score: 3, Interesting

      Seriously, I would love to see an article, not on what their render farm is like, but on what their power management for it is like. Do they use remote-managed power strips or are they all just popped in, hoping that they don't blow a fuse. How many UPSs do they have and what kind of on-battery runtime do they shoot for?

      What kind of power draw do the blade chassis have? What blades? What version of Red Hat?!?!?!

      Unfortunately TFA is very short on details and reads more like "Peter Jackson went out and bought 500 computers! Woo!"

      --
      "Beware of he who would deny you access to information, for in his heart, he dreams himself your master."
  5. Explain this "new" math to me... by pla · · Score: 5, Insightful

    They have been added to the centre's existing bank of 1144 Intel 2.8GHz processors, boosting its power by 50 per cent to create a supercomputer with the equivalent power of nearly 15,000 PCs

    Total processors: 1644.

    Now, the Xeons do a bit better than the run-of-the-mill P4, but 10x faster? No way.

    For that matter, they don't run faster at all. They just do somewhat better (as in, 10-25%, not 913%) on certain types of memory-heavy tasks.

    Someone either made a major typo or pulled numbers from their netherregion...

    1. Re:Explain this "new" math to me... by brunes69 · · Score: 4, Informative

      The 250 added blade servers each have two 3.4 GHz CPus, while the existing 1144 only had two 2.6 GHz CPUs.

      Every two new servers is approx. as powerful as three old servers. It is more like they are now running 1519 dual 2.4 Ghz machines, or 3038 2.4 GHz cores.

      Also, remember that a 2.4 GHz is faster than two 1.2 Ghz chips, because of instruction set improvements.

      So, I would not say it would be far off to say that this cluster is approx. the computing power of 15,000 1 - 1.5 GHz machines. This is probably what they are basing the numbers off of.

  6. Re:* sigh * by Average_Joe_Sixpack · · Score: 4, Funny

    Things have improved since TRON

    Except Jeff Bridges' acting

  7. Contact? by jridley · · Score: 3, Funny

    Hmm, how to get them on my SETI team....

  8. Extra-Beefy Power Supply perhaps??? by Anonymous Coward · · Score: 3, Funny

    Power of 1 PC: 250 Watts
    Power of 15,000 PCs: Enough to power a small town

    Ability to do math in your head: Priceless

    For everything else, there's xcalc.

  9. Re:And to think... by delire · · Score: 5, Funny

    Star Wars Episode III : Revenge of the Sith was processed on just a 140-processor Opteron AMD64 farm running Windows 64-bit beta.
    No wonder the film was so bad.
  10. The massive power of creating digital realism by digitaldc · · Score: 4, Informative

    The amount of power that is needed to create a realistic outdoor scene with multiple actors is simply astounding. King Kong will most likely be candy for the eyes when it is done. Halo, the next Peter Jackson movie, will probably just as amazing.

    An interesting article on building a digital animation studio (IBM) is here:
    http://www-128.ibm.com/developerworks/library/wa-a nimstudio1/

    --
    He who knows best knows how little he knows. - Thomas Jefferson
  11. Sweet! by Black+Francis · · Score: 3, Funny

    The IBM Xeon blade servers, each with two 3.4 gigahertz processors and 8 gigabytes of memory...
    ... now they might even be able to run Vista!

  12. Re:And to think... by malducin · · Score: 4, Informative

    Those numbers are misleadingly wrong. Star Wars Episode III was rendered between the old ILM location and the new Presidio facility. The Presidio has about 4000 processors used for rendering, while old ILM had about 2,500 processors. The data center of the Presidio came online (I think) late last year. So frames from Ep. 3 and The Island were rendered both at the Presidio and old ILM. So surely ILM rendered Ep. 3 on a few thousand processors.

    I think the misleading part is that some articles stated that the initial order for AMD Opteron based machines for the data center was 140 processors. But their renderfarm is crtainly 4,000 procs which I think includes about 1,000 workstations that are used for overnight rendering.

    Data Center Gets Star Treatment

    Also while ILM does have an Opteron based renderfarm they run Linux on them, not Windows64 beta.

  13. Re:* sigh * by Comboman · · Score: 3, Insightful

    The problem is that CG is used so indiscriminately now. In the begining, it was only used to do things that you couldn't do any other way (like the living water in The Abyss or the T1000 morphs in Terminator 2). Now-a-days CG is used as a cost cutting measure to do things that could be done in a more traditional way but would cost a little more. I was watching Raiders of the Lost Ark the other day (which still looks great after 20+ years) and thinking, if they shot that movie today they would have just used CG instead of finding 1000 trained snakes, and it would look like crap.

    --
    Support Right To Repair Legislation.
  14. xeons? by ruiner5000 · · Score: 3, Informative

    Did Intel do a massive discount? The standard is Opteron for render farms and for good reason. Much faster, much less power, and a superior upgrade path. 150 watts for the fake dual core Xeon, or 89 for a dual core Opteron. Hmm, tough choice there. Particularly when you look at the new Spec numbers for the dual core Xeon it is a massacre with Opteron well ahead still.

    --
    ignorance is bliss. googlefiberatx.com
  15. Re:* sigh * by Total_Wimp · · Score: 3, Insightful

    Am I the only one who prefers models and stop motion animation to the CGI garbage of the last 15 years?

    It depends on the CG. If I don't notice that it's CG I tend to like it. If it looks like CG I tend to groan.

    Jurassic park was extremely well done CG and I loved it. Spiderman was, well, cartoony at best (but a good story and Kirsten Dunst go a long way). In Gone in Sixty Seconds they should have just used real cars in all the scenes. There was no excuse for CG shenanegans. But the New York scene in AI was flawless and would have been impossible to film in scale models alone.

    Notice a trend? If the director is a master of visuals and refuses to accept compromise (just try to tell Spielberg "that's the best I can do") then your CG is gonna work. If your level of visual excellence is better exemplified by Xena the Warrior Princess then you may just be willing to settle.

    I don't mean to bash Raimi. I loved a lot of his stuff, including Spiderman. But did any of you really think Spiderman's level of CG excellence met the level of Spielberg? Directors and producers need to be more demanding of their digital special effects. They should reject mediocre work as readily as wire work with, well, visible wires.

    TW