Slashdot Mirror
One Data Center To Rule Them All
1sockchuck writes "Weta Digital, the New Zealand studio that created the visual effects for the 'Lord of the Rings' movie trilogy, has launched a new "extreme density" data center to provide the computing horsepower to power its digital renderings. Weta is running four clusters that are each equipped with 156 of HP's new 2-in-1 blade servers, and use liquid cooling to manage the heat loads. The Weta render farms currently hold spots 219 through 222 on the current Top 500 list of the world's fastest supercomputers."
Imagine a Beowulf cluster of these...
a balrog cluster of them!
How do they release the heat in the hot water?
Two Towers?
I'll show you heat loads.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
I can't mention who I work for for obvious reasons, but we did some experimenting with "extreme density" computing some time ago as part of a black ops project for the government. We achieved densities previously unheard of by man.
Unfortunately, we got greedy. We increased the density so far that the entire facility ended up collapsing into a black hole, wiping out much of the state of North Dakota. We were able to contain the damage, and we've managed to keep it a secret by replacing the state with a hologram projection, but eventually someone is going to go there and figure out that something is amiss.
Vote For Flight Safety - Vote Against John McCain
I'm of a mixed mindset when it comes to water-cooled datacenters.
On one hand, you've got the makings of a biblical scale disaster with all that water and electricity mixing.
On the other hand, you can't argue with it's effectiveness.
I'll stick to non-catastrophic issues when my
air conditioner breaks down.
Check out my sysadmin blog!
That will be your last mistake. Ever.
I think it's great that they are using liquid coolants for their system. Whenever I see a traditional server farm, I just can't help but think that air conditioners are so inefficient for the task of cooling computers. Not only do you have to cool the air, you also have to blow it around. The floors in some data centers are raised just to allow better airflow. And if you think about it, only the insides of the computers have to be cooled, not the entire freaking room. I hope this ushers in a new age of more power-efficient computing.
I also think it's pretty funny that a supercomputer is used to make movies.
A NYC lawyer blogs. http://www.chuangblog.com/
thanks freddie. greed, fear & ego are unprecedented evile's primary weapons. those, along with deception & coercion, helps most of us remain (unwittingly?) dependent on its' life0cidal hired goons' agenda. most of yOUR dwindling resources are being squandered on the 'wars', & continuation of the billionerrors stock markup FraUD/pyramid schemes. nobody ever mentions the real long term costs of those debacles in both life & the notion of prosperity, not to mention the abuse of the consciences of those of us who still have one. see you on the other side of it. the lights are coming up all over now. conspiracy theorists are being vindicated. some might choose a tin umbrella to go with their hats. the fairytail is winding down now. let your conscience be yOUR guide. you can be more helpful than you might have imagined. there are still some choices. if they do not suit you, consider the likely results of continuing to follow the corepirate nazi hypenosys story LIEn, whereas anything of relevance is replaced almost instantly with pr ?firm? scriptdead mindphuking propaganda or 'celebrity' trivia 'foam'. meanwhile; don't forget to get a little more oxygen on yOUR brain, & look up in the sky from time to time, starting early in the day. there's lots going on up there.
http://news.google.com/?ncl=1216734813&hl=en&topic=n
http://www.nytimes.com/2007/12/31/opinion/31mon1.html?em&ex=1199336400&en=c4b5414371631707&ei=5087%0A
http://www.nytimes.com/2008/05/29/world/29amnesty.html?hp
http://www.cnn.com/2008/US/06/02/nasa.global.warming.ap/index.html
http://www.cnn.com/2008/US/weather/06/05/severe.weather.ap/index.html
http://www.cnn.com/2008/US/weather/06/02/honore.preparedness/index.html
http://www.nytimes.com/2008/06/01/opinion/01dowd.html?em&ex=1212638400&en=744b7cebc86723e5&ei=5087%0A
http://www.cnn.com/2008/POLITICS/06/05/senate.iraq/index.html
http://www.nytimes.com/2008/06/17/washington/17contractor.html?hp
http://www.nytimes.com/2008/07/03/world/middleeast/03kurdistan.html?_r=1&hp&oref=slogin
http://biz.yahoo.com/ap/080708/cheney_climate.html
http://news.yahoo.com/s/politico/20080805/pl_politico/12308;_ylt=A0wNcxTPdJhILAYAVQms0NUE
http://news.yahoo.com/s/nm/20080903/ts_nm/environment_arctic_dc;_ylt=A0wNcwhhcb5It3EBoy2s0NUE
is it time to get real yet? A LOT of energy is being squandered in attempts to keep US in the dark. in the end (give or take a few 1000 years), the creators will prevail (world without end, etc...), as it has always been. the process of gaining yOUR release from the current hostage situation may not be what you might think it is. butt of course, most of US don't know, or care what a precarious/fatal situation we're in. for example; the insidious attempts by the felonious corepirate nazi execrable to block the suns' light, interfering with a requirement (sunlight) for us to stay healthy/alive. it's likely not good for yOUR health/memories 'else they'd be bragging about it? we're intending for the whoreabully deceptive (they'll do ANYTHING for a bit more monIE/power) felons to give up/fail even further, in attempting to control the 'weather', as well as a # of other things/events.
http://www.google.com/search?hl=en&q=weather+manipulation&btnG=Search
http://video.google.com/videosearch?hl=en&q=video+cloud+spraying
dictator style micro management has never worked (for very long). it's an illness. tie that with life0cidal aggression & softwar gangster style bullying, & what do we have? a greed/fear/ego based recipe for disaster. meanwhile, you can help to stop the bleeding (loss of life & limb);
http://www.cnn.com/2007/POLITICS/12/28/vermont.banning.bush.ap/index.html
the bleeding must be stopped before any healing can begin. jailing a couple of corepirate nazi hired goons would send a clear message to the rest of the world from US. any truthful look at the 'scorecard' would reveal that we are a society in decline/deep doo-doo, despite all of the scriptdead
Note the vague phonetic similarity between "North Dakota" and "Mordor."
Did this project happen to hand out t-shirts... or, hm, something less obvious like... rings?
If their rendering farm has enough network bandwidth to be on the top 500 list, then they're not doing it right.
Hi, I am a DARKIE!! But at least I didn't use the N-word...
I wonder how much they have to pay for external bandwidth. I always thought that "super data centers" are used to help split the job between multiple special effects studios, so, say, group in London can work on part of the shot and still have all data in the same place.
Except in New Zealand there are no "unlimited" plans, and there are severe bandwidth caps in place.
Hyperom.com
"Weta Digital, the New Zealand studio that created the visual effects for the "Lord of the Rings" movie trilogy, has launched a new "extreme density" data center to provide the computing horsepower to power its digital renderings. Weta is running four clusters that are each equipped with 156 of HP's new 2-in-1 blade servers, and use liquid cooling to manage the heat loads. The Weta render farms currently hold spots 219 through 222 on the current Top 500 list of the world's fastest supercomputers."
I think people should keep the above in mind next time they talk about GPUs being a replacement.
...will still pale in comparison to ILM. LOTR had some of the worst compositing I had ever seen. The fact that ROTK won an Oscar for Visual Effects only further illustrates the irrelevance of the politics behind the Oscars.
Just curious, but what does ILM run?
I'm from 2053 and my watch would hold today's 1st to 16th spots . Only 16 cores, I know it's lame but then again I only paid 25 EuroYens. Damn cheap piece of crap made in the United States of Canaxico, you know how it is.
I also think it's pretty funny that a supercomputer is used to make movies.
It was pretty funny forty years ago, too.
The LHCb experiment has a large processor farm for their online data analysis, all water cooled. Apparently it makes the computer scientists very nervous. OTOH, the main computing centre just uses air cooling, so we've got a real mix of technology.
xterm -n 8
Amazing! So what was North Dakota like before it became the vast, desolate wasteland devoid of any trace of humanity it is now?
but have you considered the following argument: shut up.
Does it run chrome?
I was working on a show which had a bunch of SGI Boxes similar to this. Two servers (2 dual core processors per server) in a 1U case. Only problem with it was that if something went wrong with one of two servers in the box you had to send the whole box back to SGI to get fixed.
Now you would think that normally it wouldn't be a problem if 8 CPUs out of 200 go down on a farm but the way they usually set up the farm (in my experience) is by assigning some processors to each department using a priority structure. So lets say the simulation department had sent some stuff to the farm. 20 min later the stuff finishes. Now lighting can get those CPUs at a low priority until the sim guys send another job to the farm. Problem is if your have a small section of the farm and one of those machines in that small section broke (which happens a lot when the farm starts to get flooded near the end of a show) your stuck not being able to do work for a while until IT gives you priority on some other machines (which at the end of a production can take a ling time).
What I would love to see is the farm software seperating the job by CPU cycles so that when I come in early and there nobody's around I could get the full processing power of the farm to my self.
The Weta render farms currently hold spots 219 through 222 on the current Top 500 list of the world's fastest supercomputers.
That's a lot of orcs.
I drank what? -- Socrates
After I virtualized way back in 2003 I discovered ... that if I built my own "sky" each "cloud" I deployed in its own virtual machine was very simple and easy to understand and maintain.
Just like Unix itself, which takes small working units and connects them together to make a complex whole, the "sky" is complex but each "cloud" is simple.
And makes it easy to deploy and move "clouds" to super-centers like the New Zealand data-center as it makes sense (speed, location, power, price) to do so.
Today Fedora Core + VMware's GNU solution (aka Vmware Server 1.x, formerly VMware GSX). Both are free. Virtual machines can run anything. No connectivity to the host, even put the firewall in a v.m. Runs fine on a dual Xeon P3 / 1.2GB ram and easy disk management using LVM. A backup (duplicate machine) gives me twelve clouds in my current sky playground.
Richard showed up a couple years ago at the Wairarapa Railway Modellers annual BBQ with a 7 1/2" scale locomotive that was built at Weta Workshops for a film. In the photo located at http://www.certsoft.com/NZ2/richard_taylor.jpg Richard is the one in the middle.
North Dakota is 2-dimensional, you insensitive clod!
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
to rule them all... (sorry, had to do it)
Impetuous! Homeric!
I can't seem to find the video I'm looking for but I'm a fan of the new Sun Data Center cooling design. Works on the idea of cooling each cab individually and not all the rest of the rubbish air floating around. I know its been done before but this time its done right. Info somewhere here