Weta Digital's Render Farm Upgrade
Headspace2 writes "Weta Digital (The graphics company behing LOtR computer effects) has just purchased 220 2.2GHz dual Xenon machines, each with 4GB of ram, to add to their current render wall of 350 1 Ghz P3 systems. They have also placed an order for another 256 Xenon servers. And it's all running Linux. My favorite
quote is 'it is thought the server farm will be the most powerful processing site in the Southern Hemisphere'. They should use that in the FotR ad campaign... 'Rendered using the most powerful processing site in the southern hemisphere' Congrats the guys that get to play with all those clock cycles.
Make more movies.
Powered by noble gas. Woohoo!!
Wow. I wonder what their Distributed.net keyrate would be?
Xenon is an element. Xeon is an expensive CPU. I see "Intel Xenon" too many times at work. Please not on Slashdot too.
High-speed Road Trip (18.000KPH)
g to the oatse
c to the izzex
fo shizzle my nizzle my work account is temporarily banned, so i have to troll from home.
Dual Xeon ?
$45 per U Colocation Special
Give the the warm orange glow of dual neon machines any day.
Wow. It could only be more politically correct if the chips were Athlon MPs.
Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
could they also claim that it is the most powerful processing site in the eastern hemisphere?
...will correct you on "Xenon" while completely missing "behing".
Typical Slashdot.
"The servers run in parallel and major jobs are broken down for each server. It is networked together with 100Gbps ethernet and Foundry networking switches...."
A hundred gigabits per second? Dude! Sign me the hell up!
You could use xenon to power a quantum computer. Dual xenon = 2 xenon atoms = 2 qubits, which could be roughly 64 bits, or the processing potential of a potato.
I'm the Devil the Windows users warned you about.
... in some of the decentralized computing efforts, coming from the southern hemisphere?
...
Team LotR strikes at Distributed Folding, ECCP, Folding@Home, Genome@Home, OGR (24 and 25), RC5, Sengent D2OL, SETI, UD
Cheers, Nostrada
No, I haven't read the article. Is that necessary?
And yet they still can't make Frodo look like a guy.
Since there are all kinds of benchmarks, which would be most appropriate here? I'm thinking heavy fpu performance. Most gaming sites only compare single processor performance. Can anyone dig up a benchmark of a dual P4 of 2.2GHz or thereabouts? Then compare to the P3s. I'm guessing this more than doubled their capacity.
...without a front-page type or two? Xenon machines, FotR... I understand typos happen everywhere, but when you're putting out a product like slashdot.org, even asking people for money for ad-free browsing, you would think you could expect some basic editing of the stories. Is it that much to ask to have some one read over the story once or twice before it's posted?
Out of 10 or 12 stories a day, there are always one or two with bad grammar and/or spelling. This definately takes away from any sense of professionalism slashdot.org presents on itself. Consider this editors, everyday this website is your best resume`. You wouldn't submit a resume` that has grammatical errors on it, would you?
Slackware forever. Honestly, what else would you trust when it absolutely positively has to be stable, secure, and easy
Weta Digital's Render Farm Upgrade
is it the "Render Farm" of Weta Digital that is being upgraded?
My head hurts...
Thanks to file sharing, I purchase more CDs
Thanks to the RIAA, I buy them used...
Any chance they could use those extra cycles for cracking RC5 blocks? Give the slashdot.org team a run for their money, eh?
I wouldn't say make more movies, I'd say make better movies.
Michael C. Hollinger
Sure, who cares about plot or character development? We've got a server farm!
Who submitted this? George Lucas?
"Why should we leave America to go to America Junior?" - H. Simpson, on visiting Canada
that is a typo. to some it's obvious, but to some it's not.
I wonder, with todays CPUs becoming more and more powerful, won't the actual internal architecture and the standard data transmission system (copper or gold or whatever those conduits are made of) end up outdated and not being able to cope with the CPU's speed? Maybe this applies to other pieces of hardware as well? HDs not being able to feed procs the data they require fast enough or RAM suffering from aforementioned problem...
How do they solve these kind of issues anyways, especially in extremely large computer arrays like this one mentioned here but also in supercomputers or maybe even in our home PCs which are getting faster and faster as well every few months...
Hate me!
How about XPs modded into MPs?
$5 bucks says they just want be able to play Doom III at full detail ;)
How do you count qubits?
I'm pretty sure that there's no direct equivalence formula between qubits and bits...
-Kz-
"F"ellowship of the Ring?
>> They should use that in the FotR ad campaign... 'Rendered using the most powerful processing site in the southern hemisphere'
So that morons like Taco can point this out to their long-suffering girlfriends?
Who gives a fuck. Seriously dude, get a hold of youself and try not to be a weiner all your life.
Assuming all the assumptions above were correct, how does the cost compare to something comparable stateside? Of course, I'm ignoring the "100Gbps" network(!) and the Foundry switches, but I don't think they'll add more than a couple of hundred bucks per machine _at most_.
Come on guys. We got rich pickens on THEreg and noONE
is posting the story here. Whats up??
umm, fellowship of the ring
I wonder how much below cost Intel did this for, to score the bagging rights? All those guys they are about to lay off, paid for this.
"Intel-We come for you!"
Greek Geek
Pshaw! That'll only appeal to geeks! Oh, wait a minute...
"Yeah, well, Dracula called and he's coming over tonight for you and I said okay."
"typo" maybe?
Is it that much to ask ...
"Is that too much to ask ..." or possibly "Is it too much to ask ..." depending on what you wanted to do with the rest of the sentence.
Slashdot and professionalim in the same sentence has to be some sort of error.
You wouldn't submit a resume` that has grammatical errors on it, would you?
Surely you mean "in it".
My point? Enough with the bitching about the spelling/grammar. Most of people here aren't any better and of the remainder most don't care.
Nerd: Derogatory term typically directed at anybody with a lower Slashdot ID than you.
I bet they get killer frame-rates!!
but i figured out.. Fellowship of the Ring
Assuming a base platform cost (without processor) of $400 for MoBo, memory etc., the P4 Xeon would have to be 17% faster than the Athlon to justify the premium. According to the benchmarks on Tom's Hardware, Intel would have a hard time attaining that.
Jan
Isn't the Intel processor called Xeon not XeNon?
I think there's an extra N in there---but I'm not sure.
on THEIR SETI@home team!
Setting his threshold to 5, Sparky eliminated most of the trolls on /.
So, would this sort of a server farm be /.-effect-proof?
"Evil will always triumph because good is dumb." -- Dark Helmet
Donate any down-time to Folding@home.
Opinions on the Twiddler2 hand-held keyboard?
I wonder what kind of monitor you'd get for that setup?
The opposite of progress is congress
Yeah .. thats great ... except there's only 5 or 6 people in the southern hemisphere.
"[superlative] in the southern hemisphere" is the standard appellation for anything that New Zealanders (or Australians) are proud of, but isn't actually that huge.
Eg: "New Zealand has the most DSL connections per head of population in the southern hemisphere", or "Australia boasts more camel-related accidents than any other country in the Southern Hemisphere".
How long does it take them to render a reasonably complicated frame in that render farm?
Anyone know?
They used some insider knowledge to get the hardware specs for Doom IV and acted on some more insider dealer connections. Eat that Bush!
From the article:
The system allows artists to render textures, shading and resolution of scenes in an iterative process. The faster they can do that the better the pictures.
Now theres some interesting editor commentary. Since when does drawing somthing faster make it better? It might get it out a month earlier, but really... better pictures?
Finally there's machine that could run WinXP with decent speed (well... atleast like Linux on p90), hmn.. now if only someone would donate it to me.
Plus, with moore's law, those machines they bought won't be worth the electric bill in a few years.
-Milinar
Talk about vaporware!
Marcelo Vanzin
It is networked together with 100Gbps ethernet and Foundry networking switches.
*sigh* My puny Netgear 100Mbps switch is feeling quite inadequate right now.IWARS.
People, in general, disappoint me. Politicians even more so.
do these render farms use any graphics chips or are they done entirely in cpu?
the reason I ask is that linux does not have any high quality open source opengl that supports the latest graphics boards. nvidia is probably the best for opengl support but not opensource.
... what happens to these fine machines when they are retired, or when the studio deems them too slow?
I sure can use one of these, gee, 4GB of RAM, that's more than the entire HD on my current machine.
Ok, don't tell me to go buy a new one. My machine, as old as it is, it running Linux just fine, thank you. Has been serving me for almost 5 years, and 3 or 3 more years, than I'll consider... hehe.
can it play GTA3 at more than 15 fps?
I'm a minister!
a single one of these!!!!
The previous "largest server farm in the Southern Hemisphere" was in Tonga where 7 486s could render a scene from Tribes 2 in less than 17 minutes.
So suck on that Tonga. And you never had the first dawn of the new millenium either.
:wq
Gee, that's one unit upgrade that Blizzard obviously forgot to include in Warcraft III.
A plough here, a grain store there, and voila, +50% to your food output. I'm surprised that nobody's thought of it before...
"Accept that some days you are the pigeon, and some days you are the statue." - David Brent, Wernham Hogg
At least it's what I thunk whens I readed they were doing to go with Xenons. Didn't you thunk that to? It's amazed me that we're readed thats tripe.
"During times of universal deceit, telling the truth becomes a revolutionary act." - George Orwell, 1984
Slashdot and professionalim in the same sentence has to be some sort of error.
Professionalism methinks?
Scotty: "I just can't do it captain! I don't have the power!"
:)
"It's not the size of the render farm, it's how you use it."
And of course, let's all imagine a Beowulf cluster of... oh. wait. Right.
(Obligatory. Didn't say it was funny)
Umm, graphic (or video) cards don't actually do any rendering. It's always the cpu. The only thing you need a high-end video card for is (pre)visualization when you are modelling. No sir, it's just that roomful of noble gas doing the work there.
"During times of universal deceit, telling the truth becomes a revolutionary act." - George Orwell, 1984
There's always a bottleneck somewhere. It's been the drives, the bus, the expansion slots, the network, the ram...
Our biggest issue right now (in my mind, anyway) is physical media. Sure, ATA 133 is burstable to 133, but who actually thinks they'll get 133 for any length of time. If you Cause Win98 to hang at the End Task screen, the buffer on the drive might fill up and you could get maybe a half-second of 133.
The only way to get great speeds out of media is RAID striping or other such technologies.
Don't know if the cluster they have set up uses much (decentralized) storage, but the network has got to be huge.
That what was all this school was for... to teach us how to solve our own problems. -- janeowit
I know this is offtopic...but probably close to the heart of many /. readers
why can't I see the value of my Karma any more? The number has been replaced with a descriptive word.
Does your Karma show up as a number still?
Comment removed based on user account deletion
Funny how the article mentions they use RenderMan for rendering, yet neglects to mention they use Pixar's "Alfred" software for job distribution/control....
http://www.pixar.com/renderman/artist_tools/too
I looked on the Foundry website, 'only' 10Gbit.
I hate those exponential powers!
I am not a robot. I am a unicorn.
Hmm, are these perhaps the new Dell PowerEdge 2650s? We just got one for our lab (dual Xeon 2.4 Ghz, 4GB DDR RAM). Mehopes that they know how loud these machines get, since one of them litterally sounds like a jet engine and is the loudest computer I have ever heard. Just search groups.google.com for "PowerEdge 2650 Loud Fan" to see what I mean.
nak
I live near where Weta is situated, so once I graduate at the end of the year in Software Engineering (Bachelor of Engineering) I'll be trying there!!!
can't you spell 'cool' properly?
and if you like to smoke the linux feces, then why would running XP be 'way cool'?
I've always wanted to use Blender's "Render daemon" button...
Seriously though, does anyone know what kind of modelling and render tools these guys are using ?
'sapientia potestas est'
Rendered at the most powerful site in the Southern Hemisphere ... on Linux!
Everyone is born right-handed; only the greatest overcome it
-Eugenia
+all women
What a gas!
example.org - powered by Linux!
There's a show on FOX called "American Idol", the long and short of it is that they hope to create a "pop superstar" ala NSync or whatever. The voting for this show is done by the public via telephone. Please call this toll-free number 1 (866) 436-5707 to vote for the most retarded of all the contestants just to undermine the results of this show. Must call within the next 2 hours. Pass it on!
I'm an extremely technical person, but in the world of materials engineering and such. I read /. for computer news and have for 2.5 years now. However, I'm only now starting to do any programming just out of sheer interest. So I don't know squat about this.
Do the render farms work simply as individual nodes rendering individual frames as doled out by some server, or do they work collectively. As in: are they done serially or in parallel? I don't mean this part as a joke, but is the beowulf cluster concept meant to be a single fast computer made of many?
Dumb questions brought on by enough misunderstanding.
to email me: take my
And after all
You're my renderwall....
Thank you Chicago -- Goodnight!
A blend of Maya, Softimage, Houdini, proprietary code, and Photorealistic Renderman
Yah! It was gonna be called Freon!
Strangely, the idea got a very chilly reception, though everyone complained when it eventually got banned.
Get off my launchpad!
a cheap Mac Dual G4 1 Ghz is FASTER.
Its $2799 and comes with a 300 dollar DVD-r burner as part of the price.
It gets over TWICE as many RC5 keys per second than the fastest dual AMD MBs... and per dollar, the XEON is way slower than a Mac at RC5.
Macs have a huge L3 cache and AMDs have no l3 cache so that might be one reason macs are twice as fast, plus a cold page of memory can be simultaneously write and read faster on a mac than a pc typically. That helps for some benchmarks perhaps.
but if you go to TOP500.org they maintain a ranked FRESH list of all the top 500 render/gfop cpu farms.
xeons are rare in the list and Powerpc boards dominate it. check out the list yourself.
Admittedly a better list would be megabyte per gigaflop... a conttest each fall determines that and typically dual Pentium 3 boards with three netwrok cards each win that award.... not this overpriced xeon garbage.
I like xoen for one thing... it has PCI-X now and has for over a year and no AMD has PCI-X shipping yet. Check Pricewatch.com yourself.
PCI-X will ship on macs soon and tahts all I care about.
Somebody has to say it:
Can you imagine a beowulf cluster of these?
Urgo: "I want to live. I want to experience the universe and I want to eat pie!"
Jack: "Who doesn't??"
In ten years, you can get a 120 Terabyte drive. Only one problem: What the hell would you put on it to fill it up?
MS Windows XP 8. Duh.
.
I wouldn't call $4 million NZ (or even US) to be a tremendous amount of capital, for a project like this. If they never use it for another project, they will still have gotten their money out of it. Sure, its a lot of money to me, but I bet they spent more than that on film stock.
But I think the other guy is right and Peter Jackson will make good use of this equipment and these people in the future.
Don't moderate flamebait as Troll. Know the difference or you will be Meta-moderated.
Everytime I read an article like this, it ticks me off.
If they can get this sort of application running on a Linux system, why can't dreamweaver and Adobe port their products to Linux.
Hell and damnation.
I mean it can't be that hard.
I guess the reason is because noone would buy the ports.
Is Linux ultimately only useful to the custom solution and server crowd. Will the professional and consumer desktop ever be tamed?
evanchik.net
They should use that in the FotR ad campaign... 'Rendered using the most powerful processing site in the southern hemisphere'
mm hmm, that would definitely sell all... the.... people who were going to see it anyway (geeks) =)
Obviously they don't know about FreeBSD, else they would dump Linux faster than a hobbit runs from adventure. Perhaps someone should clue them in.
Shouldn't the link be:
220 2.2GHz dual Xeon machines
not
220 2.2GHz dual Xenon machines ?
Not that I'd expect any sort of technical accuracy from this site. It's not like it's the most visited geek site on the 'net or anything...
nuclear presidential echelon assassination encryption virulent strain
Whizzmo
Geek MPAA movies (eg LOTR): good
Linux: good
Using Linux to further the MPAA: good
</karmaburn>
sorry, couldn't resist
Live today. Tomorrow will cost a lot more!
Like most of the post houses they use combinations of software, Maya, Renderman, in house tools, etc.
the hardware cost is usually quite small compared to the cost of the software licenses which get licensed per CPU, that's why you go for 'smaller' numbers of fast CPUs especially when you have enough to get over the minimum number to keep all the artists flowing.
Faster CPUs generally does mean better pictures as you get more goes at it to get it right. There is a degree of artistic control you get from getting feedback...
They also have to work to deadlines and fit in with client demands/expectations.
With this many machine it does pose interesting server requirements... all those images (usually about 12MB each) do have to come and go from somewhwere, oh and you have to be able to find them easily, etc. etc.
...Maybe they can force southern-hemisphere toilets to flush counter-clockwise!!!
-
Granted, they may be using a motherboard with more than 4 DIMM/RIMM slots....
But even still. The prices don't skew THAT much for DDR (though they skew a lot more highly for RDRAM).
Chas - The one, the only.
THANK GOD!!!
... the thought of the most powerful processing site in the southern hemisphere is located in a country whose poeple think a PII(2) machine is soemthing to yell about.
Oh the shame...
Sometimes I wish I was a plumber, then I'd know how to deal with other people's shit.
Slashdot's so funny,
"Down with the MPAA, movie makers are the bad guys" in one article then,
"Go movie makers! Using lots of Linux, yipee"
go here
btw - my FP!
My other sig is a Porsche.
They use Maya for modelling, and renderman for rendering.
The revolution will not be televised. It won't be on a friggin blog either
Has it occured to you that the southern hemisphere consists almost entirely of southern africa, south america, australia, indonesia and a couple islands? It's not saying all that much to have the fastest computer, Were probabally talking about a population half that of the USA, and most of it is the "developing world."
People who think they know everything really piss off those of us that actually do.
Massey university just announced that it is going to build a 128 node beowulf cluster (no imagination necessary!). Auckland University have recently got an IBM Regatta class machine.
Just a (quite impressive) stone's throw away from Weta is NIWA's Cray T3E
bash-2.03$ uname -a
sn6908 kupe 2.0.5.51 unicosmk CRAY T3E
I love running that uname :-)
Backups are for wimps. Real men post their data in comments and have slashdot mirror it
Shouldn't we be disappointed that "the most powerful processing site in the Southern Hemisphere" is doing nothing more important than entertainment? Surely there is some real problem we could be solving with our collective resources!
Oh well, see you at the theatre on opening night -- I'll be at the 12:05 a.m. show.
"We reject as false the choice between our safety and our ideals." --The American President (20.1.2009)
er.. not entirely true if you read the article and the one in cinescape you will see that all the crowd rendering is done by an in house renderer called Grunt
I love seeing these Beowolf clusters.
1 question tho, where the quake? They have to have a massive connection to the internet and they have the most clock cycles.
If they haven't install it yet, I would be more than happy to lend my services.
mmmhhhh xenon
I bet they could find aliens in a day if they ran seti on all of those machines.
a Beowulf cluster of these beowulf clusters...
...and not one of them is a Mac!
But... but... I thought you couldn't be a creative professional without using Steve's greatly insane doorstop!!!
And not before time. Before this purchase all of New Zealand's computing needs were met using a time shared Commodore 64. (I'm a Kiwi, I "can" make jokes like this.)
If anyone from the company is looking on /. trying to get some rid of their old almost-super computers to make way for the new super computers, I can probably manage to take a few off your hands.
:kicks it into an open fire:
Bart: 256? Oh and I'm stuck with this useless 252
Gamestation 252: Don't destroy me! I can still make you happy! To the maaaax!
$8.95/mo web hosting
Good thing I got them on sale!
3. Profit!
2. ???
1. On Soviet Slashdot, a Beowulf cluster of alien Natalie Portman overlords welcomes YOU!
Because if it would/could have been, they would probably have gone for a dual AMD system.
You pay a lot more per XEON CPU, you pay a bit more for RAM (and the bit more adds up pretty quickly with 200 machines with 4GB), you pay a LOT more for the motherboard. I've had do to a renderfarm with budget restrictions, I got twice as much machines for the same price if that intel-based setup (and almost twice as much power).
Stability? not any unusual issues that I wouldn't have got also with Intel-based stuff. I bought TYAN TIGER MPs, with dual athlon XP, and the hardware is top notch. The only issue I could see is if everything is heavily optimized for SSE2 and money isn't a problem, then it would make sense to grab P4 XEONs, but that's the only case I'd see (aside from marketting or direct rebates from intel for free exposure, etc etc) that could make someone take such a decision.
My 0.02 cents.
--- Metamoderating abusive downgraders since my 300th post.
Dual Xeon's, handles at least 4 GB RAM, and either on-board or PCI Gbit ethernet (or is it just 100Mbit as someone pointed out?). AND it's in a 1U case? I didn't know Xeon's (even the new, non-Slot 2 ones) fit in a 1U case.
The Dell 2650 is 2U.
I wonder why they didn't go with Blades? 476 servers equates to 1 1/2 racks full of blade servers. Not something that comes across as the most powerful render farm in the southern hemisphere.
WRONG! Many standard Apple Powerpcs in Top500.org!
...
:
You never heard of "ASCI Blue"?
It uses PowerPC 604e... the same chip my PowerMac 8500 uses.
There are many Power3 top500 clusters, such as position #2, but there are also many PowerPC604e spots
check the list yourself and quit lying by saying that there are not mac-style cpus : here are some (7) I saw in current ranked list
ASCI Blue-Pacific SST, IBM SP 604e
Bank Administration Institute (BAI) SP PC604e 332 MHz
BCDI - SP PC604e 332 MHz
Metallurgical Industry Co. - SP PC604e 332 MHz
BASF SP PC604e 332 MHz
DeTeCSM
DeTeCSM
And NONE of the top500.org uses linux OS... none as far as I know. That was true a year ago and is still true.
Macs are indeed 1 Ghz , by the way. Thats why the XServe is over twice as fast as the fastest amd dual mp at rc5... macs arent only 500Mhz, they are 1Ghz in recent months, standard and cheap.
If you are lamenting poor literacy on Slashdot then perhaps you should have known "become" would be a better choice of word rather than "gotten".