Slashdot Mirror
Pixar Eclipses Sun with Linux/Intel
lieutenant writes "Pixar Animation Studios is replacing servers from Sun in its render farm with eight new blade servers from Rackspace. In all, the blade system contains 1,024 Intel 2.8GHz Xeon processors, and it runs the open-source Linux operating system. Pixar has ported its Renderman software to run on Linux." I'd love to see their electric bill ;)
With that type of processing power, they should be able to calculate to infinity...and beyond.
If you have a task that can be easily partitioned off (oh like each individual frame would be an easy break for this) you can send each task to a different machine allowing you to parellelize the task.
This is a poor mans version of NUMA (Non Uniform Memory Access) created and popularized by Sequent (now a division of IBM) where rather than have a single pool of addressable memory, you have multiple pools of memory, some with very fast access, some with slower.
What I am wondering is what do they do for the cluster cross connect. In large scale cluster environments, this tends to be a significant bottleneck. In large scale clusters you start seeing things like HIPPI, VIA, and soon to be Infiniband... wonder what this is stocked up with
I have mod points and I am not afraid to use them
As far as I know Rackspace is a managed hosting company. Rackable Systems makes servers - Yahoo and Google both use them. Anyone know if the article has it wrong, and Pixar is actually using Rackable machines?
isn't
720 hours @ 75,776W = 54,558,720kWh.
actually 54,558,720Wh (watt-hours, not kilowatt hours), which is 54,558kWh
making it not 3.2million, but only $3200 a month?
I teach MCSE courses down in Chatsworth, recently we got a lot of Engineers from boeing coming over for Windows XP classes. Why? They're dumping all their Sparc Stations and moving to XP on cheap Intel hardware. Its faster, and 2/3s of the applications they need run it already. The last third they were working on.
The IT people I talked to were surprisingly happy with XP so far. These were all Unix only kind of people actually.
The other thing they were doing were looking into dumping their Crays in favor of LINUX clusters. The comments were along the lines of how much faster and cheaper it was to put together a cluster of a 100 cheap Intel boxes than getting a new Cray. That, and they were all already familiar with the unix style interface. On top of it all, the GUI interface (I think they were running Gnome) was so much nicer than CDE on Solaris.
So Sun it getting it from both sides- Cheap Wintel boxes and Cheap Linux boxes. No wonder they finally relented and released Solaris 9 on Intel.
They have half-depth 1U boxes. That's right, two servers in 1U, back to back.
Includes space between the two for cabling and cooling.
They specialize in delivering easy to manage (physically) racks of highly commoditized systems.
(I work with them in a reseller relationship)
Imagine a 71U rack(minus 1U for a switch), with 142 boxes, all dual proc. 248 procs in a rack!
Man, I wish they'd put the right link in there.
Striving to achieve a lower state of conciousness
> > For around $25,000 you too can make Pixar quality movies
>
And if that's too expensive you can forgo the creative talent and make Star Wars prequels!
Sheesh, evil *and* a jerk. -- Jade
A lot of people are going to be saying "just one example of how Sun is dying", but coming from a place that runs several hundred Sun machines (and being a Sun fanboy), I can understand why they made this switch. For shere processing power on-the-cheap, the x86 world has had a lead on Sun and other big UNIX vendors for a few years. Having a decent OS (linux) to run on those machines, makes it even easier to switch.
It's about using the right tool for the job, and now that x86/linux/bsd has matured to a point where it can be used for some professional applications, it only makes sense to see things like this happen.
Sun is going to be around for a long time. As many other people have pointed out, they're just retreating somewhat to more a of niche market, where they are the right tool for the job.
The executives at my company are very interested in linux, because of the outrageous leap in processing power per dollar, and the reductions in CPU-based licensing costs for software like Oracle is staggering. The concern, though, is stability.
Sun Fire and Enterprise servers are really expensive, but they stay up all the time. Swapping a failed processor or NIC or memory stick without halting the box is really important on a mission-critical server. Likewise, a well built Sun box never panics, and if it ever does, Sun will insist that their engineers look at the crash dump to figure out what went wrong.
I think Linux has won the performance battle, but what about the stability battle? You need to win both to win the war.
I recently attended a talk by Google's chief engineer. They have approximately 15,000 x86 machines running Linux at seven data centers in the United States.
Weird failures occur so often, such as disks returning garbage without the controller informing the OS, that Google does a checksum on _every_ data structure in their user-level software. He also talked about how Linux is good enough for them, but it doesn't perform well with respects to I/O under heavy load. He says they like Linux because they have the source-code and that they minimize excessive I/O loads on their machines. Nobody asked why they don't use FreeBSD but I suspect its because Linux has better hardware support and Google builds their own machines with numerous different components based on the latest technology.
... I guess we know which one you are.
1982 - born in Nebula - incorporated with 4 employees
1984 - protostar - NFS is introduced
1995 - main sequence begins - Java Released
1996 - red giant - Using Java technology, NASA engineers develop an interactive application allowing anyone on the Internet to be a "virtual participant" in the space administration's groundbreaking mission to Mars.
SUPERNOVA - Sun battles MS over Java and Windows
Blackhole - TODAY!
References:
http://www.sun.com/aboutsun/coinfo/history.html
Its all about the distinction between shared and distributed memory architectures. Different applications benefit from different types of parralelism which the above architectures provide. If to solve the problem independent chunks of code can be run that require no communication at run time then clearly a blade type solution (distrbiuted memory) is viable, but if the calculations are co dependent on each other and require communication of interrim results then the overhead of communication can quickly become the critical path and shared memory parallelism becomes a better solution. It also depends on the level of parralelilsm built into the implementation of the algorithms inside pixars redering program itself.
Its one damn thing before another. (Dick Bird 1999)
Sorry Guys... This article looks to be a bit off base!
/. I though you guys did better about checking this kind of thing out! Just because it's on c-net doesn't mean it's accurate. Well kudos to who ever really got this job.
-- Not an Official RS response --
I work for Rackspace Managed Hosting. The company the link "Rackspace" references in the C-Net article. This kind of cluster is not consistent with our business. We are most focused on web-centric managed hosting vrs colocation. A rendering cluster is something that, from my experience, we've never done. Also We don't carry Blade servers. C-Mon
Matthew Montgomery
Rackspace Managed Hosting.
I just eclipsed Windows with Linux on my home system.
I just eclipsed my old toothbrush with a new one.
I just eclipsed the shit in my ass-crack with toilet paper.
Now, don't I sound FUCKING STUPID? Yes, I do.
This sig is part of your complete breakfast.