Slashdot Mirror
Apple Wins VT in Cost. vs. Performance
danigiri writes "Detailed notes about a presentation at Virginia Tech are posted by by an attending student. copied most of the slides of the facts presentation and wrote down their comments. He wrote some insightful notes and info snippets, like the fact that Apple gave the cheapest deal of machines with chassis, beating Dell, IBM, HP. They are definitely going to use some in-house fault-tolerance software to prevent the odd memory-bit error on such a bunch of non-error-tolerant RAM and any other hard or soft glitches. The G5 cluster will be accepting first apps around-November."
mfago adds, "Apple beat Dell, IBM and others based on Cost vs. Performance alone, and it will run Mac OS X because 'there is not enough support for Linux.'"
Dell - too expensive [one of the reasons for the project being so "hush hush" was that dell was exploring pricing options during bidding]
;)
Who could have guessed?
I fought the corporate America, and the corporate America bought the law.
How does that work?? How does software even KNOW if there was a glitch? Can I get this on my non-ECC Linux box????
IMHO the lack of ECC RAM is the only flaw in an otherwise perfect machine (well that, and the massive HEAT).
traditional methods [fans] would have produced windspeeds of 60+ MPH....
I wonder how they are all orientated? I wonder if photos of the final result will be released.
I'm suprized not to find Power consumption/heat dissapation in the presentation. You'd think that the cost go running 1100 CPU 24/7 would be mentioned in there. Although it's likely that Apple would have wone that also.
Full operation by Jan 1 2004- that's cool.
3 MW power, double redundant with backups - UPS and diesel
1.5 MW reserved for the TCF
2+ million BTUs of cooling capacity using Liebert's extreme density cooling (rack mounted cooling via liquid refrigerant)
traditional methods [fans] would have produced windspeeds of 60+ MPH
Does it render?
Yes, incidentally, it does. The units came with high end graphics cards
Aside from games, when is a high end graphics card needed for rendering and not just displaying a rendering.
One of the primary concerns for a multi-node cluster is insured latency among all components within the cluster. It doesn't have to be the fastest, it just needs to insured exacting timing for latency across all nodes. IBM can do this with their "wormhole" switch routing on SP and has done this with Myranet on their Intel X-series clusters.
m l
From most of my reading with Infiniband, it was designed from the ground up as a NAS style solution, than for large multi-node cluster computing. I'm curious as to if they have any issues with cluster latency.
http://www.nwfusion.com/news/2002/1211sandia.ht
The primary timings and white papers I've seen published for Infiniband have been for small clustered filesystem access. Although it's burst rate is much higher than Myranet, it's hard to find any raw retails for their multiple node latency normalization.
I hope it scales, since Intel's solution appears to be less cost prohibitive than some of the other solutions offered on the market, and would really open up the market even for smaller clusters (16-36 node) for business use.
I wonder if by "lack of support in linux," that they're refering to the fact that the fans are controlled by the operating system in the powermac? Or the fact that there are relatively few support companies for ppc linux?
Any insiders care to comment?
You say
Okay before we get going with the same discussion about ECC vs. Non ECC, and all the flames start from people perusing slashdot who think they are more in the know than the PhD's at VT who have been working on this for months I want to point a few things out.
1. The majority if not all of the bit errors that ECC corrects are caused by thermal noise. Thermal noise is an issue in a cluster of rack mounted 1U units due to the difficulty of cooling such tightly spaced units generating so much heat in so small a space. It is not an issue in a cluster of DESKTOP machines utilizing a Liebert system with way more cooling capacity than is needed.
2. Even if somehow a none-thermal bit error occurs, each node has 4GB RAM. The probability of it being in an OS or application critical (especially given the converging nature of many long running calculations) piece of RAM as opposed to an empty piece of RAM is small.
How many of you are reading this from a desktop without ECC RAM that has an obnoxiously huge uptime? ECC is a non-issue in a well-cooled cluster of desktop cased machines.
Looks like the costs come out to $23,636 per node, or $4727 per machine. According to the Apple Store, an equivalently specced machine (dual proc G5, 160GB HD, 1GB RAM) comes out to just a little over $3,000. I suppose you might want a display on the management machine in each node, but that won't raise the price that much (say, $3,200 per machine instead). So that leaves ~$1,500 per machine for the networking hardware and whatever other expenses.
pb Reply or e-mail; don't vaguely moderate.
Certain things are easy to imagine in large quantities, but dude.
Just....dude....
Kick in the Head
A Multiplayer Strategy Game for Mac OS X, Windows, and Linux
I am a scientist, and lots of money gets put into transforming the tons of numbers that supercomputers produce into images that make sense to the human brain.
e Arctic Region Supercomputing Center:
The system doesn't have to be chaotic, just complex:
Watching protein folding simulations.
Watching full 3-D seismic waves propagate through the Earth.
Watching, in general, any kind of 3-D model or simulation of a complex process evolving over time.
A couple links:
The Scripps Institute of Oceanography Visualization Center:
http://siovizcenter.ucsd.edu/library/objects/
Th
http://www.arsc.edu/news/mdflex.html
ECC memory too.
And never forget the costs of installing these puppies. Cooling systems, power busses, cable harnesses, UPS, Diesel backups, Air filtering, locks, redundant parts.
and what about the disk servers....
Some drink at the fountain of knowledge. Others just gargle.
I wasn't sure how they were using the word 'node' there. That would raise the price to... $5,120.00 per machine! Their consumer prices for RAM must be hugely inflated, seeing as how you could get a 1U dual 2Ghz Opteron with 4GB RAM for $4,500.00...
pb Reply or e-mail; don't vaguely moderate.
I don't know if the cost of installing them was included in that estimate, but maybe it was.
As for the disk 'servers', I figured they were just sharing all of the 160GB HDs over the network, seeing as how 160GB x 1100 ~= 176TB (ok, it's more like 172TB, but who's counting...)
pb Reply or e-mail; don't vaguely moderate.
A few years ago I asked apple if they would be willing to sell me 200 laptops without the screens, disks, video cards, and keyboards. They were interested helping me build my cluster but, the the engineers said it would actually cost them more to have a special manufacturing run than woul dbe saved by deleting the hardware.
my plan was stack these things on water cooled chill plates. Basically this would be like a blade.
In my circumstances, adding a well ventilated computer room to the building I was in would have been probibitively expensive. but water cooling and a high density configuration made this very appealing. And if I could have gotten the costs down and reliability up by deleting the screens, keyboard, video, and disks I'd have an affordable system with low sys-admin costs.
I still think its a good idea. Cooling/power costs (including building retrofits) and sys admin costs can dominate the differential purchase price of vairous cluster configurations. In my building the space alone was >120/sq foot, so even the footprint mattered.
Some drink at the fountain of knowledge. Others just gargle.
that apple is going to push back the delivery of the dual-2giggers for us regular folk again so they can pump these things out to this university (like they did last week w/ their announcement about holding up orders for the dual-2giggers to ship them to highschools)...
Large print giveth, and the small print taketh away
http://nowlab.cis.ohio-state.edu/projects/mpi-iba/
If they do not fit into a standard rack enclosure, I would be curious to learn what customization was required to rack the G5s.
(Especially seeing as a G5 XServe will probably be at least several months away -- at least until most of the desktop orders can be filled.)
-Alex
Man they really blew it. They should have ordered it from macmall. it would have come with 1000 free printers and 1000 ipod cases.
Some drink at the fountain of knowledge. Others just gargle.
I believe you can get a VT for well under $1000, and I've even heard that some of them now support advanced "sixel" graphics.
And they scroll MUCH more smoothly than OS X.
"How to Do Nothing," kids activities, back in print!
on macslash this story talks about the crazy speeds they are claiming on the G5 running Gentoo Linux. Says they can not go superfast because of fan control issues still unresolved, but yikes! too good to be true?
no, you're thinking of maine. the state of maine bought laptops for every 7th grade student, if i remember correctly.
Actually, Srinidhi Varadarajan (who gave the first portion of the presentation) said that there would only be 4GB of RAM in each machine. Why not 8GB, I don't know.
-Waldo Jaquith
Sorry, i couldn't help myself. Really i am. Go ahead an mod this to the deep bowels of /.. I'l soo sorry i did this.
I want to know how VT was able to do it's cost analysis so fast.
From what I've heard, VT ordered the G5 the day they came out, or shortly after. But if one were to perform a cost vs performance, they would need background data. Also, they should have been hesitant to accept Apple's specs on the machine, and hoped for some real world test, or maybe some in-house testing of a few machines.
I find it hard to believe that VT was able to truly compare the G5 to competitor products, with out prior data of the machines.
On the other hand, I would say that some of the software features of OS X would make the machines fairly cheaper in terms of setup time. I don't know if they use Rendezvous (I know that some software uses it for distributed computing, Final Cut Pro and XCode in particular). Also, the other OS X features that have already been mentioned like cooling and the fact that it is UNIX based.
Anyways, this is enough of a rant, and I'll let someone else have the floor now.
As an American company, Dell is a huge disgrace. Please read the "Environmental Report Card" produced by the Silicon Valley Toxics Coalition. Dell received a failing grade and is little better than Taiwanese companies, which are notorious for destroying the environment and the health of workers. Dell even resorted to prison labor to implement its pathetic recycling program.
These are PowerMac G5s, not XServes. The default card for the G5 is AT LEAST a GF FX 5200. Apple doesn't make low-end crap cards. VT got a SMOKING deal on these boxes, and for Apple to pull them all apart to put in crappy videocards (or even just to take them out) would have been a huge expense.
There is nothing worse than having a computer without ECC or parity memory, and trying to detect and diagnose subtle pattern sensitivity memory problems.
Besides thermal noise, you also have to consider alpha particle emissions from packaging materials and cosmic rays. There are also electrical noise issues in memory assemblies.
Mea navis aericumbens anguillis abundat
From the slide note:
additional player: PSC [yeah I don't know either]
PSC -> Pittsburgh Super Computing Center running by Carnegie Mellon
Yeah, but NO BUGS!
You know what?
This is a joke, not a troll, you silly Apple fascists.
-73, de n1ywb
www.n1ywb.com
Yes, but the -1 (Stupid) moderation option hasn't been implemented yet by Taco. ;)
-T
And someone decides I'm a guy. Thanks. Not that any of that matters of course... as per the cluster, they've been assembling at an amazing pace - such that the shifts I volunteered for have all been cancelled as they will not be needed after all. They even cabled the sucker without me... sadness! -Myuuchan