Slashdot Mirror
Apple Wins VT in Cost. vs. Performance
danigiri writes "Detailed notes about a presentation at Virginia Tech are posted by by an attending student. copied most of the slides of the facts presentation and wrote down their comments. He wrote some insightful notes and info snippets, like the fact that Apple gave the cheapest deal of machines with chassis, beating Dell, IBM, HP. They are definitely going to use some in-house fault-tolerance software to prevent the odd memory-bit error on such a bunch of non-error-tolerant RAM and any other hard or soft glitches. The G5 cluster will be accepting first apps around-November."
mfago adds, "Apple beat Dell, IBM and others based on Cost vs. Performance alone, and it will run Mac OS X because 'there is not enough support for Linux.'"
Dell - too expensive [one of the reasons for the project being so "hush hush" was that dell was exploring pricing options during bidding]
;)
Who could have guessed?
I fought the corporate America, and the corporate America bought the law.
3 MW power, double redundant with backups - UPS and diesel
1.5 MW reserved for the TCF
2+ million BTUs of cooling capacity using Liebert's extreme density cooling (rack mounted cooling via liquid refrigerant)
traditional methods [fans] would have produced windspeeds of 60+ MPH
IANAS, but:
- Graphical representation of turbulance systems?
- Wheather analysis?
- Any graphical representation of Chaotic systems?
Like I said, IANAS, but there HAS to be a reason, methinks.Quod scripsi, scripsi.
I'm suprized not to find Power consumption/heat dissapation in the presentation.
Power consumption would have only made Apple look better.
Healthcare article at Kuro5hin
One of the primary concerns for a multi-node cluster is insured latency among all components within the cluster. It doesn't have to be the fastest, it just needs to insured exacting timing for latency across all nodes. IBM can do this with their "wormhole" switch routing on SP and has done this with Myranet on their Intel X-series clusters.
m l
From most of my reading with Infiniband, it was designed from the ground up as a NAS style solution, than for large multi-node cluster computing. I'm curious as to if they have any issues with cluster latency.
http://www.nwfusion.com/news/2002/1211sandia.ht
The primary timings and white papers I've seen published for Infiniband have been for small clustered filesystem access. Although it's burst rate is much higher than Myranet, it's hard to find any raw retails for their multiple node latency normalization.
I hope it scales, since Intel's solution appears to be less cost prohibitive than some of the other solutions offered on the market, and would really open up the market even for smaller clusters (16-36 node) for business use.
# 3 MW power, double redundant with backups - UPS and diesel * 1.5 MW reserved for the TCF
# 2+ million BTUs of cooling capacity using Liebert's extreme density cooling (rack mounted cooling via liquid refrigerant) * traditional methods [fans] would have produced windspeeds of 60+ MPH
Seems that they did talk about both.
I wonder if by "lack of support in linux," that they're refering to the fact that the fans are controlled by the operating system in the powermac? Or the fact that there are relatively few support companies for ppc linux?
Any insiders care to comment?
You say
Okay before we get going with the same discussion about ECC vs. Non ECC, and all the flames start from people perusing slashdot who think they are more in the know than the PhD's at VT who have been working on this for months I want to point a few things out.
1. The majority if not all of the bit errors that ECC corrects are caused by thermal noise. Thermal noise is an issue in a cluster of rack mounted 1U units due to the difficulty of cooling such tightly spaced units generating so much heat in so small a space. It is not an issue in a cluster of DESKTOP machines utilizing a Liebert system with way more cooling capacity than is needed.
2. Even if somehow a none-thermal bit error occurs, each node has 4GB RAM. The probability of it being in an OS or application critical (especially given the converging nature of many long running calculations) piece of RAM as opposed to an empty piece of RAM is small.
How many of you are reading this from a desktop without ECC RAM that has an obnoxiously huge uptime? ECC is a non-issue in a well-cooled cluster of desktop cased machines.
Looks like the costs come out to $23,636 per node, or $4727 per machine. According to the Apple Store, an equivalently specced machine (dual proc G5, 160GB HD, 1GB RAM) comes out to just a little over $3,000. I suppose you might want a display on the management machine in each node, but that won't raise the price that much (say, $3,200 per machine instead). So that leaves ~$1,500 per machine for the networking hardware and whatever other expenses.
pb Reply or e-mail; don't vaguely moderate.
Certain things are easy to imagine in large quantities, but dude.
Just....dude....
Kick in the Head
A Multiplayer Strategy Game for Mac OS X, Windows, and Linux
for any represantation, you need only 1 graphics card : the one the monitor is attached to. Parallelizing realtime display-only stuff is not much good since you'd lose to much time in data transmission.
So they could equip one G5 with a radeon9800 and let that one display the results. No need to buy another 1099 Radeons.
When will I end this grieving ? When will my future begin ?
I am a scientist, and lots of money gets put into transforming the tons of numbers that supercomputers produce into images that make sense to the human brain.
e Arctic Region Supercomputing Center:
The system doesn't have to be chaotic, just complex:
Watching protein folding simulations.
Watching full 3-D seismic waves propagate through the Earth.
Watching, in general, any kind of 3-D model or simulation of a complex process evolving over time.
A couple links:
The Scripps Institute of Oceanography Visualization Center:
http://siovizcenter.ucsd.edu/library/objects/
Th
http://www.arsc.edu/news/mdflex.html
A few years ago I asked apple if they would be willing to sell me 200 laptops without the screens, disks, video cards, and keyboards. They were interested helping me build my cluster but, the the engineers said it would actually cost them more to have a special manufacturing run than woul dbe saved by deleting the hardware.
my plan was stack these things on water cooled chill plates. Basically this would be like a blade.
In my circumstances, adding a well ventilated computer room to the building I was in would have been probibitively expensive. but water cooling and a high density configuration made this very appealing. And if I could have gotten the costs down and reliability up by deleting the screens, keyboard, video, and disks I'd have an affordable system with low sys-admin costs.
I still think its a good idea. Cooling/power costs (including building retrofits) and sys admin costs can dominate the differential purchase price of vairous cluster configurations. In my building the space alone was >120/sq foot, so even the footprint mattered.
Some drink at the fountain of knowledge. Others just gargle.
http://nowlab.cis.ohio-state.edu/projects/mpi-iba/
If they do not fit into a standard rack enclosure, I would be curious to learn what customization was required to rack the G5s.
(Especially seeing as a G5 XServe will probably be at least several months away -- at least until most of the desktop orders can be filled.)
-Alex
Man they really blew it. They should have ordered it from macmall. it would have come with 1000 free printers and 1000 ipod cases.
Some drink at the fountain of knowledge. Others just gargle.
What did you think last weeks announcement was???
Mod point free since 2001
I believe you can get a VT for well under $1000, and I've even heard that some of them now support advanced "sixel" graphics.
And they scroll MUCH more smoothly than OS X.
"How to Do Nothing," kids activities, back in print!
Actually, you're not totally right. You can spread the rendering job across multiple radeon chips, each handling only a portion of the display. The performance and depth of the rendering could be greatly enhanced. SGI does something like this...
Not all renders are real time, not all renders are onto a screen.
.... on a twin G5 with a Radeon 9800 ... then you can render 24fps in real time. Real time lord of the rings, anyone?
Now that "consumer" graphics cards run in floating point and have comparitively complex shader engines, it's quite possible to start working on rendering movies etc. with the substantial quantity of hardware acceleration possible on these things. You don't have to hit 60fps, and you can have as many passes as you like.
Mind you, with 1100 nodes if you can render a frame in 45 seconds
Dave
I write a blog now, you should be afraid.
the crazy speeds they are claiming on the G5 running Gentoo Linux
:).
Why hell, i get blazing speeds with gentoo on my Athlon, i'd sure hope that you'd get them on the g5 as well
As an American company, Dell is a huge disgrace. Please read the "Environmental Report Card" produced by the Silicon Valley Toxics Coalition. Dell received a failing grade and is little better than Taiwanese companies, which are notorious for destroying the environment and the health of workers. Dell even resorted to prison labor to implement its pathetic recycling program.
I bet at the time of initial consideration of vendors, there were no competitive Opteron or Itanium solutions (none with chassis, the slides say), and I am also willing to bet that Apple had at least a hardware prototype they could demonstrate, at least a motherboard + dual CPU setup, even if the chassis was incomplete and the not all the major subsystems were 100%
Just enough to demonstrate that Apple *would* have a solution, and enough that VT could narrow down the decision to a possible, pending the actual production and purchase of a single machine... then, the contract being 99% complete, they just had to sign a couple papers and purchase, overnight, 1,100 dual G5s.
On the flip side I bet they had a similar contract in the wings with other vendors, all pending on 'simple' bottlenecks.
GPL Deconstructed
that is useful only of those multiple cards are IN THE SAME MACHINE. In the cluster case, those cards are spread over multiple computers, requiring that you transfer the rendered result over the network to the "master" video card which sends it to the monitor. I seriously doubt the efficiency of such a solution for realtime display.
When will I end this grieving ? When will my future begin ?
That's not necessarily true. Over at Stanford for the project they built a graphics system with 32 PCs that render to a tiled display. Imagine a display made of 1000 monitors in a 40x20 grid. That would be pretty freaking cool.