Slashdot Mirror
Dell $38m Supercomputer [not] More Costly than VT's G5s
An anonymous reader writes "According to the Austin Business Journal, Dell's 3-teraflop, 600 server supercomputer cluster cost the University of Texas $38 million. As The Apple Turns has pointed out that this is 7 times the cost (and a quarter of the power) of Apple's cluster at Virginia Tech! " Update: 10/14 17:56 GMT by M : worm eater writes "The Register has posted a correction to the widely-reported story that a 3.7 terraflop Dell cluster cost the University of Texas $38 million. As it turns out, the computer cost $3 million, vs. $5.2 million for the 17.6 terraflop Mac G5 cluster at Virginia Tech."
Monday, 5:57 PM: Virginia Tech's G5-based supercomputer is (sort of) running-- with 17.6 teraflops of theoretical performance. Meanwhile, Dell tries to build something (sort of) similar, but it winds up with a quarter of the power and seven times the price, and Apple (sort of) announces Xgrid, a product for "parallel and distributed high performance computing"...
Monday, 5:57 PM: Today's holiday episode is now broadcasting. Don't forget to take your shot for a free AtAT shirt (tee or turtleneck) by entering the Q4/03 Beat The Analysts contest; guess closest to Apple's final reported quarterly profit or loss, and you get the garment-- or your choice of creaky old software from the Baffling Vault of Antiquity(TM). You've only got until Wednesday at 4 PM, and in the likely event of a tie the earliest entry wins, so why wait? Enter now!
Up, Running, & Kicking Tail (10/13/03) Fun fact: believe it or not, folks, AtAT's wild success isn't confined to these here United States. No, seriously, it's true! The show actually has semi-regular viewers holed up in such far-flung corners of the world as Iceland, the Dominican Republic, and Delaware-- and for the benefit of those fans, we thought we'd explain that, here in the U.S., today we celebrate a holiday called Columbus Day. Columbus Day, for the uninitiated, is one of our most sacred occasions: a day on which we reflect on the many cultural and technical achievements of the city of Columbus, Ohio. We celebrate Greater Columbus's world-famous quilts, its shrubberies recreating Pointillist masterpieces, and (most importantly) its commitment to the preservation of really old TV sets by wondering why the bank is closed and our mail never came. A good time is had by all.
So if this is such a major holiday, why are we broadcasting, you ask? Well, normally we wouldn't, but faithful viewer Nathaniel Madura pointed out that Slashdot just referenced a BBC World report on that G5 supercomputer down at Virginia Tech, and we're just a little giddy about the existence of a Mac-based cluster than can chew through 17.6 trillion floating point operations per second. Yes, the thing is up and running (at least enough to run performance testing), and reportedly it pumps out 17.6 teraflops of raw perforated aluminum muscle while sucking down enough juice to power 3,000 average homes. Wow, is it getting warm in here, or is it just us? (It's just us-- the G5s are cooled by means of 1,500 gallons of chilled water pumped through every minute. Ooooo, frosty.)
Kudos to the Virginia Tech team who pulled this off, because frankly, this is the sort of technological triumph we'd normally only expect to come out of, say, Columbus, Ohio. Now, what's interesting about that 17.6 teraflop figure is that if you scope out the last compiled list of the world's top 500 supercomputers (from last June), you'll notice that, if 17.6 teraflops is Virginia Tech's "theoretical peak performance" score, it'll probably slot in nicely at number three. (Scores are ranked by "Maximal LINPACK performance achieved," so it's just guesswork so far.) The top-ranked Intel-based cluster is currently ranked at number three, with 2,304 2.4 GHz Xeons and a theoretical peak performance of 11 teraflops. Gee, more processors, a higher clock speed per processor, and 63% of the performance. Now that's efficiency, baby!
We'll have to wait until the next top 500 list comes out in November to see if "Big Mac" (as the VA Techies apparently call it) really takes third place, or if the real-life LINPACK scores push it down lower-- but we figure a top five placement is a safe bet. One of the world's bestest supercomputers made of Macs and running Mac OS X? Why, it's a Columbus Day miracle!
4x The Bang, 1/7 The Buck (10/13/03) Meanwhile, we know that the G5 supercomputer is delivering more pluck per processor than any other supercomputer out there, but what about bang for the b
Don't anthropomorphize computers, they don't like it.
Indeed. However, a lot of it comes from your code design and your network layout. If your model is written well enough, you'll be maxing out CPU just before you max out network bandwidth. Else, the whole idea behind having a massive number of CPUs goes down the drain with dropped packets.
tasks(723) drafts(105) languages(484) examples(29106)
Well, VT are clearly getting a very good deal on their hardware. $5m for 1100 nodes works out at $4,500 a node.
Speccing up a dual G5 at the Apple store comes out at over $5,000. They also need to pay for the power and cooling hardware to run the thing.
Looks like they are getting a very good price from all their suppliers/contractors...
The retail price for the processing hardware for the UT cluster is very similar, a dual PowerEdge 2650 with 4Gb of RAM is also about $5,000. If they had taken the workstation route favoured by VT (by using Dell PWS 450 boxes) it wouldn't have saved them much as they come in at $4,600 at a similar spec.
The article says "The cost of the five-year project is about $38 million" and "The university plans to add at least 200 servers to the cluster within a year", so it isn't costing them $38m for the 300 node cluster they currently have.
Damn, just found the original press release showing that the Dells are 3.06 GHz boxes. That pushes the price per node up to over $6,300.
So I hit Dell's website and at educational pricing the servers they bought run around $4k apiece. Which means that this solution should be very price/performance competitive with the VT cluster.
I hit the UT page and found that the $38M number came from a press release about their investment in quite a bit of stuff, including the "Institute for Computational Engineering and Sciences (ICES), a new center for interdisciplinary research and graduate study in the computational sciences." I.e. at least one new building.
jim frost
jimf@frostbytes.com
the cost in that journal article seems much much too high. poked around and found this article at infoworld: http://www.infoworld.com/article/03/10/03/HNdellcl uster_1.html
they quote a dell spokeswoman saying that a configuration like that costs about 3 million with installation. it also states that UT gets an educational discount, but doesnt say how much they got off the $3million.
if the 38 million were correct, theyd be spending on the order of 120,000 per machine....a 2650 with highest processors and max ram only comes out to $13,500 on dells site...yeesh
Ah they run Linux and Apple, oh damn its BSD isn't it
It is well-known that *BSD squeezes a more performance out of the same piece of hardware than Linux can, plus the Apple uses a RISC processor which is a lot more efficient than the Intel CISC processors, so you get a double-whammy happening here in Apple's favor.
Dell is Austin's biggest employer. University's like to give back to their town and area, endearing the locals. Also, by pumping more money into the area it becomes a better place and more people would be willing to come there. I'd be willing to wager that most, if not all, of the work was done by the local Dell plant. This isn't a coincidence guys, this was more than just a business deal, it was giving back to the community that supports the school.
Except that the nodes are not 5 years old. The PowerEdge 1750s are new models as of this summer. The 2650s are certainly not more than 1-2 years old, either.
--
"Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
VT got the standard Apple 10% educational discount, according to early interviews by the VT program director. Apple did, however give them delivery preference, with the initial large shipments of G5's going directly to VT. This pushed back the expected delivery dates for other edu shipments to schools and students by approximately 3 weeks.
Either Dell seriously ripped off these guys, or somebody is lying about how much the cluster costs.
Let's just pretend that they used all Dell 2650 PowerEdge servers, since they're the more expensive model. Let's also assume they COMPLETELY maxed out the possible configuration on each machine, which is highly unlikely considering that means stuff like 730 GB of storage for each machine... but just to give the benefit of the doubt to whomever priced this stuff out, let's pretend they did.
If you go to Dell.com, and MAX out the compatible configuration for a PowerEdge 2650, it's tough to get the machine past $20,000.
For instance, the following beast costs $18,190:
Dual 3.06Ghz Pentium 4's
12GB of RAM
5 146GB Ultra 320 SCSI HD's.
PERC3-DI RAID 5 HD Controller
2 Gigabit Intel Ethernet Cards
Lots of other stuff... every option decked out, dual power supplies, etc. 3 year "gold" support. Etc.
Considering a $38M cluster consisting of 300 machines means each machine costs an average of about $127,000... where did the extra $108,810 a machine come from?
At $18,190 per machine, the cluster would cost about $5.5 million. In other words, it would cost about the same as the Apple cluster. (Again, that's with lots of stuff NOBODY would put in each machine of a cluster... 12GB of ram!? 730GB of Ultra 360 SCSI storage!? Don't think so.)
Again, I remind you all; this is assuming 100% PowerEdge 2650's. Their numbers make even LESS sense if you have some PowerEdge 1750's in there. (They're a lot cheaper and less powerful.)
If you price out a reasonable config for a 2650 (as in minus the storage, and "only" 4GB of ram each), you get a machine that costs about $9,500. So a cluster that will perform pretty much identically to the pricey one above would cost $2,850,000. In other words, more than 13 times cheaper than the $38M clusters that must perform WORSE because it has the slower PowerEdge 1750's in there somewhere. In addition, the "economy" cluster, as I like to call it, is about half the price of the Apple cluster. You also have to consider that Dell would probably give a slight discount to somebody buying 300 servers.
You also gotta wonder about the performance cited for the Apple setup. This is the same company that insisted for years that not only did they have the fastest hardware (a lie), but they had super computers for sale (a distortion of the truth, aka a lie).
Me thinks there is something fishy going on here.
The article makes it plain that this is the just the beginning of a five-year project that will eventually spend $38mil, and which will end up with a lot more than 300 systems (200 to be added next year alone, for example) too. Comparing this to another project without knowing all the details of both is pointless.
I posted this elsewhere...The cluster is running OS X -- page 13 --not linux you nimrod...
;)
Besides, if it was running linux it would be liable for the 'SCO tax' what with Linux' inability to do 64 bit without SCO's 'contributions'
We apologise for the fault in this post. Those responsible have been sacked. -- Signed RICHARD M. NIXON
Hey, I'm a Mac zealot, I know. I love to make Apple look good, for any reason. But this story has too many inconsistencies.
The Slashdot blurb and the article don't even match. And As The Apple Turns is being quoted as a technically informed Mac news source?
Anyways, yes, $5 million for the G5s. Now let's add in the price of those racks they sit on. Let's add in the price of the cooling system, the network equipment, cables, power supply. Contractors. $38 million doesnt' sound so far off the mark when you think about what all that stuff.
CAn'T CompreHend SARcaSm?
Umm, when is the last time windows released a 'Service release' that massively improved the disk cacheing scheme, the graphics subsystem, the optimization level of the kernel, the network que and tcp/ip stack, and virtually every other core level service. .02 better, it must not be much?
In addition to that, theres fast user switching, expose, a new finder, new mail app,... blah blah, everythings updated
reports have shown older machines (~400 mhz G3 and that ilk) getting double the speed they had under 10.2 (does win XP run faster or slower than win98 on your 600 mhz p3?).
the day an MS service patch does 1/10th this level of upgrade is the day I will re-evaluate my opinion of that evil bohemoth.
By the way, on what did you base your statement... the fact that OSX is currently at 10.2.8 and since 10.3 is only
Just as irrigation is the lifeblood of the Southwest, lifeblood is the soup of cannibals. -- Jack Handy
If this were an OS to OS comparison between Windows and OS X, perhaps we wouldn't be getting so frothed up. But this is hardware, and dammit, PC hardware is supposed to *always* be cheaper than Mac hardware!
To summarize what others have said:
1) Dell gave UT a sweetheart deal
2) Apple gave VT a sweetheart deal
3) Nobody has dredged up any information to indicate that the $38M UT spent includes the cost of a building. As csoto pointed out:"A "Center" at UT is a special term for a particular type of organized unit, often a research unit. It does not necessarily mean this place gets its own building. In fact, at UT, space is such a premium that most "Centers" don't have their own (yeah the place is huge, but has lots of people). In fact, I'd venture to guess that NO center has its own building."
4) Hardware is only a portion of the total cost, obviously. UT and VT have set up their supercomputing projects differently. This again is obvious.
5) The really important point of all this is that VT manage to put together a very powerful supercomputing cluster using Macs at a cost that in no way can be considered more expensive than if they'd used PC hardware.
You can argue that costs would have been cheaper had they built their own, or used PCs from some source cheaper than Dell. But they still would have had to deal with labor costs in assembling the PCs, or higher maintenance costs associated with keeping all of those commodity PCs running properly.
UCLA is already using OS X to run Beowulf-style clusters. Tokyo University is replacing over 1,100 Linux PCs with OS X boxes.
Even the total cost of installing, operating, and maintaining large numbers of Macs running OS X is cheaper than either PCs running Windows or PCs running Linux, people often seem incapable of absorbing that information.
You can talk all you want about the Reality Distortion Field, but the truth is that Apple is always working against an incredibly strong bias that says Apple is always more expensive.
That's simply no longer true.
Read the EFF's Fair Use FAQ
From Infoworld Dell delivers Linux cluster deep in the heart of Texas
Dell's list price of a configuration similar to Lonestar is $1.9 million, with services and installation charges expected to bring the total cost to around $3 million, a Dell spokeswoman said.
From the Inquirer: University of Texas kyboshes MacNN's cluster story.
Cost of supercomputer only part of $38 million
By INQUIRER staff: Tuesday 14 October 2003, 17:09
THERE'S MORE THAN MEETS the eye to a story published by MacNN and reported here today about the cost of a Dell cluster versus an Apple Mac OSX cluster.
See Dell Intel cluster costs 30 times more than Apple system.
Tina Romanella de Marquez, communications and development manager at the Texas Advanced Computing Center (TACC), says that the $38 million mentioned by MacNN is for far more than just a supercomputer.
She said: "The $38M total you refer to was not for a single supercomputer. It was announced in February for a total package that included:
"The establishment of the new Institute for Computational Engineering & Sciences (ICES) at UT, including:
our new endowed faculty chairs in ICES at UT
additional funding for the research endowment and the visiting scholars endowment in ICES
he completion of construction of the ACES building (the 4th floor) for use by ICES and TACC
"and the establishment of a terascale distributed computing infrastructure at UT, hosted by TACC, including:
two supercomputers at TACC (the cluster you refer to, and the other IBM system
two massive storage systems at TACC
three leading-edge components to increase UT's networking infrastructure
increases in operations funding over five years for ICES and TACC".
She adds: "There are many more things that were needed to create ICES and establish a terascale distributed computing architecture at TACC. This point was made by TACC Director, Jay Boisseau, during the Lonestar dedication ceremony. The value of the specific computer referred to was approximately $3.0 million. And, no tuition funds were used in this process. Most of the money did not even come from UT. The package included $8M in discounts and donations from about 10 leading technology vendors, and over $15M from a generous foundation." And, she continued: "The VaTech number ONLY includes the actual computer, not the cost of the building, power, cooling, people, or anything else needed to actually operate it."
So that comparison goes out the window, then.
As it turns out, the computer cost $3 million, vs. $5.2 million for the 17.6 terraflop Mac G5 cluster at Virginia Tech.
3.7 tflops / $3M = 1.23 tflops/$1M
17.6 tflops / $5.2M = 3.38 tflops/$1M
So with the Apples you get 2.75x more computing power for the same $$.
Sounds like a an easy argument to get by the Dean's office...
Share and Enjoy!
Correct me if I am wrong. I thought that Dell only used P4 and Celeron processors, due to a deal with Intel.