Dell $38m Supercomputer [not] More Costly than VT's G5s

An anonymous reader writes "According to the Austin Business Journal, Dell's 3-teraflop, 600 server supercomputer cluster cost the University of Texas $38 million. As The Apple Turns has pointed out that this is 7 times the cost (and a quarter of the power) of Apple's cluster at Virginia Tech! " Update: 10/14 17:56 GMT by M : worm eater writes "The Register has posted a correction to the widely-reported story that a 3.7 terraflop Dell cluster cost the University of Texas $38 million. As it turns out, the computer cost $3 million, vs. $5.2 million for the 17.6 terraflop Mac G5 cluster at Virginia Tech."

Comments

Here's a copy of the apple turns web page

[eyegor]

Since the Apple site is going to get knocked flat, here's a copy of the page.

Monday, 5:57 PM: Virginia Tech's G5-based supercomputer is (sort of) running-- with 17.6 teraflops of theoretical performance. Meanwhile, Dell tries to build something (sort of) similar, but it winds up with a quarter of the power and seven times the price, and Apple (sort of) announces Xgrid, a product for "parallel and distributed high performance computing"...

Monday, 5:57 PM: Today's holiday episode is now broadcasting. Don't forget to take your shot for a free AtAT shirt (tee or turtleneck) by entering the Q4/03 Beat The Analysts contest; guess closest to Apple's final reported quarterly profit or loss, and you get the garment-- or your choice of creaky old software from the Baffling Vault of Antiquity(TM). You've only got until Wednesday at 4 PM, and in the likely event of a tie the earliest entry wins, so why wait? Enter now!

Up, Running, & Kicking Tail (10/13/03) Fun fact: believe it or not, folks, AtAT's wild success isn't confined to these here United States. No, seriously, it's true! The show actually has semi-regular viewers holed up in such far-flung corners of the world as Iceland, the Dominican Republic, and Delaware-- and for the benefit of those fans, we thought we'd explain that, here in the U.S., today we celebrate a holiday called Columbus Day. Columbus Day, for the uninitiated, is one of our most sacred occasions: a day on which we reflect on the many cultural and technical achievements of the city of Columbus, Ohio. We celebrate Greater Columbus's world-famous quilts, its shrubberies recreating Pointillist masterpieces, and (most importantly) its commitment to the preservation of really old TV sets by wondering why the bank is closed and our mail never came. A good time is had by all.

So if this is such a major holiday, why are we broadcasting, you ask? Well, normally we wouldn't, but faithful viewer Nathaniel Madura pointed out that Slashdot just referenced a BBC World report on that G5 supercomputer down at Virginia Tech, and we're just a little giddy about the existence of a Mac-based cluster than can chew through 17.6 trillion floating point operations per second. Yes, the thing is up and running (at least enough to run performance testing), and reportedly it pumps out 17.6 teraflops of raw perforated aluminum muscle while sucking down enough juice to power 3,000 average homes. Wow, is it getting warm in here, or is it just us? (It's just us-- the G5s are cooled by means of 1,500 gallons of chilled water pumped through every minute. Ooooo, frosty.)

Kudos to the Virginia Tech team who pulled this off, because frankly, this is the sort of technological triumph we'd normally only expect to come out of, say, Columbus, Ohio. Now, what's interesting about that 17.6 teraflop figure is that if you scope out the last compiled list of the world's top 500 supercomputers (from last June), you'll notice that, if 17.6 teraflops is Virginia Tech's "theoretical peak performance" score, it'll probably slot in nicely at number three. (Scores are ranked by "Maximal LINPACK performance achieved," so it's just guesswork so far.) The top-ranked Intel-based cluster is currently ranked at number three, with 2,304 2.4 GHz Xeons and a theoretical peak performance of 11 teraflops. Gee, more processors, a higher clock speed per processor, and 63% of the performance. Now that's efficiency, baby!

We'll have to wait until the next top 500 list comes out in November to see if "Big Mac" (as the VA Techies apparently call it) really takes third place, or if the real-life LINPACK scores push it down lower-- but we figure a top five placement is a safe bet. One of the world's bestest supercomputers made of Macs and running Mac OS X? Why, it's a Columbus Day miracle!

4x The Bang, 1/7 The Buck (10/13/03) Meanwhile, we know that the G5 supercomputer is delivering more pluck per processor than any other supercomputer out there, but what about bang for the b

Re:Here's a copy of the apple turns web page

[alienfluid]

are there any limitations to the G5 architecture with regards to the scientific software already out there? i mean, i don't recall such a big cluster of 64 bit IBM chips before and so it might take some time to develop in house software that can utilize the power of such a system.

Re:Here's a copy of the apple turns web page

[IWannaBeAnAC]

Umm, can you provide some links to back this up? NUMA machines certainly offer this capability, but they are relatively rare and this cluster is certainly not NUMA. I have used several high-performance scientific clusters and I have never yet encountered one that is configured in this way. Surely it would require some substantial hardware support?

To the parent, the focus in high performance computing nowdays is on "portable perforamnce"; for the most part, libraries are written so that the performance critical components are either very small, or self-optimizing (or both). For most applications, the most you might need to do is optimize some of the kernels (eg BLAS) for the architecure.

Re:Here's a copy of the apple turns web page

[sputnikid]

just not interesting enough to be slashdotted...

apple vs intel
sco vs linux
ms vs everyone

boring

Re:Here's a copy of the apple turns web page

[L.J. Hanson]

No, it's not running linux it's running OS X.

Re:Here's a copy of the apple turns web page

[gsdali]

This is yet another article with the misconception that the VATech cluster is running OSX, it's not, its running a bare bones custom linux, I believe. It would be foolish to do otherwise. There's load is in OS X which is just unnecessary bloat for this kind of system. It may be administered from OSX and I'm sure xGrid will be, but even with xGrid Apple will more than likely strip the OS in the cluster nodes back to the Open BSD skeleton.

It's a great piece of news though although the figure may not be comperable. Could the 38 million be for a long term installation and maintenance contract. It still seems a lot even when taking into account the fact that the VATech cluster has been largely installed buy volunteer students (well they did the heavy lifting at least). The Dell still looks to be very pricey nonetheless.

I look forward to the performance scores with anticipation.

Re:Here's a copy of the apple turns web page

[Anonymous Coward]

Tina Romanella de Marquez, communications and development manager at the Texas Advanced Computing Center (TACC), says that the $38 million mentioned by MacNN is for far more than just a supercomputer.

She said: "The $38M total you refer to was NOT for a single supercomputer. It was announced in February for a total package that included:

"The establishment of the new Institute for Computational Engineering & Sciences (ICES) at UT, including:

four new endowed faculty chairs in ICES at UT
additional funding for the research endowment and the visiting scholars endowment in ICES
the completion of construction of the ACES building (the 4th floor) for use by ICES and TACC

"and the establishment of a terascale distributed computing infrastructure at UT, hosted by TACC, including:

two supercomputers at TACC (the cluster you refer to, and the other IBM system
two massive storage systems at TACC
three leading-edge components to increase UT's networking infrastructure
increases in operations funding over five years for ICES and TACC".

She adds: "There are many more things that were needed to create ICES and establish a terascale distributed computing architecture at TACC. This point was made by TACC Director, Jay Boisseau, during the Lonestar dedication ceremony. The value of the specific computer referred to was approximately $3.0 million. And, no tuition funds were used in this process. Most of the money did not even come from UT. The package included $8M in discounts and donations from about 10 leading technology vendors, and over $15M from a generous foundation." And, she continued: "The VaTech number ONLY includes the actual computer, not the cost of the building, power, cooling, people, or anything else needed to actually operate it."

Re:Here's a copy of the apple turns web page

[lpp]

They are running OSX. From the VA Tech website concerning the cluster, you can find a PDF slideshow concerning the configuration. Slide 13 indicates they will be running OSX on the nodes.

Here is the link:

VA Tech Cluster

Look for the "Slideshow Presentation" link in the bottom left.

Re:Here's a copy of the apple turns web page

[Dashing Leech]

If this is true, it should be modded up. $3.0M for the computers actually makes it cheap than the G5 cluster, unless of course their cost includes more than the computers too.

Perhaps somebody should do the research to find out what the actual computer costs were for each.

Re:Here's a copy of the apple turns web page

[GammaRay Rob]

Gsdali quote thus:
This is yet another article with the misconception that the VATech cluster is running OSX, it's not, its running a bare bones custom linux, I believe. It would be foolish to do otherwise. ...etc.

Wrong. OSX is being used, which makes this so much more Mac-a-liscious! Don't believe me?
http://don.cc.vt.edu/tcfslides.pdf
From one of the slides: 'The system will run the Mac OSX operating platform.
(bullet) FreeBSD based system, with "typical" UNIX development environment"

- GammaRay Rob

Re:Here's a copy of the apple turns web page

[meta-monkey]

I don't think that's true at all. I'm the syadmin for the CARRIER cluster at the High-performance Computing and Simulation Research Laboratory at the University of Florida. There are a few high-performance SANs (System Area Networks) out there that use true globally accessible shared memory, like SCI. The G5s in the VT cluster have Gigabit Ethernet, which has no built-in shared-memory features. Then, there are some environments like Unified Parallel C which abstracts a shared memory space in software, and can be run on top of either a shared-memory architecture or a message-passing architecture.

In general, parallel computers like SMPs have a shared memory space, but distributed computers, like a cluster, use message passing to communicate.

Re:Here's a copy of the apple turns web page

[akb]

"i don't recall such a big cluster of 64 bit IBM chips before"

Then you clearly know nothing about high performance computing. IBM's 64bit POWER cpu architecture is represented in the 4th and 5th fastest supercomputers in the world.

Re:Here's a copy of the apple turns web page

[Anonymous Coward]

600 motherboards on ebay: 30000$
2400 256 MB sdram ram chips: 240000$

The look on michael dell's face when you kick his overpriced "support" in the garbage, priceless.

There are some things money cant buy. for everything else, there is ebay.

Re:Here's a copy of the apple turns web page

[homer_ca]

If you excuse the expression, this is apple and oranges. The UT cluster uses server class hardware, Dell Poweredge 1750's and 2650's. They're 1U and 2U rack mount servers available with dual power supplies and remote management down to the BIOS level. The top of the line processor is a dual Xeon 3.06Ghz 1MB cache. Dell charges $1300 list just for the processor. The 2.4Ghz Xeon with 512K cache is a better value at $500. Even then a 2650 with dual 3.06Ghz Xeons, SCSI disk and dual power supplies is $4000. Not that far off from a dual 2Ghz G5. I'm not sure how you get 7 times the cost of the VT cluster from that. 300 servers at $4000 each is $1.2M. Maybe UT paid work study students $6/hr to set it up while VT had volunteer labor.

Re:Here's a copy of the apple turns web page

[danigiri]

The G5s in the VT cluster have Gigabit Ethernet...

Comes builtin, but they are using Infiniband to interconnect as stated in the press release.

Just a correction, as your post kind of implies they are using Gb exclusively.

Re:Here's a copy of the apple turns web page

[meta-monkey]

Whoops, so it seems. Well, my point still stands. Infiband has DMA across the interconnect, but it's still a message-passing protocol, and not shared memory.

Re:Here's a copy of the apple turns web page

[marklar1]

http://forums.macrumors.com/archive/topic/37289-1. html The informational session provided confirmation of some of the available information as well as some interesting details of the planning stage. As previously reported, the total cost of the Supercomputer Cluster comes to $5.2 million -- which includes systems, memory storage and "communication fabrics". Even at $5.2 million, the overall cost of the system is said to be "one of the cheapest systems of its kind". In determining which architecture to use, many vendors were considered beyond Apple -- including Dell, Sun, IBM and HP. The final decision was made on a pure Cost vs Performance basis -- with Apple's solution providing the best overall price. The PowerMac G5 systems will be running Mac OS X, and will also utilize a custom "fault tolerance" software system called Deja Vu. This fault tolerance system will allow the cluster to withstand "just about every failure". The cluster is expected to begin operations on October 1, 2003, with performance tweaking through Mid November. At that time, it will be open for initial applications, with a fully operational cluster expected by January 2004.

Re:Here's a copy of the apple turns web page

[ahknight]

It's not cheap. UT has 300-500 computers. VT has 1,100. If UT had 1,000 units the cost, at the same ratio, would break $6M. VT spent $5.7M.

Re:Here's a copy of the apple turns web page

[marklar1]

which version is 32-bit.... you mean thru 10.2.x? yes 10.3 starts 64-bit in way win-64 is, 32-BIT OS w/ support for 64-bit apps, operations, memory address.... let's see who offers the commercial full 64-bit monty first...

Re:Here's a copy of the apple turns web page

[gsdali]

Sorry, my mistake. I'm normally quite the Mac evangelist. However, surely the nodes will have a great deal of stuff removed from OSX, no need for the GUI and a lot of other gubbins.

Re:Here's a copy of the apple turns web page

[uunh haun]

And to match Tflops, UT would have had to spend over $14m

Re:Here's a copy of the apple turns web page

[weez75]

Putting a computer in a rack does not make it server-class. Nor does standing a computer on a desk make it a desktop. Further, even if the price is wrong for the Dell cluster, which it is, then you end up with a different calculation for value. The real cost of the cluster was $3 million or so. So, for 60% of the price of the VT cluster you get 25% of the processing capability. It doesn't take a rocket scientist to figure out that you still paid too much for that Dell cluster.

Re:Here's a copy of the apple turns web page

[SynKKnyS]

DEC, of course. ;)

Re:Here's a copy of the apple turns web page

[lederhosen]

Yes, of course they use message passing, but
implicit.

I.e. when you use virtual memory on a remote node
the pagefault will triger a routine that do a
RPC call, you still need 64 bit address space
in order to access the memory (who owns a cluster with a total memory of less than 4gig???.

Re:Here's a copy of the apple turns web page

[lederhosen]

Its not NUMA, it is fast interconnect network.
You pagefault and an RPC message will be sent.

Re:Here's a copy of the apple turns web page

[IWannaBeAnAC]

Really? How does the synchronization work? And if there was any form of local caching (which there would need to be, otherwise you would need a separate RPC message pretty much for each machine word that is accessed remotely), how is that kept in sync with the remote node?

Again, can you provide any links to back-up your claim that the machine has this capability?

Re:Here's a copy of the apple turns web page

[lederhosen]

I will try to find it...

It works (if I remember correct) that when you
try to *acces* a byte that you do not own,
you will get a pagefault. The program is linked
to a special library that catch the signal and
from the address calculates witch machine to
get the data from. Data is then transfeard in
chunks. I.e. you can not send data, only recive;
data will be send when another node wants your
data. You need barriers and other syncronisation
primitives of course.

I post again if I find a link...

Re:Here's a copy of the apple turns web page

[meta-monkey]

If that's the case, you have to have some extra software layer on top of which your application is running, or it has to be built with an environment like that in mind...see my link to UPC. When you compile a UPC project, it turns remote reads into MPI calls. There is no automatic, built in "shared memory in a cluster" software included in OS X or Linux or whatever these things are running (I keep hearing conflicting reports as to the software). It's not as easy as an RPC call, if you want any kind of decent performance, because the cache coherency issues eat you alive. That's what's so nice about SCI...it does all that stuff for you in hardware. However, the processes to remove a sharer from the linked list of sharing nodes is a nightmare of locks and handshaking.

A 64-bit address space can be abstracted on top of a 32-bit address space...you just need a special compiler or running environment that turns local access into 32-bit instructions, and remote accesses into calls across the network.

Regardless, your post said that clusters use shared memory, when, except for very rare occaisons, they do not. Most anybody's cluster (and I run a big one, and work through Globus with many others) uses explicit message passing for communications. UPC is trying to change that by mapping a shared-memory paradigm on top of a message passing implementation, but it's nowhere near there yet.

Re:Here's a copy of the apple turns web page

[stilwebm]

From the press release itself:

"As evidenced by Virginia Tech's cluster, the combination of industry standard servers, Linux, and InfiniBand creates a new standard in clustering and is changing the way computer power is deployed."

Re:Here's a copy of the apple turns web page

[Che Guevarra]

I know I'm late to this debate, but any way you look at it, she has a sweet rack.

This just in from Apple.

[Walterk]

[Nelson]Ha-ha![/Nelson]

Common sense

[suso]

But using a cheaper system that had more power would make sense.

Re:Common sense

[the_2nd_coming]

soooooooo....how is a 38 million dollar system cheaper or more powerfull when it is also 1/4 the power of a 5.2 million dollar Apple system?

See?!

[Short Circuit]

I told you Macs were cheaper!

Seriously, though: How?

Re:See?!

[PierceLabs]

And you wouldn't get those same discounts for buying large volumes of PCs? I'm intrigued by this :)

Re:See?!

[AKAImBatman]

> And you wouldn't get those same discounts for buying
> large volumes of PCs? I'm intrigued by this :)

Bundled software (i.e. Windows) costs Dell something whether they install it or not. Since Apple owns the OS, they can lower the price of the machine below the price point of OS X and make a profit on the machine but not the software.

Re:See?!

[aldoman]

Or they gave them it for a loss, just to get the PR and marketing hype...

Re:See?!

[stilwebm]

I told you Macs were cheaper!

Seriously, though: How?

The base price for a Dell Power Edge 1750 is $3,913 (with two processors). That is with a pair of 2.4GHz Xeons (533MHz FSB) with 512MB 266MHz DDR RAM.

A base Apple PowerMac G5 with Dual 2GHz processors (1GHz FSB) and 512MB 400MHz 128bit DDR is $2,999.

Re:See?!

[sql*kitten]

I told you Macs were cheaper!

Then I hope they're waterproof too, since the $38M also included the cost of a building to put them in...

Re:See?!

[Best ID Ever!]

And you forgot to mention they're also solar powered. It's one of the fastest supercomputers, but only during the day.

Re:See?!

[Zimm]

Then I hope they're waterproof too, since the $38M also included the cost of a building to put them in...

Of course macs are waterproof, the whole user interface is made of water. Don't you think that would have caused a problem before now? They don't call it Aqua for nothing. Duh!

Re:See?!

[Spruitje]

The base price for a Dell Power Edge 1750 is $3,913 (with two processors). That is with a pair of 2.4GHz Xeons (533MHz FSB) with 512MB 266MHz DDR RAM.

A base Apple PowerMac G5 with Dual 2GHz processors (1GHz FSB) and 512MB 400MHz 128bit DDR is $2,999.


And what does a comparable Power Edge 1750 with dual XEON 3,05 cost?
Yes, a lot more that the G5 dual 2 Ghz..
The Xeon 2,4 ghz is a lot slower than a G5 2 Ghz..

Re:See?!

[marklar1]

Hometown Team "Wins" Bid for SuperComputer... Beating out who? Like they had any other choice in Austin, TX....seriously....

Costs

[BWJones]

Of course Apple gave them a little bit of a deal on these systems, but on the whole, the bid process was made based upon who gave them the best deal. Apple won out in the free market making this supercomputer cluster one of the most inexpensive supercomputers in the world. Imagine it, we have ASCII blue, ASCII red and ASCII white guarded by guys with guns, and here we have a tech school that appears like they are going to enter the 500 list at potentially number 2. Cool.

Re:Costs

[Eric_Cartman_South_P]

Imagine it, we have ASCII blue, ASCII red and ASCII white guarded by guys with guns... ...Cool.

Wanna really go for cool? Guard the Apples using girls with guns!

Re:Costs

[shrinkwrap]

... here we have a tech school that appears like they are going to enter the 500 list at potentially number 2. Cool.

Cool indeed!

I was once a student, grad student and professor at Va. Tech. As a grad student in 1975 I made the bold prediction that these microprocessor "toys" just coming out would never amount to anything.

I guess they've come a long way since I've left! LOL

Re:Costs

[LoudMusic]

And I'm sure UT got one heck of a deal from Dell.

More things that I'm sure people will talk about: The Dells are 1U and 2U boxes designed for rack enclosures meaning they'll be more heat and power effecient not to mention they take up about 1/3 the physical space as the enormous PowerMac G5.

If VT had waited for the XServer with G5s in it they would have a better cluster. But I realize the desire to be in a big computing list with other big dogs.

Re:Costs

[vlad30]

Dual 2Ghz G5 RRP = $US3000 (see apple store US)
multiply x 1100 = 3.3 million dollars Still cheap even a retail note this included superdrive
assume education discount ~%10 + ~%10 big order only 600k saved My guestimate is the macs cost $US2.7 million exact configuration of the machines would be nice but guessing is fun

Re:Costs

[robbieduncan]

Apple did not offer them a special deal on the pricing. They sold them 1100 G5s at standard educational discount. What they did offer was a bump up the queue to ensure that the cluster would be running in time to make the November list.

Re:Costs

[Rhys]

Which'd be insightful if it were the hardware they are guarding, not the data on it.

Re:Costs

[sloth jr]

More things that I'm sure people will talk about: The Dells are 1U and 2U boxes designed for rack enclosures meaning they'll be more heat and power effecient not to mention they take up about 1/3 the physical space as the enormous PowerMac G5.

Rack-designed enclosures don't mean that there'll be less heat - 1655MC blade servers, for instance, effectively consume .5 U, but pump out gobs and gobs of heat, and are rated at 6A nominal/12A maximal. Placing 11kw in one 42U rack is enormously difficult to cool. Recent article in ComputerWorld talks about this issue.

Re:Costs

[Brett Johnson]

Actually, it was a blind bidding process. The vendors not only did not know the prices quoted by the other vendors, they supposedly did not know who the other bidders were. I don't know if Apple explicitly knew Dell was in the running, but I do know that Dell did not know that Apple was a strong contender.

Re:Costs

[King Babar]

Wanna really go for cool? Guard the Apples using girls with guns!

And, because this is Apple we're talking about, you make a movie about it and call it Kill Bill, Volume 3.

Sorry about that.

Re:Costs

[admiralfrijole]

its already been benchmarked....i dont know what youre talking about it not staying up....

Re:Costs

[11223]

The lack of knowledge of basic physics around here simply astounds me.

If you put a processor into a large box, it does not output any more heat than if you put it into a small box. If anything, the large box will have a more efficient means of heat exchange. Of course, this also means that the building will need the same cooling requirements too - since the same amount of heat is generated, the same amount of cooling is required.

It might be more space efficient, yes, but space mostly means money, and obviously VT did alright on that.

Re:Costs

[clifyt]

I thought it was already discussed that they were using specially designed software that would deal with the memory errors and redundancy.

Just because its not in hardware doesn't mean it can't be handled in software. Personally, I would assume it be slower to do it in software, but ECC really only helps correct for 1 flipped bit and can only note 2 flipped bits (without error correction). Maybe the software is a bit slower, but all in all more robust than this.

I don't know...I'm just an ignorant mac user...don't accept this as the truth...

Re:Costs

[gl4ss]

apple essentially gave them quite a much more than that(slapping them with 4gb of mem for one, which is nearly 900$ or so alone, per computer).

moreover, apple wouldn't be able to provide these computers consistently at that price point to companies. making the whole comparision of "they should have bought apples" quite a stupid argument in these other, seperate bids(if they're not bidding how you choose them? you don't).

the g5's ain't bad price/perf/usability, but the vt cluster really yells 'marketing' all over it. using desktop cases for one, bumping them up on the queue, slapping the memory to go as well, along with optical drives, that just isn't how you do a cluster for as cheap as possible in reality, and yeah i'm aware they can be reused as desktop machines but that's not how you do a cluster either, the parts are worth more as one combined cluster than they're individually that's the whole point of the cluster in the first place.

just for kicks, their (vt's) slides say that macosx is freebsd based..

Re:Costs

[sgage]

No, no, guard the Apples using girls with swords!

Re:Costs

[BigBir3d]

Apple won the deal based on delivery time. Dell/HP/IBM etc were all months from now. The school wanted on 2003 list of most powerfull computers.

Re:Costs

[GoRK]

Are you serious?!? I sure as hell would expect pretty steep discounts if I ordered 1000 of ANYTHING from ANYONE if an individual unit cost more than a few bucks...

Re:Costs

[DAldredge]

How does this software work?

If they are running the same code multiple times and comparing the results, shouldn't that be taken in to account when computing how fast it is?

Re:Costs

[b17bmbr]

I was once a student, grad student and professor at Va. Tech.

and now you post on /. your resume is now complete.

Re:Costs

[Joseph Vigneau]

You're naive if you believe apple didn't do any dirty cheap dealing with this one...

Offering discounts in compensation for marketing isn't "dirty cheap dealing", it's good business, particularly if the value of that marketing exceeds the discount amount.. Dell could have done the same...

Re:Costs

[shotfeel]

if an individual unit cost more than a few bucks...

Why limit yourself?

I'd expect a discount even if I was buying 1000 pencils.

Of course I'm cheap (no, not that kind of cheap!).

Re:Costs

[pmz]

Guard the Apples using girls with guns!

Mmmmmm....how do you like dem apples?

Re:Costs

[pmz]

Placing 11kw in one 42U rack is enormously difficult to cool.

I put Arsenio Hall, Justin Timerberlake, and Billy Ray Cyrus into my 11KW server room, and things weren't cool at all!

Re:Costs

[DAldredge]

Badly done is implied when you use the word anime.

Re:Costs

[Thing 1]

"Stand tall, don't think small, don't get your back against the wall. Shoot straight, can't wait, aim for the heart and fire away."

Tommy Shaw rocked back in high school... ;-)

Re:Costs

[ivan256]

$900 x 1100 = $990,000

That doesn't come close to making up the cost difference.

Let's go full retail here:

$5,120 * 1100 = $5,632,000 for the machines. (That's with the Apple Store $2,350 for the 4GB of memory. These machines would be *way* cheaper in real life.).

It's time to face facts. Apple computers aren't any more expensive than name brand x86 PCs of similar feature sets. In fact they can be *much* cheaper. This has been the case for a long time if you took resale value into account, however these days it's reflected even in the original purchase price. The "Apples are more expensive than PCs" myth should die.

Re:Costs

[skrysakj]

Well said. It's too immature to rant and rave about Apple's "Marketing" and tactics, but not put any numbers too it. Even if the G5's were bought at consumer level prices, the idea still stands, it's a far cheaper cluster than anything else out there.
Face facts folks, and mod up ivan256 for being rational and looking at the numbers to show the truth.

Re:Costs

[Multispin]

I've seen slow PCs and old HP-UX systems guarded by guys with guns.

It has nothing to do with being fast or powerful. It has everything to do with what programs are being run and what data is stored there.

Re:Costs

[jazman_777]

The lack of knowledge of basic physics around here simply astounds me.

We're dealing with American government education here. The lack of knowledge of basic [insert any topic here] around here simply astounds me.

Re:Costs

[be-fan]

Well, you also have to factor in the cost of Mellanox and the coolng setup, which can easily run into the thousands per node.

Re:Costs

[Hoser McMoose]

Price them out yourself. The G5's had 4GB of memory, Radeon 9600 Pro video card and the Apple Superdrive. Cost: $5,320.00/node

Multiply by 1100: $5.85 million

Now, my understanding is that the "standard" educational discount is 11%, which brings the cost down to $5.2 million.

So, with a total system cost of $5.3 million, VT must have got one hell of a deal on the VERY expensive infiniband hardware they are using. Actually, if they got that hardware for less than $1.1 million ($1000/node) they got a deal, let along getting it all for only $100,000.

Either way, you can hardly compare the prices of these two clusters, when UT was quoting the cost to build EVERYTHING, including renovating the building in which the cluster was to be housed.

Re:Costs

[ivan256]

Agreed, but in real live they didn't pay over $2000 for memory per unit, and they got a standard discount, so even at $1000/node for the interconnect and cooling (I would estimate cooling at ~$150/node judging from what I've seen in the pictures, BTW assuming the space was already succiciently air conditioned) it's still cheaper than the figure I listed.

The fact of the matter is, they paid 7x less than for the dell cluster, and nobody lost any money on the deal. If anybody got special treatment or a "sweetheart deal" in this saga, it was Dell.

Re:Costs

[be-fan]

Actually, they didn't. The Dell cluster was cheaper, 3m vs 5.2m. The article was corrected.

Re:Costs

[Dirtside]

I'd think that Kill Bill, Volume X would be more appropriate. :)

Re:Costs

[marklar1]

fembots w/ machine gun jumblies, yeah!

Re:Costs

[bm_luethke]

Well, to be fair, the ASCII project is guarded by guys with guns not because of the machines but what is on those machines. I doubt VT is going to be running classified nuclear weapon simulations on thier cluster. If they do then they will have the guys with guns around them also.

Re:Costs

[ivan256]

That makes more sense. If you scale up the node count the price is still higher and the performance is still lower though.

Good to know that UT didn't really get ripped off.

Re:Costs

[be-fan]

Yep. Not surprising, since the G5 is essentially a scaled-down Power4, so it should be good at this sort of thing.

Re:Costs

[Ninja Programmer]

The apple faithful modded my post as a troll of course.

Your question is a good one -- good enough, in fact, to seriously question the whole exercise. Automated hardware error detection and correction mechanism are a must because building them into software at a node-by-node level is at best application specific, and at worst reduces the cost benefit to 1 third.

The way you do this in software is to run *3* nodes doing the identical calculation. If all three agree there is no problem. If two agree then choose the agreeing results. If none agree reboot the machines and try again. Any detected failures are flagged and technicians may be automatically called to replace the failing units.

That's it. This Apple cluster is 3 times slower than advertised.

Re:Costs

[scottgfx]

QUOTE: "Apple have spent more on one advertising campange than the VT cluster cost entirely. You're naive if you believe apple didn't do any dirty cheap dealing with this one..."

You sound as if you don't approve? At least they don't have a trio of Apple Interns doing commercials. And let's use some logic here. What in the hell is Dell thinking? Imagine the liability of having three interns going out on a road trip in a company vehicle. The van could have stalled out on a railroad crossing, or they might have run into Bill Clinton!

Hmmm, let's see ...

[Daniel Dvorkin]

UT is in Austin. Dell is in Austin.

Can you say "sweetheart deal," boys and girls? I knew you could.

Re:Hmmm, let's see ...

[NineNine]

And I'm sure that VT paid full retail for their Apple boxes, too. Riiiight.

Re:Hmmm, let's see ...

[Space Coyote]

And I'm sure that VT paid full retail for their Apple boxes, too. Riiiight. Here's where you give some sort of justification for why you say that Apple has diverted a huge chunk of their first shipment of top-end G5s to a project where you think they're subsidizing the cost of the hardware. What? You don't have an explanation? Just look at the costs, there's plenty of room above the $3000/machine cost for interconnect technology and extra RAM. Nice troll, but not true in this case.

Re:Hmmm, let's see ...

[Cancel]

UT is in Austin. Dell is in Austin. Can you say "sweetheart deal," boys and girls? I knew you could.

Does this address the issue, though? Apple cost less and performed better. If UT got a deal from Dell, it looks like it wasn't much of one.

Re:Hmmm, let's see ...

[akgunkel]

Well, the cost of the project was said to be about five million dollars and consists of 1100 dual-G5s.

These retail at the Apple store for $2999USD.
Multiply that by 1100 and you get... $3,298,900.
They very well could have paid full retail, as that would still leave $1,701,100 for everything else they would need to build this thing.

Re:Hmmm, let's see ...

[knick]

Here's where you give some sort of justification for why you say that Apple has diverted a huge chunk of their first shipment of top-end G5s to a project where you think they're subsidizing the cost of the hardware.

Two words. P R

Re:Hmmm, let's see ...

[LordNimon]

Dell is Austin. AMD is in Austin. What's your point?

Re:Hmmm, let's see ...

[Brett Johnson]

VT got the standard Apple 10% educational discount, according to early interviews by the VT program director. Apple did, however give them delivery preference, with the initial large shipments of G5's going directly to VT. This pushed back the expected delivery dates for other edu shipments to schools and students by approximately 3 weeks.

Re:Hmmm, let's see ...

[Daniel Dvorkin]

"Sweetheart deal" doesn't mean the deal where a manufacturer gives a discount to a customer who buys in volume; that's standard business practice. It means the deal where a large purchase is made as a favor to a friend, rather than on the basis of rational cost-benefit analysis. Often there's some kind of direct kickback involved.

This kind of thing happens all the time in small businesses, of course, and I have a hard time arguing with it; often, it's the only way a small business can survive those first couple of years. But when you're talking about organizations the size of Dell and UT -- the latter spending taxpayers' money! -- it is, to say the least, a bit shady.

Re:Hmmm, let's see ...

[akgunkel]

Oh yeah... I forgot about that...

Still, my point was that the project could be done at the price they claim without special discounts. Although I'm sure VT appreciated saving an extra third of a million dollars.

Re:Hmmm, let's see ...

[Hawthorne01]

And I'm sure that there's no chance Dell was planning on bragging about their cluster. Nope, they were just doing it out of the goodness of hearts, no chance it would ever be used for PR.

Re:Hmmm, let's see ...

[banzai51]

..And that 1.7 million won't cover the cost of the building you need to house them.

Re:Hmmm, let's see ...

[akgunkel]

Sweet Christ...
According to this PDF from vt.edu, the $5.2M price only covers the cost of the systems, memory, storage, primary and secondary communications fabrics, and cables.
The building would obviously be a seperate expense because a building would have a life-span many times that of the computer. Duh.

Re:Hmmm, let's see ...

[Valar]

VT got a better deal from Apple, relatively, than UT got from Dell. From what I hear, Apple wouldn't give UT the same deal on longstar (because the VT cluster was Apple's "hey, look, you can cluster these" move. they don't see a need for two of those). It should be pointed out that the cluster in question hasn't been benchmarked yet, because it hasn't been fully installed yet. Anything at this point would be speculation. That said, I think VT's cluster will win in the end by a (smaller than predicted) margin.

Re:Hmmm, let's see ...

[extra88]

Apple is in Austin too and over the years Macs have had a solid presence on campus. Austin is Dell's headquarters but there are a lot of big tech firms in Austin, including IBM, so I don't think being neighbors was a significant factor.

Re:Hmmm, let's see ...

[mduell]

VT got the standard Apple 10% educational discount, according to early interviews by the VT program director.

This can't be true. Price the nodes yourself at the Apple edu store, they come to $4814 each, which is $5295400 for all of them. Thats not even including the Infiniband hardware...

Re:Hmmm, let's see ...

[Paradise Pete]

Apple is in Austin, DeLL is in Round Rock, Intel is in Austin, Motorola is in Austin, IBM is in Austin...

Not only that, but Tiffani is in Austin, too.

Wasteful? Maybe, but...

[otisaardvark]

Please don't expect to "scale" supercomputing with any sort of linearity, either in terms of "cost" or "power" (doublequotes because methinks both concepts are rather ill-defined in this situation)...

Re:Wasteful? Maybe, but...

[Short Circuit]

Indeed. However, a lot of it comes from your code design and your network layout. If your model is written well enough, you'll be maxing out CPU just before you max out network bandwidth. Else, the whole idea behind having a massive number of CPUs goes down the drain with dropped packets.

I'm happy!!!

[midifarm]

Nuff said! Go Apple! Let the sheople go with the Farmer in the Dell... The wolves are coming!!! Peace

Shocking!

[Bob9113]

So wait, let me see if I get this straight. Are they acutally implying that supercomputers built starting five years ago are actually more expensive per unit of computing power than supercomputers built today? Why, if that were true, and if the same thing applied to workstations, then you'd be able to get a 2 gigahertz machine today for what a 500 megahertz machine cost five years ago. Ludicrous I tell you. Simply ludicrous.

Re:Shocking!

[sharkey]

Are they acutally implying that supercomputers built starting five years ago are actually more expensive per unit of computing power than supercomputers built today?

Except that the nodes are not 5 years old. The PowerEdge 1750s are new models as of this summer. The 2650s are certainly not more than 1-2 years old, either.

Re:Shocking!

[Hieronymus Howard]

No, this isn't a five year old cluster against a brand new one. The dell supercomputer is brand new - just like the apple one. See http://www.bizjournals.com/austin/stories/2003/09/ 29/daily37.html

HH

Re:Shocking!

[dbirchall]

In 1998 (five years ago), a system with an Rpeak of 16 (theoretical maximum performance of 16 gigaflops) would have been near the bottom of the June 1998 TOP500 list. (I believe there were some 333MHz 24-processor Suns in that range?)

One Power Mac G5 2.0GHz Dual (introduced in June 2003) has an Rpeak of 16 (if you look solely at the multiply-add operation pair, thus giving each FPU 2 operations/cycle). And Virginia Tech went and got 1100 of them. I guess they didn't want their system to "fall off" the bottom of the list for a while.

Isn't progress wonderful? We live in a world where you can go to the mall (if your mall has an Apple Store), call up a mail-order place, or even order on-line, and get yourself an aluminum box with enough goodies inside that a mere five years ago it would have been on the Top500 list.

Paying $126k for each server -- idonfinkso

[gregor_b_dramkin]

$38e6 / 300 servers = 1.2667e+05 $/server

Methinks the price tag includes a lot more than the hardware costs.

The comparison with the VT supercomputer is almost certainly not apples to apples (so to speak)

Re:Paying $126k for each server -- idonfinkso

[EinarH]

Sure. However if one exclude support (the article contained no info about this and I'm not sure about the Apple/VT deal) it looks like UT paid far more for the interconnect and network stuff in their cluster.

The VT use Mellanox InfiniBand (InfiniScale 96-port 10Gb/sec switches and InfiniHost dual port host channel adapters ).
On the other hand UT probably paid many milions for the Cray solution. Faster(?) and with lower latency, but with a worse price/performance ratio.

Re:Paying $126k for each server -- idonfinkso

[earlytime]

Absolutely!

A direct comparison is way out there in left field. Somebody didn't bother to do the basic math. The actual article is full of grand assumptions. They may have a point that the apple cluster is a better value than the dell cluster, but you'ld have to look at several factors before you can realyl say that....

What was UT using previously to perform their calculations/sumulations, and what did it cost to operate? How much does it cost to customize/code the apps that they will be running on the cluster? How do the support costs of the old vs new solution compare? Is the new system fully supported in the 38Mil price tag, or are there additional costs (i.e. AC/power/volume/staff) ?

Re:Paying $126k for each server -- idonfinkso

[LoudMusic]

I maxed out a Dell 2650 at less than $15K, and you know an order of 600 computers is going to get your AT LEAST 25% off. Lets call it $12K per box making it a grand total of $7.2M. Where are they spending the other $31M? Floor space, power, cooling, and maintenance don't cost that much. Some tax payer in Texas is gonna be PISSED!

Re:Paying $126k for each server -- idonfinkso

[1millionmhz]

It'd be ironic if the price tag includes a lot of services for set-up, installation, and maintenance of the supercluster. I just saw a Dell TV ad that was lambasting IBM for the services that get sold alongside major server iron.

Re:Paying $126k for each server -- idonfinkso

[Brett Johnson]

Actually, the costs quoted for both systems include infrastructure costs, not just the hardware. The VT project required extensive cooling and power delivery modifications to the building housing the system. The quoted $5.2M includes the engineering costs, electricity budget, racks, cables, and switches, in addition to the cost of the 1100 dual G5 computers.

Re:Paying $126k for each server -- idonfinkso

[JamesP]

Of course

Dell = overbloat, Intel suckers..., "exclusive technology"

Only compared to Compaq in terms of "quality"

Re:Paying $126k for each server -- idonfinkso

[cdrudge]

I maxed out a Dell 2650 at less than $15K, and you know an order of 600 computers is going to get your AT LEAST 25% off.

Right. Like Dell has a 25% markup on their servers. Please. I would be very suprised if Dell's margins were >10%. Once apon a time they were 30%, but not any longer. An order of 600 computers while considerable isn't going to get you 25% off. Now the free advertising that Dell gets would help, but even considering that, it's probably not 25%.

BTW, a 2650 is easily over $15 grand. After some playing around, I got one to $24,500 minus the OS and all the extra stuff (monitors, backup software, etc). Only the hardware that fits inside the case.

Re:Paying $126k for each server -- idonfinkso

[JamesP]

Of course...

they will install it, teach you how to use it and EVEN give you a free AOL CD so you can surf the net with your Supercluster in no time!!!

Yes, I didn't read the article yet

[platypus]

but the numbers jumped at me:

38E+6/6E+2 $ is about 60000 $ per machine. Seems to be a little much for a cluster of "cheap" machines, right?
Isn't there more to it?

Ok, off, reading the article.

Re:Yes, I didn't read the article yet

[Andorion]

Something definitely doesn't add up - the Dell cluster price probably includes labor and other hardware, while the G5 price quote is ONLY FOR THE MACHINES

1,100 G5 machines, 1/7 the price of the Dell cluster = 4,545 per machine.

I cal shenanigans, wish someone would look into these biased articles before posting them.

~Berj

Re:Yes, I didn't read the article yet

[Andorion]

From the article: The cost of the five-year project is about $38 million

How is it fair to compare five-year cost (which includes all kinda of things like upkeep, all the supporting hardware, etc) to the base machine cost of the G5 cluster?

~Berj

Re:Yes, I didn't read the article yet

[axxackall]

From what I see in my experience, the hardware is just 1-10% of TCO per unit in a regular IT environment, depends if is it a workstation or a server. I am not sure how much is it in clusters.

Re:Yes, I didn't read the article yet

[platypus]

After reading the article on this apple site, it's clear that neither the author, nor his expected audience have a clue about clusters (what did I expect, they're Mac-heads afterall ;) ).

Additionally, as one can read at
http://www.tacc.utexas.edu/resources/hpcsystem s/
TACC seems to have invested quite a bit into the network, interconnect and storage technology, while by the calculations of your parent poster, there shouldn't be much financial leeway for stuff like that in the apple cluster (and no, gigabit ethernet can not be compared to Myrinet).

Oh, and everyone knows that peak performance numbers mean squat in most real situations.

Re:Yes, I didn't read the article yet

[Clock Nova]

I seriously doubt that labor is going to cost them $32 million (the difference between the two price tags.)

Re:Yes, I didn't read the article yet

[neuroklinik]

See this PDF at the Virginia Tech site.

Note the text: "Switched Network. Each node connects into the network at 20 Gbps full duplex bandwidth."

http://don.cc.vt.edu/tcfslides.pdf

Re:Yes, I didn't read the article yet

[Alan Partridge]

err...

the VT cluster doesn't use the Macs' built in GigE as its primary message passing infrastructure, but rather uses Mellanox Infiniband PCI-X cards in each machine and 24x 96 port switches, bandwidth is claimed as 20Gbps and latency as 4.5microseconds.

The GigE is used for "NFS, control, job startup and "typical"IP traffic" according to VT, and is supported by 5x Cisco 4500 series switches with 240x GigE ports per switch.

Re:Yes, I didn't read the article yet

[platypus]

Yeah, neuroklinik pointed that out, too, I didn't know that this pdf existed and speculated purely by the numbers they paid.

Just one note, they didn't say it was point-to-point performance ;) - actually, the whole architecute is "fat tree", so no point-to-point.

So we can
I guess a 96 port Mellanox switch costs around 50000$ + around 1000 $ per card (someone may look that up, but the numbers surely aren't that off). Since the 5.5E+6$ seem to include that money, we are at around 3.5E+6$ for 1100 dual G5s.
Makes around 3000$ per dual G5, good deal, I'd say.

Anyway, who really believes that the quoted costs are really comparable is nuts.

Re:Yes, I didn't read the article yet

[platypus]

Makes around 3000$ per dual G5, good deal, I'd say

Ahem, 4GB, I forgot to mention.

Re:Yes, I didn't read the article yet

[j-turkey]

It sounds like a fanboy article to me. Apple has been claiming supercomputer power for years. I've even seen claims from that Apple has "The World's first 64-bit processor!". Sure, this system sounds like it's the real deal, but it doesn't change the fact that the article in question is from a less-than-objective source (to say the least). Fanboy stuff is trash -- it doesn't matter which type of fanboy we're talking about (Apple, Linux, Wintel, Amiga, Cray, whatever). If it comes from a fanboy, it's completely subjective...and this stinks of fanboydom.

"A Columbus Day Miracle"? Come on! I think it's clear who that article came from.

--Turkey

Re:Yes, I didn't read the article yet

[w3weasel]

appleturns.com is a rumor/humor/drama site, with loads and loads of tongue in cheek humor. It is not intended to be a by-the-numbers technical rundown of the performance points of cluster A vs. cluster B.
appleturns is also heavily biased and makes no pretense of being unbiased. If you treat articles from this site with the same scepticism as those you get from cnn, you'll be fine.

Re:Yes, I didn't read the article yet

[w3weasel]

The claim Apple makes is "The Worlds first 64-bit personal computer"... as opposed to server or workstation.
Intel proponents seem to be so unwilling to actually investigate and compare in an unbiased way, and then point their fingers accusing proponents of any other platform of the exact same thing. sheeeeeesh.

Re:Yes, I didn't read the article yet

[shotfeel]

I've even seen claims from that Apple has "The World's first 64-bit processor!".

I'm assuming that's tongue-in-cheek. I don't know anyone who's made such a claim.

Sure, this system sounds like it's the real deal, but it doesn't change the fact that the article in question is from a less-than-objective source

So go look at the authoritative sources. Do they say anything different (with regard to specs, costs, etc)? They don't from what I've seen, but if you know different, please let me know.

Re:Yes, I didn't read the article yet

[j-turkey]

1. That's not the quote I read.
2. Whoever said anything about me being an Intel proponent?
sheeeeeesh.

--Turkey

Re:Yes, I didn't read the article yet

[j-turkey]

So go look at the authoritative sources. Do they say anything different (with regard to specs, costs, etc)? They don't from what I've seen, but if you know different, please let me know.

As far as the /. price/performance comparison to the Dell -- it's total fanboy BS. There's absolutely no refuting this point. First of all, if you check out the UT release, you'll notice that their cost is a total cost for a supercomputing center...not just a supercomputer/supercluster that includes a building to house it (among other things). Furthermore, I didn't see much in the way of technical details about the hardware being used in the Apple cluster (other than the CPU's in the machines and how many). How fast will the cluster be? There's no telling because the type of clustering interconnect was not disclosed. (Unless you think that cluster speed just magically scales as you add processors). They gave FLOPS estimates for how fast the system might be -- but it's meaningless without real testing and cannot even be objectively evaluated without looking into the clustering hardware being used. Absolutely no conclusion can be drawn from the data in the linked article. As far as finding an article to quote that has more authoritative sources -- I don't have it, and I don't care. The point was that the linked article was a fanboy article. I didn't say that Apple is full of shit -- and I didn't say that Apple fanboys are any more full of shit than any others. The point is that fanboys, of any type, are completely subjective. The article linked is, without any question, a fanboy article.

I've even seen claims from that Apple has "The World's first 64-bit processor!".

I'm assuming that's tongue-in-cheek. I don't know anyone who's made such a claim.

The quote in question is not tongue-in-cheek. It's an example of the various fanboy bullshit that flies around. I can't tell you the first place I saw it, but if you open up to page 20 of Volume 121C of the PCMall catalog, that exact quote appears in bold letters...not an ironclad reference, but it's the first one that comes to mind.

--Turkey

Re:Yes, I didn't read the article yet

[Trurl's Machine]

I've even seen claims from that Apple has "The World's first 64-bit processor!".

No, you didn't. You could see claims that Apple has "world's first 64-bit desktop processor". The semantic difference between desktop and workstation (and actually even a server) is vague enough that you are justified to argue that the previous 64-bit chips were not "desktop processors", as they were not primarily designed for desktop machines. It was obviously a bunch of PR, but there was nothing as blatantly false as in your version.

Re:Yes, I didn't read the article yet

[j-turkey]

Read the thread before speaking, man. There is nothing "blatantly false" in my comment(s). While I agree with your assessment of desktops Vs. servers Vs. workstations, that has nothing to do with my comment with regard to fanboy quotes (and marketing quotes in print), which I was able to cite. It may not have been from Apple, but I never even eluded to this. As far as what I saw and didn't see -- you are horribly disrespectful and I'm done with you.

the extra 30 million

[Triumph The Insult C]

went towards mice with more than one key

Re:the extra 30 million

[heir2chaos]

Yep. That way they could actually utilize the power of the machine.

Re:the extra 30 million

[11223]

Your mice has keys? And to think that all this time I've been using one with buttons!

Re:the extra 30 million

[DarkRecluse]

Well you only really need one any key. Anything else is just redundant. Wait, I already said that...

In other news...

[djupedal]

Supporters of the University of Texas' $38 million Dell cluster 'investment' today asked for their money back so they can build their own G5 cluster and hopefully regain at least a portion of their self-respect.

One UT suppporter was quoted as saying "We didn't get lousy clusters....we got lousy cluster planners!"

Re:In other news...

[Clock Nova]

We got crummy dads.

Re:In other news...

[Mononoke]

We got crummy dads.

Macintosh Jr.
The Power to Crush the Other Kids

/announcer voice

Re:In other news...

[djupedal]

ummm...cluster....project...same thing if your money was well spent. Different things if you paid too much.

Re:In other news...

[djupedal]

Now being run as a corrected story, with the Dell cluster costing $3mil vs. $5.7mi for the Apple cluster, we see a per teraflop comparison of:

• Dell: $810,000.00 per teraflop
• Apple: $295,000.00 per teraflop

It therefore cost roughly 3 times as much per teraflop for Dell as opposed to Apple.

Still looks like Dell has some splain'n to do.

So what you're saying is...

[DingoBueno]

So if I were to imagine a beowulf cluster of Dell clusters, I would really be imagining an G5 cluster...

deep...

Someone sure got bent over a table...

[alchemist68]

WHACK! BANG! Ouch! That hurts! I'll bet DULL never even kissed U. of Texas on the back before the back oriface insertion, to let them know it was coming. There's nothing quite like the feeling of getting SCREWED, especially with cheap commodity PC hardware. And to add salt, vinegar, and isopropyl alcohol into the inury (mustn't allow any infections), the DULL supercomputer has SUPER electricity usage too, and even more power hungry COOLING requirements. Well, we all live and learn. RISC-vectorized computing is the way of the future, not more MEGAHURTS.

Re:Someone sure got bent over a table...

[Captain Gingersnaps]

You sir, are hilarious.

Re:Someone sure got bent over a table...

[alchemist68]

Thank you, thank you. I really appreciate it when people REALLY understand my arguments and even laugh at them.

PC users rationale

[teamhasnoi]

The Dells have twice as many mouse buttons, so THERE!

(/me happy new mac user)

Re:PC users rationale

[sharkey]

The Dells have twice as many mouse buttons, so THERE!

My mouse has 5 times as many buttons! I scoff at your 2-button DELL mouse.

Stereotypes

[emerrill]

This goes to show that allot of the things that ppl have to say about Apple and their computers are stereotypes from the days when they were true. People (esp. in the /. crowd) bitch about apple and their cost, mainly because they don't like the OS, or because it is a closed set of standards, but in reality, the price/performance ratio of Macs are just as good as PC's are out there.

Yes, you can't build your own Mac for $300, but you get what you pay for.

Re:Stereotypes

[shepd]

>People (esp. in the /. crowd) bitch about apple and their cost, mainly because they don't like the OS, or because it is a closed set of standards, but in reality, the price/performance ratio of Macs are just as good as PC's are out there.

Hardware to hardware, then, I challenge you to bring up a Mac system (non-laptop) which I can't beat with similar or better PC hardware, of similar or better quality. Please give me a link to the Apple store for the machine you'd like me to try to beat. And no, I won't be getting the parts from all over, they'll all come from one shop.

If it's a laptop, the comparison is more difficult, as even PC laptops aren't customizable to fit the same spec as an Apple machine.

>Yes, you can't build your own Mac for $300, but you get what you pay for.

True, and a $300 PC system isn't going to of much quality without making some horrible sacrifices.

However, one can get into a PC system built out of brand name quality parts for under $500.

Re:Stereotypes

[Clock Nova]

Okay, you're on. Build me a supercomputing cluster of Wintel machines that is cheaper than and can outperform a similar cluster of G5's. Let's see what the numbers on that are like.

Re:Stereotypes

[shotfeel]

OK, spec out the equivalent of the top-of-the-line iMac. Keep in mind one important hardware specification is the systems footprint on a desk.

Hardware's cheap. Well-engineered systems are not.

Re:Stereotypes

[Feral Bueller]

Speaking of stereotypes:

However, one can get into a PC system built out of brand name quality parts for under $500.

BULLSHIT.

Build me a system that will run XP Professional using brand name quality parts for under $500 and interact with the rest of the corporate universe in a meaningful fashion. That POS you're advertising on your home page doesn't count, either.

When we're buying 300 desktops for a department upgrade, we're not going with a single-source vendor: we're going with a big fucking multi-national corporation, because we're a big fucking multi-national corporation, and big fucking multi-national corporations tend to feel more comfortable when they deal with each other.

That *thing* on your home page running with Open Office as it's productivity app doesn't count either, because it's not a useful comparison for those of us who actually has to work in a corporate environment. I'm not talking about coding either: my IDE is a text editor, a couple of terminal windows and vi in a pinch, and I can do that on any box here in the building as long as I've got my faithful copy of putty.

Believe it or not, I actually have to interact with people who do things like send me documents with embedded OLE objects, Word docs with embedded tables, Excel worksheets with Pivot Charts and 90 MB behemoth PowerPoint presentations with embedded QuickTime Movies in them (forget about our Digital Asset Management system...), so we need at least a 2 ghz CPU, 512MB of RAM, and 20GB of storage. At this point we're at your $500 mark, when you factor in the fact that I need XP Pro, and a CAL or two to get it on my domain and using Outlook.

Do I want to have to do these things? Hell no. But that's why it's called work, and I can do it a hell of a lot faster on my Mac than you can on a *nix box: all of the guys I work with who are *nix heads end up at some point in a given work week having to (a) boot into Windows (b) use a second Windows machine or (c) ask me to do it for them: I never have to leave my Mac.

Try getting some two-digit IQ workerbee over in to jump through those hoops -- not going to happen. God forbid they call Tech Support. All of which shoots your ROI out the window.

Macs are cheaper and easier to support on the desktop. Period. Anyone who says differently is either a liar or incompetant. I did it for over a decade, before moving over to Software Development (also easier on the Mac, I've noticed, but that's purely anecdotal at this point.

If you want to start talking about Micro$oft alternatives in a corporate environment, this article is a good place to get the ball rolling: Ernie Ball dumps M$ Try doing it somewhere else though: this is the Apple channel: we have a working alternative already, thank you. We've had it for over a decade. It works great, and from the looks of it, only promises to get better.

In the meantime, could you see about a version of Evolution that is usable by abovementioned workerbees?

Thanks.

Re:Stereotypes

[shepd]

>Build me a system that will run XP Professional using brand name quality parts for under $500 and interact with the rest of the corporate universe in a meaningful fashion.
>That POS you're advertising on your home page doesn't count, either.

Obviously. It's less than half the price of $500 US. This is going to be EASY.

So, we need a NIC, obviously. We need a reasonable hard drive (20 GB should do). 512 MB RAM should be plenty. A CD-RW would be nice, but you don't mention it.

If I missed something, let me know. Your BIG FUCKING MULTINATIONAL CORPORATION should have its own it department for dealing with this stuff.

>That *thing* on your home page running with Open Office as it's productivity app doesn't count either, because it's not a useful comparison for those of us who actually has to work in a corporate environment.

I'll agree. Most people like to get some office action going on. However, normally BIG FUCKING MULTINATIONAL COMPANIES deal with software licensing themselves. Even medium sized local businesses do. It only makes sense. So let's not argue about software, because I assume your company is at least as smart as where I worked before (trust me, that wouldn't be difficult at all).

Okay, so since you should be dealing with software (you're a BIG FUCKING MULTINATIONAL CORPORATION with a clue, right? Oh God I hope you aren't paying BIG FUCKING MULTINATIONAL VENDOR prices for software. If you are, give MS a call *right now*. Your boss is sure to give you a raise for saving them the cash on this one.)

So, let's spec her out, huh? Sub $500 US. Hell, I'll try to make it sub $500 CDN (that's a bit of a squeeze, but we'll see).

• Generic Case (Kinda hard to bend metal any special way the system is "better") - $30 CDN
• Enermax 300 watt Power Supply - $45.99 CDN
• ASUS A7N266 motherboard - $100 CDN
• XP 2.5+ Barton CPU - $150 CDN
• Realtek LAN card - $10 CDN
• Floppy - $10 CDN
• 20 GB WD HDD - $73.99 CDN
• 512 MB DDR - $110.99 CDN

Total: $530.97 CDN (way less than $500 US).

Any parts you think are of subpar quality? If you want, I can upgrade the case to something with some really good reviews (one of the AOpen cases) for another $60 CDN, but it's kind of pointless, since it won't affect the use or overall quality of the machine. The rest of the stuff is pretty much top of the brand names. If you don't like enermax, WD, or ASUS, you'll have a tough time convincing me Dell (or whomever you get your PCs from) doesn't use those parts.

Of course, buying in bulk from a wholesaler (again, I hope to hell you aren't buying from Dell if you are a BIG FUCKING MULTINATIONAL CORPORATION, that would be pointless, you should have your own repair departments if you are that big!) you'll get a FAR better deal. For *sure* you can shave about 25% off that price, and end up with parts so identical, they're even from the same run. That is, if you're buying in the hundreds quantities you're talking about.

>In the meantime, could you see about a version of Evolution that is usable by abovementioned workerbees?

Why? They seem to be happy running Win XP. Let them use what they're good at. Don't saddle them with a second job -- learning a new OS -- unless you are sure you can get a good ROI on it (oh, those managerial buzzwords are fun to say, aren't they?)

If you're somewhat surprised at these prices, email me. I'll give you a national contact at our wholesaler who would be happy sell you thousands of pre built systems at a time (at my other job, we'd happily order a few hundred from them every year without trouble -- we saved a lot on licensing by purchasing direct from MS, too -- our guess is about $30 or $40 grand).

>All of which shoots your ROI out the window.

Why did you assume I was going to tell you to put linux on those computers? I certainly do support any such initiatives, but you're your own best judge of what your company needs. If you want to use windows, be my guest.

Re:Stereotypes

[shepd]

>Let's see what the numbers on that are like.

You can't seriously think I'm going to test that just to satisfy your curiosity, can you?

Get reasonable and try again.

Re:Stereotypes

[Feral Bueller]

Damn you tricksy Canadians and your exchange rates!

Throw in a bit of back bacon, and you've got a deal...

You're right, I was being tremendously presumptuous that a Linux user would be trolling the Apple channel.

Re:Stereotypes

[shepd]

>Damn you tricksy Canadians and your exchange rates!

Heheheh... time for a worldwide currency!

>Throw in a bit of back bacon, and you've got a deal...

We could manage it, I'm sure. The weird thing is, a lot of shops call it "peameal bacon" here... :-)

>You're right, I was being tremendously presumptuous that a Linux user would be trolling the Apple channel.

No, you weren't really, I was a bit miffed at the whole "big corporation" thing, sorry. I was even one who boosted linux for every purpose. Having run a business for a while, I can see that I was wrong. It works tremendously well for some tasks (our Point Of Sale machine runs Quasar flawlessly, and there's no debate that linux runs our servers better than we could possibly need [BIG THANKS DJB!]) but is useless for others (Video Eiditing is just so much easier on windows. Don't know about on a Mac. I've heard it's easy on one of them too, but I can't bring myself to afford one.)

Linux, windows, MacOS, they all have their place.

Re:Stereotypes

[Clock Nova]

Hey, you made the bet, not me. Besides, I'm not really curious. I know what the outcome would be. (hint: read the headline).

Re:Stereotypes

[shepd]

>I know what the outcome would be. (hint: read the headline).

That's with Dell. They're a PC manufacturer so overpriced it's laughable. If you thought I was making a "bet" when encouraging people to give me some systems to beat, I'll double it when it comes to any Dell PC (except those crazy left over inventory below-cost zillion-dollar rebate blowouts -- those sorta scare me, quality and support wise).

It wouldn't surprise me if Cray could put something together cheaper than Dell, too. :-)

Re:Stereotypes

[Clock Nova]

I'm shutting my mouth now. The post he's talking about makes my argument better than I can.

So, how about it, Shep?

Re:Stereotypes

[shepd]

>Um - where's the monitor?

Sure. You want an eMac level quality monitor? I think I can do that for about $140 CDN (ssssh! I don't normally sell that crappy of a monitor!). Yes, I've used an eMac, the monitor is pretty fuzzy... I can't imagine the lifetime of it will be much over 5 years, and that's being optomistic.

Now, iMac level monitor, that's going to cost more. Then again, iMacs cost a *lot* more than eMacs as well...

>Where's the mouse? Where's the keyboard?

Hmmm, another $40 CDN for some nice MS quality ones.

>Don't I get some multimedia speakers - even a cheap 5$ pair?

If you're paying the price I quoted there, I'll throw them in for no cost! :-D

It's still *way* cheaper than an entry level Mac. Way, way, way cheaper. And equal, or better quality components, too.

My e-penis...

[Placid Lake]

Is bigger and cheaper than your e-penis. Seriously, if people are willing to spend whatever amount of money on a solution that does the job, who cares how it stacks up price/performance wise against another system?

Re:My e-penis...

[BoomerSooner]

Lol, so you're willing to pay for a Ferrari Modena 360 at $155,000 but actually receive the Honda Accord. Same thing, gets the job done.

In the real world you want the best performance for your budget. 38 million is a waste, as the comparison shows.

Re:My e-penis...

[HiredMan]

It's not the size... it's what you DO with it.

Yeah, that's what every guy who is worried about size says.

and It's not heavy, it's just awkward means it's heavy. Just to clear that up too.

=tkk

PS Sorry - I couldn't help myself. ;)

Re:My e-penis...

[Placid Lake]

I guess i just find it all so petty. If people don't want to research all their options then fine, they deserve to be ripped off.

Project cost vs. machine cost

[dbavirt]

The cost of the five-year project is about $38 million...

This is rather vague. $38M probably includes a lot more than the cost of the machines. I think before we go off comparing Apples and oranges, we'd better calibrate the scales first.

Price reversal

[EdMcMan]

The average Mac is probably 7 times more expensive than the average Dell, so how is this possible? I know that G5s are powerful, but are they really that much more powerful?

Re:Price reversal

[Chanc_Gorkon]

But here's the thing. My bet is that this Dell supercomputer may have more redudant items in each node then the G5 cluster does. The Poweredge 1750 has dual hot swappable power supplies. If one node looses a power supply, the other supply keeps this node up and running. G5's don't have this. You loose a power supply, the node is done. Depending on what you want your supercomputer for, you probably want a little redundancy in your setup! :) I'd take the redundancy over the power! :) But these and other server type redundacies is why the Dell cluster costs more. Each unit definitely cost more then a G5.

Re:Price reversal

[nullard]

The avereage BMW costs a lot more than the average Kia too. Dell makes some really cheap machines, so their average is low. Apple makes only moderately high to high-end machines, so their average is higher.

A better comparison would also consider what you get for the money. There is little price difference between similarly configured Dell and Apple machines -- and Apple usually comes out cheaper by a bit.

Re:Price reversal

[Alan Partridge]

"I'd take the redundancy over the power! :) But these and other server type redundacies is why the Dell cluster costs more. Each unit definitely cost more then a G5."

So the fact that you could pull an entire G5 box and replace it SEVERAL TIMES for less money is a BAD thing? Oh, and the the Vt cluster has MORE nodes, so a failed node has less impact on overall performance than in the Dell machine. Seems to me that clusters of commodity computers should be just that, and the more specialised the hardware gets, the more outrageous is the cost. Commodity machines don't have redundant ANYTHING, that's why they're cheap.

at least give credit where it's due...

[baseinfinity]

Would be more appropriate to chalk this one up as IBM beating Intel, since neither Dell or Apple are very resposible for developing the technology in the products they sell.

Re:at least give credit where it's due...

[doggkruse]

Actually, Apple designed many of the supporting chips that are important to the 1GB fsb. Very important. I would tell you which ones but i am too lazy too look it up. Try Google

Re:at least give credit where it's due...

[neuroklinik]

Apple actually participates very heavily in the development of the ICs that go into their hardware. PowerPC was, after all, originally developed by Apple, IBM and Motorola working together.

http://www.apple.com/g5processor/

Note the text: "Apple and IBM teamed up to produce the world's most advanced processor."

Apple also worked directly with IBM on the new system controller ASIC as well.

Too bad Austin BJ didn't know..

[Fallen Kell]

Too bad the Austin Business Journal didn't realize that the cluster is only 1/4 the power and 7x the price as the Apple G5 cluster just created. Now imagine if they spent all that money on an Apple G5 cluster, it would easily be the faster super computer cluster in the world. Now if only people would give Apple credit when it is due, because it is certainly due at this point in time.

Lets just hope Apple can keep up the good work and keep the G5 line updated and in pace with the x86 lines from Intel and AMD. This in my mind has always been Apple's real problem. They always release a decent product, but never seem to keep pace after they make the initial release. If they finally manage to do it this time, it should be a real boon to the IT business.

Apples and Oranges

[schnuf]

Well, VT are clearly getting a very good deal on their hardware. $5m for 1100 nodes works out at $4,500 a node.

Speccing up a dual G5 at the Apple store comes out at over $5,000. They also need to pay for the power and cooling hardware to run the thing.

Looks like they are getting a very good price from all their suppliers/contractors...

The retail price for the processing hardware for the UT cluster is very similar, a dual PowerEdge 2650 with 4Gb of RAM is also about $5,000. If they had taken the workstation route favoured by VT (by using Dell PWS 450 boxes) it wouldn't have saved them much as they come in at $4,600 at a similar spec.

The article says "The cost of the five-year project is about $38 million" and "The university plans to add at least 200 servers to the cluster within a year", so it isn't costing them $38m for the 300 node cluster they currently have.

Damn, just found the original press release showing that the Dells are 3.06 GHz boxes. That pushes the price per node up to over $6,300.

a TENTH of the power, and they'd still use Dell...

[clmensch]

Austin has its nose so far up Dell's butt that they would make a supercomputer of their PocketPC's if they were asked to. You think there was even a QUESTION of who would build their supercomputer?

And don't try to tell me that the Company-Formerly-Known-as-Compaq had a shot even though they're based in Houston...well not really anymore anyway.

Just remember

[chia_monkey]

Hey...let's just remember the standard response. "PCs are so much cheaper than Macs". "I can build a PC cheaper than your Mac". Am I missing any?

Re:Just remember

[jargoone]

Yes, you are.

"I don't need another mouse button".

"You get more for your money".

There, that's better.

Greeeeaaaat....

[BurKaZoiD]

...cost the University of Texas $38 million...

Great, just great. Now I see why the UT system gets 70% of the land grant money, and A&M only gets 30%. I'm glad Oklahoma kicked the shit outta UTA this last week. Serves 'em right.

Saw 'em off, beeeeooottch!

Dude,

[jpellino]

You're gettin' dope slapped by the provost and the comptroller and the Texas lege...

Not to mention the next site visit by NSF - that should be a hoot.

This should be on the Apple Hot News page.

Yeah but....

[raehl]

How much REAL WORK does that G5 cluster do?

Just because you have a cluster with mega TFlops doesn't mean it'll do more real work that something more expensive with less mythical peak performance.

Re:Yeah but....

[realdpk]

Heh. Let's hope that a huge competition develops between the two supercomputer departments, to always try to one-up the other. That'd be entertaining and beneficial!

$38M wasn't only for the cluster

[jimfrost]

I knew something had to be wrong with that $38M number; 600 Dell servers, good ones, should run in the ballpark of $3M, not $30M. Even with the high-performance interconnect you'd need you ought to be able to do it for under $5M.

So I hit Dell's website and at educational pricing the servers they bought run around $4k apiece. Which means that this solution should be very price/performance competitive with the VT cluster.

I hit the UT page and found that the $38M number came from a press release about their investment in quite a bit of stuff, including the "Institute for Computational Engineering and Sciences (ICES), a new center for interdisciplinary research and graduate study in the computational sciences." I.e. at least one new building.

Re:$38M wasn't only for the cluster

[greygent]

Now looks less competitve, doesn't it ?


Not really. Dell's server hardware is great, even better than Apple's workstation/server hardware. Plus those servers come with at least Gold support contracts.

Re:$38M wasn't only for the cluster

[dubiousdave]

No, but they do come with free iPod clusters.

Re:$38M wasn't only for the cluster

[cptgrudge]

Dell's server hardware is great, even better than Apple's workstation/server hardware.

I agree. We use Dell servers in our NOC where I work, and I've been nothing but pleased with it. As you said, too, they probably come with Gold Support. 4 Hour response + parts and labor is great.

Project Cost vs. Equipment Cost

[HardCase]

I realize that this is /., but even the submitter should have read the article. The $38 million price tag was the five year cost of the supercomputer project, not the cost of the computers. Project costs include much more than just the cost of the computers. That money also has to pay for infrastructure, consulting and management costs, as well as the normal day-to-day costs to keep the thing running. I suspect that the actual value of the first 300 computers themselves was well under a million dollars.

I'm sure that Dell has a similar relationship with UT as Micron Technology has with Boise State University here in Idaho, e.g., they provide free or heavily discounted equipment to the university.

-h-

Apples got nothing on this

[Anonymous Coward]

Turn That PC Into a Supercomputer By Leander Kahney
Story location: http://www.wired.com/news/technology/0,1282,60791, 00.html

02:00 AM Oct. 14, 2003 PT

A small chip-design firm will unveil a new processor Tuesday it says will transform ordinary desktop PCs and laptops into supercomputers.

At the Microprocessor Forum in San Jose, California, startup ClearSpeed Technologies will detail its CS301, a new high-performance, low-power floating-point processor.

The new chip is a parallel processor capable of performing 25 billion floating-point operations per second, or 25 gigaflops.

According to the company, the chip has the potential to bring supercomputer performance to the desktop.

An ordinary desktop PC outfitted with six PCI cards, each containing four of the chips, would perform at about 600 gigaflops (or more than half a teraflop).

At this level of performance, the PC would qualify as one of the 500 most powerful supercomputers in the world.

"That's a supercomputer on the desktop," said Simon McIntosh-Smith, ClearSpeed's director of architecture.

The souped-up PC would cost about $25,000, ClearSpeed said. By comparison, most of the supercomputers on the Top 500 list are clusters of hundreds of processors and cost millions of dollars.

The most powerful supercomputer in the world, Japan's Earth Simulator, operates at about 10 teraflops, consumes a warehouse-size space and cost $35 million.

Soon to be in prototype, the chip may be on the market within a year, ClearSpeed said. The company, which is based in Los Gatos, California, and Bristol, United Kingdom, said it will be providing prototypes to computer manufacturers by the end of the year.

When it comes to market, the chip will likely be sold to consumers as a co-processor -- an add-on PCI card that works in parallel with a PC's main processor, just like an add-on graphics card. But instead of boosting graphics performance, the chip will help compute intensive math calculations.

Similar capabilities are already built into Apple's G4 and G5 Macs, which have a floating-point co-processor called AltiVec, which handles complex, data-intensive calculations for the main processor. But whereas AltiVec is four-way parallel, ClearSpeed's chip is 64-way, the company said.

"You might class it as a big evolutionary step of AltiVec," said Mike Calise, ClearSpeed's president.

The second generation of the chip will be 128-way parallel, and then 256, and so on, Calise said.

He said server manufacturers are looking at the chip with a view to building petaflop machines -- monster supercomputers capable of a quadrillion floating-point operations a second -- or the equivalent of 25 Earth Simulators.

A petaflop machine based on the second generation of the ClearSpeed chip would take up about 20 server racks, the company said.

Calise said computer manufacturers are very excited about the new chip.

"Right now it's awe, shock and when can I get my hands on it?" Calise said.

ClearSpeed said the new chip is also very low-power, operating at about 2 watts, which would allow it to run off a laptop battery and wouldn't require special cooling.

"At 3 watts, you could put it in a PCMCIA card," said McIntosh-Smith. "With two chips on a PC Card, you can have 50 gigaflops on a laptop, running off a battery. That's equivalent to a small Linux cluster on your notebook."

McIntosh-Smith said that down the line, a PC Card with a pair of second-generation chips would perform at about 200 gigaflops, which is equivalent to a big Linux cluster and would nearly qualify the laptop for today's Top 500 supercomputers list.

Appropriately, the chip will be described at the Microprocessor Forum during a discussion of extreme processors.

Though supercomputer performance on a desktop machine may seem like overkill, Calise said there is ever-growing demand in science, government and industry, especially Hollywo

weird

[duph]

the cost in that journal article seems much much too high. poked around and found this article at infoworld: http://www.infoworld.com/article/03/10/03/HNdellcl uster_1.html

they quote a dell spokeswoman saying that a configuration like that costs about 3 million with installation. it also states that UT gets an educational discount, but doesnt say how much they got off the $3million.
if the 38 million were correct, theyd be spending on the order of 120,000 per machine....a 2650 with highest processors and max ram only comes out to $13,500 on dells site...yeesh

Re:weird

[IWannaBeAnAC]

The slashdot heading (and maybe the article - I havn't read it :-) is wildly misleading. The $38 million for the Dell machine is the total cost, including administration, support and maintenance, application development etc, for 5 years.

The '1/7 cost' for the Apple machine is for the hardware only.

Someone should wake up and smell the coffee. In these days of commodity hardware, differences like a factor of 7 (overall a factor 28 in price/performance!) is at best highly improbable.

Re:weird

[Gorbag]

Actually, the Dells are free. The $38 million is for the 1,000 graduate students to stroll up and down the aisles of machines replacing parts for five years. :-)

Re:weird

[ps_inkling]

At the Apple store, logged in as an educational user, the dual-processor G5 is discounted to $2700 from the $3000 price (standard configuration). That's a 10% discount for random students buying one machine.

As a developer, the discount on hardware is around 20%, but again for just two machines per year (one desktop and one portable).

I would imagine that an educational institution could negotiate a discount of 20% in volume on the hardware.

Dell's Diminsion 8300 is $1400 regular, and $1200 educational discount, for a 14% discount. For the dual-processor PWS 650 box, identically configured, the higher-ed box is $2938, while the small business is $3005, for a 2% discount.

Hmm, I think I see the problem. :)

Too bad you didn't read the article...

[FatSean]

...or you'd realize that you shouldn't have run your mouth...heh heh heh

let's see...

[penguin7of9]

$38 million works out to $63000 for each one of the 600 Dell nodes. But the 1750 and 2650 are your run-of-the-mill $3500-$4000 Xeon rack mounts. Where are the other $59000 per machine going? Someone is comparing apples and oranges (no pun intended).

Also, "flops" is a pretty meaningless measure of performance. If you want to use benchmarks, at least use SPEC, and on those, the Macs are not particularly outstanding in terms of price/performance.

Finally, the Macs aren't rack mounts and they require manual modification for use in clusters; that alone makes them an iffy choice. VT can get cheap student labor to do that sort of thing and may not care about bulk, air conditional, or maintenance costs, but all of those matter big time in the real world.

Sorry, but someone is dreaming if they think G5's are the epitome of cluster computing. They may be better than previous Macs, and they may have entered the realm of plausibility, but you are still going to be far better off with rackmounted 1U dual-processor Linux machines in every respect.

Re:let's see...

[penguin7of9]

"Manual modification" is the simple installation of an Infiniband networking card.

Well, that is a big deal, given that on the x86 side, many shops will sell you ready-made units (or whole racks) at little additional cost. Installing custom cards also plays havoc with repair and maintenance because it causes finger pointing among vendors.

If you think the air conditioning and maintenance costs are being ignored, you're ignorant to the entire process going on here in Blacksburg.

Desktop enclosures take up much more space than 1U rack mounts, which means they are much more expensive to install per unit than rack mounts, no matter what your per square foot cost is. Furthermore, desktops on shelves are significantly more time consuming to service, further increasing costs.

You'll also come to see that while using SPEC, the cluster is still going to destroy most clusters at ten times the price.

Oh? Care to put some side-by-side numbers behind that claim? When I calculate it, even in purchase costs along, the G5 boxes come out behind Xeon or Opteron. And the lack of rack mounting makes the costs even higher in the long run.

A 1U dual G5 Xserve may be a while since Apple seems to have trouble cooling the processors in 1U enclosures...

Cost of Operation

[toupsie]

Does anyone have a comparitive on the cost of operation for each system, i.e., cost to power the nodes, cool the nodes, cost to manage the nodes, etc? Is an Apple or a Dell "supercomputer" cheaper to run?

Stupid question: Are these really supercomputers or superclusters? I always think of a computer as one unit not a collection of units.

ASCI

[embobo]

That would be ASCI (Accelerated Strategic Computing Initiative), not ASCII. Character sets: easy. Simulating detonation of nuclear weapons: hard.

Re:Finally

[OccSub]

I wonder if the university is going to have to pay for system patches like ordinary users do, and how this is going to affect the cost for the system.

I do believe that you, sir, are a troll-and-a-half.

It is of note that Apple is getting 4x the bang, for 1/7 the buck, from which can easily be calculated 28 bangs per buck.

Linux vs BSD, CISC vs RISC

[Anonymous Coward]

Ah they run Linux and Apple, oh damn its BSD isn't it

It is well-known that *BSD squeezes a more performance out of the same piece of hardware than Linux can, plus the Apple uses a RISC processor which is a lot more efficient than the Intel CISC processors, so you get a double-whammy happening here in Apple's favor.

Re:Linux vs BSD, CISC vs RISC

[Waffle Iron]

It is well-known that *BSD squeezes a more performance out of the same piece of hardware than Linux can

IIRC, OSX uses a mach kernel, not the BSD kernel. OSX uses BSD userland apps (that way nobody will try to make them rename it GNU/OSX), but these aren't going to have a big effect on supercomputer app performance.

plus the Apple uses a RISC processor which is a lot more efficient than the Intel CISC processors

No modern processor is simple enough to be called RISC in the classic sense, and all modern CISC processors use a RISC-like core. Moreover, the Power instruction set architecture is one of the more convoluted of those CPUs that claim to be "RISC"; maybe it's not RISC at all.

These days, instruction sets and clock speed are irrelevant red herrings. Cache size, I/O bandwidth and branch prediction are the main determiners of CPU performance. From what I've seen, the integer performance of the fastest X86 chips and the fastest G5 chips (and the Itanium for that matter) benchmark to rather similar numbers. Floating-point benchmarks vary more, but that's mainly a function of how much real estate the designers put in the FPU to address the chip's target market.

Re:Linux vs BSD, CISC vs RISC

[jweatherley]

OSX uses BSD userland apps (that way nobody will try to make them rename it GNU/OSX)

The main point about BSD with OS X is that there is a BSD personality sitting on top of the Mach kernel rather than the apps. Of course having the BSD personality means it is easy to build and run BSD apps and Apple provide many. However there are some GNU apps there too and not just gcc:

bash-2.05a$ man grep | tail -n 15
BUGS
Email bug reports to bug-gnu-utils@gnu.org. Be sure to
include the word "grep" somewhere in the "Subject:" field.

Large repetition counts in the {m,n} construct may cause
grep to use lots of memory. In addition, certain other
obscure regular expressions require exponential time and
space, and may cause grep to run out of memory.

Backreferences are very slow, and may require exponential
time.

GNU Project 2000/12/06 GREP(1)
bash-2.05a$

Oh read the dumb article...

[Ceadda]

First of all, they bought 300 servers. They will be adding more in the future. Second. The 38 million is "NOT" the cost of the computers. It's their total work, hardware, research and "EVERYTHING" budget for the next 5 years. If you take out 5 years of paying salaries for setting up everything, and keeping it all running smoothly, setting up everything for all the research and projects their going to use it for... Its gonna take out a nice chunk of the budget. Also keep in mind that since they WILL be adding more compututers to it this budget might allow for a lot more processing power still to be arriving.

Re:Oh read the dumb article...

[Ohreally_factor]

So, you're saying that the TCO is that much higher for Dells? =)

Math

[midifarm]

1100 G5's @ $3000 per = $3.3 Mil

Sounds like a deal to me!

Peace

Re:Math

[IWannaBeAnAC]

But whatabout a network? That is the essential difference between a "render farm" cluster and a real parallel machine.

Do the math

[azulcactus]

I thought that seemed like a lot of money and wondered where it all went. I configured a PE1750 and PE2650 on dell.com. I could not find in this article or any other articles the exact configs of the computers so I guessed. Since the purpose of the system is raw computing power, I added 2 3.06 GHz Xeon procs to both (the most those servers will take), upped the RAM to a respectible 4GB and left the hard drives at stock (1 36gb 10k SCSI drive). Both systems priced out to just under $6k each. $6k*600 total systems (300 each according to the article) is $3.6M.

Heck, even maxing out these computers, 5 146GB SCSI drives, 12GB ram, all the fiber cards it can hold etc. puts them at about $24k each, *600=$14.4M.

I think without seeing an itimized list of costs for these two supercomputer clusters it is difficult to tell how much more one cost than another. The UT one might include things like racks, switches, cabling etc. that the Apple cluster doesn't include in main cost.

I hope the NSA is not reading this ...

[Mostly a lurker]

... can you imagine the behometh they could produce with their budget if they used a similar approach?

There's some good math for you

[haroldK]

17.9 TFLOPS/ 3 TFLOPS = 7?

Re:There's some good math for you

[haroldK]

Their math appears to be as good as my typing. that was supposed to be a 4 at the end.

i don't get the slashdot crowd

[Anonymous Coward]

besides the people that pointed out the obvious apples to orange comparison in this post...

it just seems to me that people here love to dump on dell. you repeatedly hear how they are under the thumb of MS on slashdot, but totally ignore their huge linux efforts as pointed out with these clusters

Not wanting to get picky

[goldcd]

but it's 3.67TFlops - which if you insist on taking to the nearest TF is 4. Anyway, read all about the 'LoneStar' system here http://www.tacc.utexas.edu/resources/hpcsystems/

As a second point..

[Ceadda]

Taken right from the apple news site... The Latest Apple based supercomputer plans on using 1100 dual processor G5's to reach a maximum speed of 17.6 Teraflops... Meanwhile, the Dell supercomputer is currently expected to hit 7, ... with 300 servers... The point of this being. Triple the dell, you have 900 computers, and 21Teraflops... Guess apple news is exagerating the speed bonus?

Re:As a second point..

[Rosyna]

Where do you see 7? It's 3 teraflops according to the article. So then you'd need 1760 machines to equal the performance of the 1100 G5s

This can't be!

[Cat_Byte]

This can't be! All the people supporting the free-trade agreement that is sending jobs overseas said the saved $$ would be passed on to the consumer! Seriously tho it's lame. My job was sent to India so they could cut costs and they're blowing $$ like this.

More to it

[Quixote]

There's more to such clusters than just the cost of the CPU. Things like storage, interconnect (no, you can't use the $5 Dlink cards from BestBuy), cooling, powerplant, etc. come to mind.

Of course, there's also this concept of "theoretical FLOPS", which is open to interpretation... we'll have to wait and see what the LINPACK (and other such benchmark) numbers report.

A couple of terabytes of NON ECC memory...

[zrm8y5m02]

...and you'll get silent data corruption/system panic every week. Apple's G5 system doesn't support ECC memory at this point, and it's a killer I think. Nobody with a right mind would buy such a huge cluster with tera bytes of memory without ECC or chipkill.

Re:OMFG LOL!!.. Mod parent up plz!!!.. :)

[Evil Adrian]

Not if you picture the kind of girl that would actually be guarding a supercomputer with a gun... ... beastly.

Ignoring the fact that 38m is PROJECT cost.

[Shivetya]

Whereas the previous quoted 5M cost of the BIG MAC is hardware. Unless given the breakout of the costs on the Austin machine I doubt any fair comparison can be made.

Very soon we will have that power..

[cOdEgUru]

Not that kind of purchasing power, but the super computing power rather..

According to this Wired article a small firm in CA called Clear Speed will soon revolutionize the PC space with Super computing power.

I know we will all believe it when we can find these chips on Bestbuy aisle no:4, but still currently from where I am sitting (I am sitting on a Microsoft biztalk server 2004 training session, boring as hell, being inundated by claims of innovation by a clueless trainer who programmed in Visual basic for her entire life), Clear speed is as close to Innovation that I can think of and its more of them that this industry/world need, and less of Microsoft.

Oh..lunch time.. gotta go hit on some free sandwiches. Viva la microsoft..

wrongo, raindeer

[MikeMo]

I think you are correct about Dell... they focus on cost-cutting and logistics. However, at Apple, they design everything from the case down to the custom VLSI chips on the motherboard. That's what makes them different, and that's what makes their stuff just a bit more expensive.

From the article:

[Marc2k]

"'Lonestar' consists of 300 Dell PowerEdge 1750 and PowerEdge 2650 servers running the Linux operating system from Raleigh, N.C.-based Red Hat Inc. [Nasdaq: RHAT]. Dell worked with Seattle.-based Cray Inc. [Nasdaq: CRAY] to design and install the cluster."

If Dell had to contract Cray to help them build the cluster, I'm sure that aspect wasn't cheap. I'd imagine that most of the actual costs associated with getting the Dells into an efficient cluster were more mundane things than buying the servers themselves. I can't imagine 600 Dell boxes costing 38 million, so I'm under the impression that planning, design, construction, maintenance, etc. add to the price of the servers significantly.

Re:From the article:

[NetCurl]

I can't imagine 600 Dell boxes costing 38 million, so I'm under the impression that planning, design, construction, maintenance, etc. add to the price of the servers significantly.

I think a big part of the "cost" factor is also in power consumption, not to discount how much it costs to contract Cray engineers. The G3/G4/G5 processors all use less power than an Intel/AMD/etc. alternative. When you're powering 600+ boxes, and using gobs of processing power, 24/7, 365, you have to consider how much it will cost to keep it all running for a few years. Schools consider this (businesses don't, typically) because everything needs to fit into the operating budget, dictated by the tuition cost plus the endowment.

Re:From the article:

[slick_rick]

Honestly, it sounds like the UT folks got robbed. I mean think about it, 600 dual processor Dells that retail for $5k each. Even if they are all full retail price that is still only 600 * 5000 =~ $3.00 MILL. Add in another $300k for 50 top of the line gigabit ethernet switches at full retail. Even if the data center itself cost 5 Million (which would buy one hell of a data center) that still only adds up to about 10Mill.

Seriously, think about it... 38 Mill / 600 = 63k per machine! Can anyone spell fraud?

Re:From the article:

[arivanov]

It may sound like nitpicking, but

1: These are not top of the line.

Both foundry and Extreme have lower forwarding latency and you do not give a flying f*** about management, multicast and IP for this application.

2: What f*** ethernet?

VT does not use ethernet. I would not expect a Cray designed cluster to use it either.

Re:From the article:

[d-rock]

Actually, Foundry and Extreme kick Cisco's but for layer 3 (IP) switching, too. Unless you have some really esoteric needs you're better off just saving Cisco for your WAN.

Derek

What noone seemed to notice...

[meggito]

Dell is Austin's biggest employer. University's like to give back to their town and area, endearing the locals. Also, by pumping more money into the area it becomes a better place and more people would be willing to come there. I'd be willing to wager that most, if not all, of the work was done by the local Dell plant. This isn't a coincidence guys, this was more than just a business deal, it was giving back to the community that supports the school.

Re:What noone seemed to notice...

[Blackhalo]

Dell is Austin's #1 NON-Government employer. The State of Texas (which I assume would include the univeristy itself) is the largest employer in Austin.

Re:What noone seemed to notice...

[JohnsonWax]

"University's [sic] like to give back to their town and area, endearing the locals."

Yes, that's why our supercomputer is powered by John Deeres.

Dammit!

Making it even more meaningless

[donmiguel42]

Dell cluster: 300 machines, 3TFlops = 30GFlops/machine
Apple cluster: 1100 machines, 17.6TFlops = 16GFlops/machine

As if the numbers weren't terribly useless already, here's "conclusive evidence" that Dells are better than Apples. But really, who cares? As long as something useful is being done with the power that's there, does it matter?

Re:Making it even more meaningless

[wizkid]

Of course it matters! The Apple Bigots need the Apple cluster to be faster, and the Intel Bigots need the Intel cluster to be faster.

One positive note though, Both the Apple and Intel clusters will be faster then the Microsloth cluster ;)

Re:Making it even more meaningless

[saddino]

As if the numbers weren't terribly useless already, here's "conclusive evidence" that Dells are better than Apples.

Since when is poor math "conclusive evidence"?

Hint: (3TFlops/300 machines = 10GFlops/machine)

Re:Making it even more meaningless

[Alan Partridge]

"than"

Something's rotten in Denmark

[Dracolytch]

Hey guys, think about this a bit... If it were SO much cheaper, and SO much faster, this wouldn't have been news. Somehow the books are cooked. Unfortunately, I don't know enough about the separate systems and the FULL RETAIL costs of each to do an even comparison. Remember: All CPUs don't calculate the same. Not all clusters are designed equal. The only way to really compare, and get around the PR hype is to see some benchmarks, and to find out what each cluster would have cost your average (really rich) Joe/Jane on the street. ~D

Re:Apple is more expenssive

[falcon5768]

Yes it is true, a base Apple system loaded with everything you need (with plenty of free Unix software if your inclinded to go that route) and a system reliability thats beyond anything on the wintel side (Im still using my 1999 300 mhz b/w g3 running OS X.2.8 as a constant use personal computer, not just to play games and get mail like my buddies 900 mhz p3 turned into after 2 years and one major crash that sent his computer to HP for a motherboard replacement) is a few hundred dollars more. But then I guess thats why my SE 30 is also still running and is still used at my house and will be my childs computer someday. Guess what, $300 dells are to a yugo to $800 mac are to a BMW, you get what you paid for and the more you pay the better it will last and be used in the future. I dont understan the mentality of buy a 800 top of the line system only to replace it in 2 years. Heck my buddies 700 1.4ghz HP he bought to replace his old HP is dying after a year, it cant even play the freaking .net chime without hanging up a second. The last time I had that happen on my mac was when I was running 8.6 on a LCII. but of course you will never see that, sometimes money is money to some people and just like its hard to justify a 40,000 dollar car to some people (granted I am one of those people) its the same in justifying a 2000 personal computer.

Oh, sure it's a quarter of the power...

[Esion Modnar]

...but at least it cost 7 times as much.

Re:Apple is more expenssive

[Clock Nova]

Despite your desperate attempts to dig for excuses that explain this apparent incongruity with your feelings toward Apple, the facts remain:

"Cost per Tflop: Dell - $10.3 million; Apple - $295,000. It's almost shameful."

In truth, Apple machines seem more expensive only because they don't sell $400 machines. Thus, if you want a Mac at all, you have to pay at least $1000. But that grand will still get you more computing power (and other advantages) than if you'd bought a PC.

Re:Misleading anyone

[javiercero]

Nope... it would cost you the same -assuming you are a student- as it did to VT

Even w/o the student discount, each dualie G5 sells for $3K roughly, 1100 machines in the cluster: Voila! $3.3 Million....

See math ain't that hard. They may have got a break in the memory since their machines come with 4GB per, and I believe the base dual G5 comes with 512MB.

Shouldn't be THAT expensive

[WebMasterP]

That's interesting. While the clusters we have here at the UW Atmospheric Sciences deptaretment aren't that big, they sure as hell didn't cost that much money. http://eos.atmos.washington.edu/clusters/

We have two 18 node dual processor (Athlon MP 1800+'s) clusters with dual Athlon MP 2000+'s in the head node. They came to less than $25,000 per cluster... Now, if they were to scale this up to around the same amount of nodes UT has, it would only be about 400,000 + <insert cost of extra networking equipment here>. Surely, this wouldn't cost $38 million.

So, you say this isn't a comparable unit. Fine, triple the amount of nodes... you still cruising in at around $30 million less. There has to be something there doing there that costs extra money.

Re:Finally

[techsoldaten]

The features included in 10.3 would amount to a service release in Windows.

Top500.org

[DigitalDreg]

To make it to the Top 500 Supercomputer list, you have to post more than theoretical performace. Sorry, the story is interesting but you need hard benchmark numbers from both machines to make a fair judgement.

On a supercomputer, communications overhead between the nodes is a big deal. Fast processors that can't be adequately fed don't cut it.

Re:Top500.org

[linuxbikr]

Absolutely true. Programming these beasts is the true challenge, not acquiring the hardware. Theoretical peak performance is measured with every CPU maxed at 100% with little or communication latency. Tack in message passing and hungry CPUs waiting for data from another node, plus the latency of the interconnect and your performance plummets. Plus, the application domain has a huge effect. Applications that don't or can't scale to the nodes won't make use of that untapped power.

The Top 500 benchmark is a reasonable test designed to test both theoretical and "real world" workloads assuming application loads that exhibit a high degree of parallelism. Realistically, such a cluster (including the G5 based one) can reasonably expect to achieve 25-60% of its theoretical performance on real world work depending on everything factored in.

Still, a few TFlops is impressive. Sites are shooting for 100TFlops sustained by 2008-2010. Work is on the horizon on a Petaflop machine.

What's amusing is the fact that nowadays a lot of folks have sitting under their desks the equivalent of a Cray 2 supercomputer that cost millions of dollars in the 1980s and could beat it soundly. The Cray 2 was the last of the serial supercomputers and had a peak throughput of 2GFlops clocked at 500Mhz (2ns). A modern Athlon or P4 can hammer that and contains equal or better memory and storage than what that Cray had access to. That was not even 20 years ago! Isn't technology grand.

Check out O'Reilly's High Performance Computing, 2nd Ed for an excellent primer on supercomputing and clusters. Great reading if you are interested in this stuff.

Re:Top500.org

[DigitalDreg]

And a great/informed reply too ...

Having worked on parallel machines

[Mycroft_514]

It isn't just the raw machine cycles that count. How much of the machine is used to just run the OS? On the machines I worked with (200+ nodes), over 1/2 of the computing power was being used to run the OS.

And we had a "master node". The master node was a single point of failure for the whole parallel thing. It did the workload management. By my rough calculations at the time, a 400 node cluster would have pegged that machine in the day.

I'm sure that the individual cluster machines are faster, but these are valid things to consider in reviewing one machine vs another.

Re:WARNING; Reality Distortion Field Engaged

[Clock Nova]

I believe we've already come to the conclusion that, according to the math, they paid full price for every machine.

The savings come from the fact that it won't cost as much to maintain the G5 cluster.

Cray...

[Vrallis]

I'm sure a large chunk of the mark-up in the price for this cluster was due to them involving Cray.

It's interesting that his comes up. This month's Linux Journal had an article with a guy with no cluster experience building a large cluster using...drumroll...Dell systems. Cheap. And he did it with out expensive consulting services from big names. He's now on one of those international comparison lists for top-performing clusters.

Re:Apple is more expenssive

[cens0r]

You like every other apple zealout refused to understand what you read. The cost of the dells is the total cost of the 5 year project. This includes interconnects, support, the building, cooling, electricity, and the cost of more computers in the future. The actual cost of the computers was much cheaper. Not to mention the Dells are rack mounted and actually meant to be servers. They include things like hot swapable parts.

Re:Comparision

[NickV]

You're an idiot.

Apple "DOS"?

Both machines are UNIX. (then again, the dell cluster might be windows, and then you transposed the two command prompt examples.)

The Apple cluster is also 64 bit.

[Performer Guy]

Notice that the Apple system is a 64 bit system, the Intel based system uses Xeons and is 32 bit. That is an interesting difference in a configuration this size. Cluster or not.

Re:The Apple cluster is also 64 bit.

[The Lynxpro]

*Notice that the Apple system is a 64 bit system, the Intel based system uses Xeons and is 32 bit. That is an interesting difference in a configuration this size. Cluster or not.*

True, but the operating system [OS X] is not [64 bit native]. Had the Dells been equipped with AMD Opterons, they could have a 64 bit version of Linux running on them, but then Dell would have to commit the mortal sin of running on chips not originating from Intel. But then, how many of the applications they'd run are currently 64 bit native?

Something smells like BS

[Anonymous Coward]

Either Dell seriously ripped off these guys, or somebody is lying about how much the cluster costs.

Let's just pretend that they used all Dell 2650 PowerEdge servers, since they're the more expensive model. Let's also assume they COMPLETELY maxed out the possible configuration on each machine, which is highly unlikely considering that means stuff like 730 GB of storage for each machine... but just to give the benefit of the doubt to whomever priced this stuff out, let's pretend they did.

If you go to Dell.com, and MAX out the compatible configuration for a PowerEdge 2650, it's tough to get the machine past $20,000.

For instance, the following beast costs $18,190:

Dual 3.06Ghz Pentium 4's
12GB of RAM
5 146GB Ultra 320 SCSI HD's.
PERC3-DI RAID 5 HD Controller
2 Gigabit Intel Ethernet Cards
Lots of other stuff... every option decked out, dual power supplies, etc. 3 year "gold" support. Etc.

Considering a $38M cluster consisting of 300 machines means each machine costs an average of about $127,000... where did the extra $108,810 a machine come from?

At $18,190 per machine, the cluster would cost about $5.5 million. In other words, it would cost about the same as the Apple cluster. (Again, that's with lots of stuff NOBODY would put in each machine of a cluster... 12GB of ram!? 730GB of Ultra 360 SCSI storage!? Don't think so.)

Again, I remind you all; this is assuming 100% PowerEdge 2650's. Their numbers make even LESS sense if you have some PowerEdge 1750's in there. (They're a lot cheaper and less powerful.)

If you price out a reasonable config for a 2650 (as in minus the storage, and "only" 4GB of ram each), you get a machine that costs about $9,500. So a cluster that will perform pretty much identically to the pricey one above would cost $2,850,000. In other words, more than 13 times cheaper than the $38M clusters that must perform WORSE because it has the slower PowerEdge 1750's in there somewhere. In addition, the "economy" cluster, as I like to call it, is about half the price of the Apple cluster. You also have to consider that Dell would probably give a slight discount to somebody buying 300 servers.

You also gotta wonder about the performance cited for the Apple setup. This is the same company that insisted for years that not only did they have the fastest hardware (a lie), but they had super computers for sale (a distortion of the truth, aka a lie).

Me thinks there is something fishy going on here.

cost clarification

[dubiousdave]

According to the FA, the $38M is not the cost of the computers, it is the cost of a 5 year project centered on the computers.

Check the math. $38M/1000 servers = $38,000/server. I'm pretty sure Dell's 1U servers are a little cheaper than that.

Man....

[acidrain69]

And I feel really stupid when I pay $15 more for a video game. Can you imagine being off by many millions of dollars?

Another thought

[bigjnsa500]

Sure its 7x the cost, but its more usable than an OS X cluster. There are already a plethora of programs that simply work. Whereas if you have an OS X cluster you'll have to reinvent or port existing apps/deamons to OS X. More man hours, programming hours, debuging hours, testing hours, then finally implementation. So that 7x the cost eventually evens out.

Re:Finally

[Ohreally_factor]

And now you've bitten down hard on the troll.

Re:OMFG LOL!!.. Mod parent up plz!!!.. :)

[Jellybob]

Supercomputer with a gun?

What happens when it attains self-conciousness, and kills it's operators?

Re:Cheaper eh?

[Clock Nova]

I will risk my karma to say this:

I'd rather be 1/4th the power than pay Apple taxes over the life of the cluster, AND be limited by the suckyness of apple to begin with

This just smacks of desparation. I find it a constant source of amazement (and amusement) that no matter what facts come to light that contradict the widely held rumours about Apple and their computers, there's always some jackass standing by ready to dig excuses out of his ass to counteract them. You want to pay 7x the cost for 1/4 the power, be my guest! But don't try to come up with some lame excuse for your stupidity, like vague references to the "suckyness of apple" or some mysterious "Apple tax." If anything, it's MS software that forces users to pay constant "taxes" in the way of forced upgrades and endless security patches (all of which are either optional or free for Mac users).

Just because it's 7 times more expensive, doesn't mean an x86 cluster is more expensive. It means an x86 cluster from Dell is more expensive.

That may be true, but since Dell has made a name for themselves as being the most affordable PC seller, it kind of makes your point...well...pointless, doesn't it? Unless you're suggesting that U. Texas should have built each and every machine from scratch out of bargain basement parts they ordered from all over.

I'm just realizing though that this isn't news, this is just pure flamebait.

But if the headline had been in favor of the non-Apple contendor, it would have been? No, this is definitely news, namely because apparently so few people expected this outcome.

Not comparing like with like

[AlecC]

this is 7 times the cost (and a quarter of the power) of Apple's cluster

The article describes this as a 50-year deal. That surely isn't just for the purchase of 600 servers and associated clobber. This sounds more like a 5-year build-and-manage deal. Probably some building costs, certainly some installatiopns costs, and five years wirth of (probably 24/7) sysadm. The Apple was just the hardware, IIRC

Yeah, but...

[onthefenceman]

How many FPS will it get in Quake?

Re:Yeah, but...

[mlk]

Who cares? Will it run nethack?

Retire to become desktops

[Cadre]

More things that I'm sure people will talk about: The Dells are 1U and 2U boxes... ...about 1/3 the physical space as the enormous PowerMac G5.

The trouble with 1U and 2U rackmount boxes are they are aren't suited for sitting on your desk. In another couple years, VT will be able to retire the cluster and then distribute the G5s (which are already conveniently enclosed in a desktop case) to faculty and other staff.

But I realize the desire to be in a big computing list with other big dogs.

Nope, you obviously didn't realize anything.

Ok...how much did the G5 Cluster REALLY cost

[Chanc_Gorkon]

Some folks are laughing at the "weaker" Dell cluster costing more, but here's thigns taht you need to consider:

The per node cost (just the node, not dividing the cost by 600)is more. 4000-6000 is alot more then a run of the mill G5 costs. IF they maxed it out (and that's a big if!) they would have cost around 8000 a piece. They probably did not max it out as they may be using NAS for storage. Then the racks on the Macs were literally custom jobs due to the fact that G5's are not rack mountable.

Infrastructure is EVERYTHING on a supercomputer. Miranet ain't cheap. We are not talking gigabit ethernet here. The interconnects are very specialised. Granted, it COULD be ethernet, but it depends on what your calculating.

Redundant power means alot now adays and the G5's ain't got it.

Pluse there's the sys admin salaries, the programmers....etc etc. My bet is the G5 supercomputer did not divulge those costs.

Every Mac Head is so totally convinced that the cost of running a farm of Macs is cheaper and I am not so sure of that.

Re:Ok...how much did the G5 Cluster REALLY cost

[forsetti]

But the Dell implementation would have these costs as well. Miranet, Sysadmins, porgrammers, etc..

Redundant power is "cheap" in comparison, but even so, is it strictly necessary in a fault-tolerant cluster/grid? Of course, this assumes that the cluster is fault tolerant, which it should be to a certain extent.

Comparing apples to oranges would be bad, true, but since these pieces are common, they can be removed, and leave the comparison as apples to dells.

Re:Ok...how much did the G5 Cluster REALLY cost

[dubiousdave]

The point is, though, that the number for the Dell cluster includes these costs, and it's unclear what was included in the numbers for the Mac cluster.

Re:Ok...how much did the G5 Cluster REALLY cost

[John Newman]

My bet is the G5 supercomputer did not divulge those costs.

Actually, according to the people who built it, the $5.2M includes "system itself, memory, storage, and communication fabrics."

http://www.chaosmint.com/mac/vt-supercomputer/

Infrastructure and admin costs are presumably separate, but are also probably shared with other uses.

Re:Ok...how much did the G5 Cluster REALLY cost

[danigiri]

Interconnects entail the same complexity and costs on G5's and Dell's (of course this is a bit simplified, but in the big picture, you can rule out this difference).

Redundancy, it may very well be that it is definitely cheaper to use software redundancy than to increase hardware complexity, costs and maintenance factoring in hardware power supply redundancy, ECC RAM and such. VT uses a custom built software called Deja Vu to create software redundancy. Yeah, this soft. could be factored in to the cost of the whole setup, but it might be unquantifiable, as would be costs like 'coming up with the idea and feasibility study'.

You mean that a UNIX(MacOSX) sysadmin and a Linux sysadmin would differ greatly in income or in training level? In fact, it can definitely be argued that MacOSX sysadmins would require a lower level of knowledge.

Programming: do you think that you are seen plain vanilla VC++ coding on those two beasts? Coders at that level know what they are doing big time. PhD's developing parallel code usually are not stopped by the paint in the boxes.

dani++

Re:Apple is more expenssive

[Clock Nova]

I understand everything I read. Well, except for Gadamer. Boy did I have a hard time with him.

teraflops

[yarbo]

teraflops are a poor measure of performance. Since both computers use different architectures, each flop could be of different complexity, it can take more or less flops (floating point operations) to perform the same operation depending on whether you're dealing with cisc or risc.

Not to mention ECC ram on the Dell's

[lazn]

It is my understanding that the G5 doesn't support ECC RAM, so how can you trust it's results? With that many machines, the statistics of a bit error in RAM gets quite high.

So you have fast incorrect data.

Please correct me if I am wrong.

Re:Not to mention ECC ram on the Dell's

[addaon]

It is my understanding that the Dell servers only support detection of two-bit errors, at most, so how can you trust it's results? With that many machines, the statistics of a bit error in RAM gets quite high.

So you have fast incorrect data. And, since it's a supercomputer, they're not exactly checking the results in their spare time.

Oh, wait. What do you mean they might have taken that into account?

Re:Not to mention ECC ram on the Dell's

[justins]

It is my understanding that the Dell servers only support detection of two-bit errors, at most, so how can you trust it's results?

Sorry, but you really have no idea what you're talking about here. Single error correction, double error detection (SECDED) is used by those Dell systems. It's also used by traditional Cray supercomputers. It works. http://www.cray.com/craydoc/20/manuals/S-2346-22/h tml-S-2346-22/z1018025656.html

Maybe you were being facetious. Maybe you really think three bits being flipped in the same instant is a common problem.

Re:Not to mention ECC ram on the Dell's

[Blimey85]

You are wrong. Apple's cases are just as strong as Dell's and can support quite a lot of ECC RAM piled on top before caving in. In recent lab tests it was found that a shiny new Apple case could support almost three times as much ram piled on top of it than the comparably sized Dell case.

Re:Not to mention ECC ram on the Dell's

[mbbac]

The VT team has stated that they've developed software that renders this weakness moot.

Whoa, slow down the spin!

[MhzJnky]

From the article: The cost of the five-year project is about $38 million. That leads me to think the cost includes more than just the equipment cost.

If it was just the matieral cost, that would be around $63k per server (given 600 servers costing $38 million). Some how I doubt Dell could build a single server that would cost so much.

I'm gussing there will be alot of man power required to make use of this thing. You'll probably find a lot more of your expense there. Not to mention facility costs to house, power, and cool the thing.

Yes Indeedy - Read The Story, People

[Lagged2Death]

The article makes it plain that this is the just the beginning of a five-year project that will eventually spend $38mil, and which will end up with a lot more than 300 systems (200 to be added next year alone, for example) too. Comparing this to another project without knowing all the details of both is pointless.

Re:Apple is more expenssive

[falcon5768]

I have, and I actually own one myself, actualy I own two. The problems have to do with first gen icebook models which have not a motherboard problem but a problem with their connections to the inverter. While yes I do believe apple dropped the ball on this one, the later ones dont have the problems (which is what I have) and if you hassle them a little bit (ie talk to the supervisor not the phone monkey) , even if you dont have applecare or it ran out they have been fixing the older ones for free. But I seem to remeber Dell having a simular problem with its computers recently where they recalled a bunch, in fact of 120 new dell desktops we bought 30 have been sent back for repairs to their boards after 1 year and 5 didnt work at all OUT OF THE BOX) and I have already told you about my two friends numerous HP fiascos.

Re:Apple is more expenssive

[bizard]

You like every other apple zealout refused to understand what you read. The cost of the dells is the total cost of the 5 year project.

And if that isn't the pot calling the kettle black...The Virginia Tech cluster at $5.2 million is also the cost of running the entire project for 5 years including the computers, communications fabrics, and cooling equipment. They did, however, skimp on labor costs by convincing students to lug the 19 tons of equipment into the facility for them.

beowulf this @#$#$% #$%#$%!!!

[bojan]

imagine a beowulf cluster of Dell's....

no... no.... NOOOOOOo!

For the last time, it runs OS X not Linux...

[ErnstKompressor]

The cluster runs OS X -- page 13 --

As much as you all would like it not to be so, it is, and that's that :)

$38M over five years- 300 servers is just a start

[Noren]

The article states that the $38M is to be spent over the next 5 years, and that 200 more servers will be added in the next year. Apple then assumes that all the money is spent on the initial set of 300 servers(completely ignoring the 200 specified by the article as being ordered and on the way as well as next 5 years of computer spending) and gets silly results. If you were given a $38M budget for the next five years to buy computers, would you spend it all right now? They didn't either.

Here's another article on the new cluster- apparently the initial 300 servers will be dual 3.06 Xeons connected with Myrinet.

38 Million...

[Zebra_X]

cannot be correct. or if it is, it includes the cost of the construction of the facility, all hardware associated with networking the cluster, and providing it power. Even at 20,000 a piece, generous for a 2 proc box, the cluster should cost in the neighberhood of 12 Million for the machines. Apple is cool but, boys and girls lets not get carried away.

A score of '2' for incorrect FUD???!!!

[ErnstKompressor]

I posted this elsewhere...The cluster is running OS X -- page 13 --not linux you nimrod...

Besides, if it was running linux it would be liable for the 'SCO tax' what with Linux' inability to do 64 bit without SCO's 'contributions' ;)

Re:A score of '2' for incorrect FUD???!!!

[Mod Me God]

Nimrod??? You mean anti-submarine aircraft built on the frame of the 1950s Comet?

I think Deja Vu takes cae of fault tolerance

[Phat_Tony]

I think this special "Deja Vu" program written by Srinidhi Varadarajan is supposed to take care of all the fault tolerance issues, like machines crashing in the middle of an operation, or making mistakes.

I don't understand much about how it works, but I think it's supposed to provide real-time error correction on a massive scale, which probably means it's doing a lot of redundant work.

I wonder if the quoted FlOPS figure is with or without Deja Vu running for error correction? If these machines are cheaper than the ones used in most supercomputers, but have more faulty components, then it seems unfair to quote performance measurements made without the resource overhead of this fault-tolerance software (which will presumably always be running) subtracted out first.

-Phat Tony.

Re:I think Deja Vu takes cae of fault tolerance

[Hamshrew]

I don't know much about the Deja Vu program, but Srinidhi knows his stuff. I used to work with him on cluster visualization now and then, and he was always talking about fault tolerance problems. That's actually one of the biggest concernts with clusters, and at the moment it seems to be done 'in-house' by most. And the quoted FLOPS figure is theoretical... it's likely not anywhere close to that in reality. It's difficult to get more than 60-70% of that, but I'm betting Srinidhi can tweak a bit more out with familiarity of the system. He's good at that sort of thing.

Re:Bad Math

[yomegaman]

The 7x number is also bullshit. FLOPS isn't a very good performance metric across architectures. If you go by SPECmarks you'd see that a G5 barely keeps up with a Xeon 3.06. Why don't you Apple zealots take a little time to learn something about computers? It would keep you from making such fools of yourselves.

See this Inquirer.net article

[Wavy]

University of Texas kyboshes MacNN's cluster story

Facts in the orginial story corrected

[im2xlt]

This retraction of the original story should put an end to this discussion: http://www.theinquirer.net/?article=12114

The REAL cost of the Dell system

[jeremysilver]

is about $3 million!

An article at The Inquirer reports a spokesperson from UT describing what the $38 million entailed, including a building, an academic chair, and a couple computer systems.

http://www.theinquirer.net/?article=12114

Re:a TENTH of the power, and they'd still use Dell

[Luyseyal]

Not to mention Apple having a decent sized office in town...

-l

Nancy-boys...

[ErnstKompressor]

Who are you weenies? Talk about panties in a bunch -- you can't accept the fact that Apple delivered a kick-ass cluster for pennies on the dollar compared to a Dull? And it smokes the competition? I never got my hackles hackled back in the day whenever IBM announced a new 'Deep Whatever' that could smoke my Dell P1 166(no joke!)

Sheesh, I think some folks around here need to lay off the Intel-kool-aid. AMD is not a religion, though it sure seems like there are a lot of zealots out there;)

95% of the marketplace? That makes for a lot of rabid PC-fanboys. Makes the 5%-ers look downright domesticated...

Re:Nancy-boys...

[uunh haun]

look a couple posts up and you'll see that the story is not true

Volunteers vs. Dell Services

[thecalan]

Though, I think that anyone of us can clearly see that $38m for 300-600 Dell servers is a bit much. It doesn't take a great leap to realize that this is for many more things then just 'puters.

That said, it is interesting to note that when the figure of $5.2 mil was published for the VT system, it was also stated, much like the TX system that this was the 5 year cost of the total project, not just the computer itself. Is that a real world project cost or just a Public Relations number remains to be seen.

After reading an article from the VT school newspaper, it appears at least, that the project was designed at VT, VT staff worked out the software details, and that a great amount of savings was realized at VT by the use of a small army of volunteers, the article also mentions that the whole project was completed in 3 months time which also helped cut costs. In fact, the G5's didn't even arrive at VT until September.

http://www.collegiatetimes.com/index.php?ID=2029

BBC also has a nice little warm and fuzzy.

http://www.bbcworld.com/content/template_clickon li ne.asp?pageid=666&co_pageid=3

The press release I found on Dell's website, states that "Dell services worked with Cray to design and deploy the cluster." I imagine that those services did not come cheap. Even if the services were ultimatly donated by Dell, I doubt dell would let TX not include the "retail cost" of those services in thier numbers so that Dell atleast receives the "good will" of thier charity (ala MSFT).

http://www1.us.dell.com/content/topics/global.as px /corp/pressoffice/en/2003/2003_10_03_aus_000?c=us& l=en&s=corp

My feeling is that at the end of the day, the project at VT will still come out to have a better cost / Tflop using the Apple G5's. What intersts me the most however, is the notion that the VT project was in fact designed by the VT staff and set up was aided by many volunteers much like the Open Source software community approaches building app's, while the TX project appears to have been designed by Dell and Cray and installed by Dell services, no mention of volunteers at TX thus far.

It has been clear in the the articles I have read, including the first notice asking for volunteers to help in installation, that the budget for the VT project was kept to a low threw the VT staff and volunteers pitching in to make the project happen in such a short time.

Besides, how hard would it be to get a bunch of us to volunteer to help install a Super Computer Cluster at a University? That's certainly a volunteer T-Shirt I wouldn't mind having and a day off well spent.

Hopefully, when the dust settles, VT's system will serve as another example of how a community of volunteers can come together and participate in the completion of project for the greater good of community, knowledge and education. If the economics of the VT project hold true, could this be the advent of an "open source like" movement in super computer installation around the world?

You're Absolutely Right!

[pastafazou]

Because I just went to Apple's web site, and 1100 G5 dual CPU machines at their regular price would cost me a whopping 3.3million bucks! WTF!!!! How did VT get such a sweet deal? Then, to find out how I was really getting shafted by Apple, I went to Dell.com and configured a Dell 650n workstation as follows: dual 2.4GHz Xeon CPU's, 512MB DDR266 Non-ECC RAM (Apple's is DDR400), 160GB IDE HD (Apple's is Serial ATA), 48XCDRW (Apple's is a DVD-RW), ATI Fire GL (Apple's is a Radeon 9600), RedHat Linux, and the price for 1100 of these only came to 3.2901million bucks! What a ripoff those G5's are! Apple's close to $10000 more expensive! And who cares if the G5 has Firewire 800 and the Dells only have Firewire 400? Or that the Dell only supports 4GB instead of 8GB like the G5? Who would ever need more than 4GB anyway? I can't believe Apple's still in business! It's just a matter of time before people catch on and realize these G5's are a ripoff and Apple goes out of business.

Too much Spin

[Tokerat]

Hey, I'm a Mac zealot, I know. I love to make Apple look good, for any reason. But this story has too many inconsistencies.

The Slashdot blurb and the article don't even match. And As The Apple Turns is being quoted as a technically informed Mac news source?

Anyways, yes, $5 million for the G5s. Now let's add in the price of those racks they sit on. Let's add in the price of the cooling system, the network equipment, cables, power supply. Contractors. $38 million doesnt' sound so far off the mark when you think about what all that stuff.

Re:Too much Spin

[Graff]

$5 million for the G5s. Now let's add in the price of those racks they sit on. Let's add in the price of the cooling system, the network equipment, cables, power supply.

If you take a look at the PDF of the slide presentation on the VT cluster, on page 7 there is this statement:

The total cost of the asset, including systems, memory, storage, primary and secondary communications fabrics, and cables is $5.2M.

So it's $5.2 million for the whole thing, including infrastructure and all.

Re:Too much Spin

[Tokerat]

/me places "RFTA" sticky note on monitor and dunce cap on head.

Re:OMFG LOL!!.. Mod parent up plz!!!.. :)

[djdavetrouble]

He said:
What happens when it attains self-conciousness, and kills it's operators?

I can't wait for the self-concious computer!
Sounds like a product of the Sirius Cybernetics Company,
one of those GPP (Genuine People Personality) type features.

"Dave, is my butt getting too big?"

Re:Finally

[w3weasel]

Umm, when is the last time windows released a 'Service release' that massively improved the disk cacheing scheme, the graphics subsystem, the optimization level of the kernel, the network que and tcp/ip stack, and virtually every other core level service.
In addition to that, theres fast user switching, expose, a new finder, new mail app,... blah blah, everythings updated
reports have shown older machines (~400 mhz G3 and that ilk) getting double the speed they had under 10.2 (does win XP run faster or slower than win98 on your 600 mhz p3?).
the day an MS service patch does 1/10th this level of upgrade is the day I will re-evaluate my opinion of that evil bohemoth.
By the way, on what did you base your statement... the fact that OSX is currently at 10.2.8 and since 10.3 is only .02 better, it must not be much?

"You can make redundant calculations."

[DAldredge]

Then I assume they will take this into account when the benchmark it?

Having a tough time accepting the numbers?

[Infonaut]

It's interesting to see how many people in here are getting themselves in a lather refusing to believe that it might cost less to run a supercomputer on Macs than on Dell boxes.

If this were an OS to OS comparison between Windows and OS X, perhaps we wouldn't be getting so frothed up. But this is hardware, and dammit, PC hardware is supposed to *always* be cheaper than Mac hardware!

To summarize what others have said:

1) Dell gave UT a sweetheart deal

2) Apple gave VT a sweetheart deal

3) Nobody has dredged up any information to indicate that the $38M UT spent includes the cost of a building. As csoto pointed out:"A "Center" at UT is a special term for a particular type of organized unit, often a research unit. It does not necessarily mean this place gets its own building. In fact, at UT, space is such a premium that most "Centers" don't have their own (yeah the place is huge, but has lots of people). In fact, I'd venture to guess that NO center has its own building."

4) Hardware is only a portion of the total cost, obviously. UT and VT have set up their supercomputing projects differently. This again is obvious.

5) The really important point of all this is that VT manage to put together a very powerful supercomputing cluster using Macs at a cost that in no way can be considered more expensive than if they'd used PC hardware.

You can argue that costs would have been cheaper had they built their own, or used PCs from some source cheaper than Dell. But they still would have had to deal with labor costs in assembling the PCs, or higher maintenance costs associated with keeping all of those commodity PCs running properly.

UCLA is already using OS X to run Beowulf-style clusters. Tokyo University is replacing over 1,100 Linux PCs with OS X boxes.

Even the total cost of installing, operating, and maintaining large numbers of Macs running OS X is cheaper than either PCs running Windows or PCs running Linux, people often seem incapable of absorbing that information.

You can talk all you want about the Reality Distortion Field, but the truth is that Apple is always working against an incredibly strong bias that says Apple is always more expensive.

That's simply no longer true.

Re:Having a tough time accepting the numbers?

[ramk13]

3) Nobody has dredged up any information to indicate that the $38M UT spent includes the cost of a building. As csoto pointed out:"A "Center" at UT is a special term for a particular type of organized unit, often a research unit. It does not necessarily mean this place gets its own building. In fact, at UT, space is such a premium that most "Centers" don't have their own (yeah the place is huge, but has lots of people). In fact, I'd venture to guess that NO center has its own building."

cough cough...
http://apple.slashdot.org/comments.pl?si d=82220&ci d=7210864
http://apple.slashdot.org/comments.pl?s id=82220&ci d=7209071
(didn't want to steal credit from other posters)

Bottom line to me is that you really can't draw much of a comparison from any of the information everyone is throwing about. You're better off going to Apple's and Dell's website and making estimates from educational pricing, even if that misses installation costs.

Re:Having a tough time accepting the numbers?

[mattgreen]

I think a big part of this is the tech community's old attitude toward Apple. Now that their old stereotype of Macs being more expensive isn't holding up they're caught in cognitive dissonance.

Get over it, its just a computing platform.

Re:Having a tough time accepting the numbers?

[mr.capaneus]

read this and let me know how that "cognitive dissonance" is treating you.

Re:Having a tough time accepting the numbers?

[Dwebb]

If I flip a coin, it doesn't *always* land on heads or tails. Sometimes it will land edgewise. This is the same "sometimes" that applies to Apple hardware being cheaper than PC hardware.

You make it sound like Macs and PCs are equal in setup & maintenance costs. If they were, then your comparison might hold water. Don't underestimate the advantage Apple has in setup & maintenance due to the fact that it builds the hardware AND the software together.

Re:Having a tough time accepting the numbers?

[maraist]

Don't underestimate the advantage Apple has in setup & maintenance due to the fact that it builds the hardware AND the software together.

How often do you get custom server application software (that you wrote) pre-installed on a server? Ok, rhetoric question.. How about, how often do you accept factory-defauls of hardware? A friend of mine that kept OS X on his recent mac still spent weeks finding / installing OS X varients of Linux software to enhance productivity / work with our product cycle.

Ok, so lets say you're seasoned and you've got this down to a science. Then what about patches? Upgrades? cluster configuration?

Well, since we're talkign about clusters it's highly likely that a single machine will be set up and a CD will be made to batch install/configure/boot the machine (boot in the case of disk-less machines which requires zero per machine maintanance). This batching process is trivial compared to the time it'll take to physically mount all the machines, so the difference in "maintanance" of two differing pre-built machines styles is largely irrelevant for said clusters.

So now we're back down to power requirements, physical space requirements, per-machine costs, and per-machine performance.

And for those claiming that G5's perform faster than P4's, I'm not convinced that "gcc compiled" custom cluster apps running at either 3.2GHZ with hyper-threading (gack, I mean CPU-threading), or AMD's quad proc'd opterons can't outpace the G5s. While the G5 is RISC and has fewer stages than a P4, it doesn't have the clock speed of the raw integer pipe (running at 6.4GHZ if I'm not mistaken). When it comes down to it, P4's / Xeons are faster than opterons, and in many cases faster than G5s. But you need to see the particularly compiled apps to really know which platform is going to run quicker. And my bet is that the opteron/G5 running a full GHZ slower than P4's isn't going to offer tremendous additional overall value.

Too bad the Itanium isn't catching on (and thereby getting economies of scale discounts); it's really a bargain when you get into $5k+ machines (which we're talking here).

-Michael

Re:Having a tough time accepting the numbers?

[Dwebb]

You may be right about the setup/maintenance of high volumes of server-class machines. But the comment I was responding to referred to the cost of Macs vs. PCs in general.

Re:Having a tough time accepting the numbers?

[mattgreen]

Treating me just fine, I don't use a Mac. I just think its hilarious that everyone gets worked up when there are claims that Macs are faster.

Re: Apple and cost-cutting

[King_TJ]

Well, I'm sure Apple has the ability to give some pretty deep quantity discounts on projects of this sort without "bidding below cost" just to get the sale and publicity.

Some recent reports on their 17" Powerbook, for example, showed their cost of production to be roughly 50% of what they sell them for. That's a much better profit margin than most vendors get for their portables - especially for a new model that just cost them a bundle in "tooling up" expenses (such as dies made to cast the plastic parts in,etc.).

The G5 tower is quite likely a similar story.

good to see objectivity

[Sebastopol]

i think this thread is proof that macheads are zealous, but not idiots. even the shortcomings of this article's bias aren't enough to shutter out the common nerdity of us all. cheers to that.

Re:Finally

[general_re]

It is of note that Apple is getting 4x the bang, for 1/7 the buck, from which can easily be calculated 28 bangs per buck.

I do believe that you, sir, didn't bother reading the article.

It is of note that this $38 million is the cost of the entire UT clustering project over the next five years, not simply the cost of this initial 300 node system. The article clearly states that another 200 nodes will be added within the next year. Considering that nobody here appears to know what the final UT product will actually look like, nor does it appear that anyone here knows how much the VT cluster will cost over the next five years, it is painfully obvious that this article is simply being used as a pretext for trash-talking by various fanboys, without the benefit of any real factual information to support such.

Re:Apple is more expenssive

[ErnstKompressor]

You language skill was and still is more worse in need of upgrading every two years than my mac language ability which can easily outlast some four year at a time and all the High School trough you little fanatic will NOT change that fact...Whew, that...sure...was...something...

Dual 2Ghz G5 not what you're talking about?

[ErnstKompressor]

I thought this was settled...$3000...dual 2Ghz G5...

I don't think anyone would really argue the low-price-point-computer issue, but I think we could safely cede the mid to high-ground argument once and for all...

Re:Dual 2Ghz G5 not what you're talking about?

[shepd]

>I thought this was settled...$3000...dual 2Ghz G5...

Well, it is for Apple. But one never trusts a seller to be impartial.

Others differ strongly in their opinions.

Sadly, Apple's claims are as questionable as ever, but what's astonishing is how quickly the truth has come out. Almost immediately after the keynote, while Mac fanatics worldwide continued chortling over their perceived victory, people around the Web began looking into the benchmarks Apple used to prove the G5's prowess. Predictably, things aren't as simple as Apple's followers would like to believe. More alarming, even dual processor G5 machines still don't match the processing power of a single processor Pentium 4 system, contrary to what Apple announced Monday.

And that's from a *happy* Mac owner:

And the company's hardware is of tremendous quality (I own two Macs and an iPod), with the PowerMac G5 clearly continuing this trend.

I'd hate to see what the Anti-Mac guys have to say.

There's probably no point in showing you the cost of a single P4 system. And, this little foible is probably what has sparked Apple to sell these machines for under $2,000 US. Nobody wants a machine that's slow for $3,000.

Virgina Techs quote not full cost

[peter303]

You have to include the design cost, e.g. a prof or university IT persons cost would be easily $200K-300K when overhead is folded in. Then you ahve to include housing costs for both systems: How much does it cost to feed and cool 300K watts, and so on.

Yeah, Sure...

[ErnstKompressor]

I bet Dell and UT 'probably didn't know what they were doing...'

It's not like they are a big computer company or anything, right? And I mean what does a college computing department really know about computers, right? I bet they are kicking themselves wondering "if only we had the vast knowledge of that one AC on /., then we really would be cooking with gas." I guess you just need to be a little quicker next time in order to help the PC world save face. The computing world really needs great minds like yours...right...

This PDF is the INITIAL DESIGN SPEC

[harveyswik]

Yes, slide 13 does indicate that they will be running OSX.
Did you notice that slide 11 indicates that they are using Gigabit Ethernet? They're not. They're using InfiniBand which is 10ghz. This PDF is months old, it even says so if you'd read it.

I love my mac, but stupidity should be painfull.

Re:This PDF is the INITIAL DESIGN SPEC

[Hoser McMoose]

FWIW the initial setup of the cluster is supposed to use Gigabit ethernet. Once it was setup and working, they were planning on switching over to infiniband. I don't know where VT currently is in the switchover process, but at least at some point in time, the PDF is/was correct.

Re:a TENTH of the power, and they'd still use Dell

[tartanblue]

IBM has a site in Austin that does some of the development on the eserver xSeries line (PC servers). However, since Sam P. lives in Armonk, NY, the previous poster is correct about the location of Austin's nose.

Of Course They're Waterproofed

[m_niessner]

Of course they are waterproofed. Do you think apple would leave that out. Come on, they are leaders in Industrial design.

Additionally, they have been linked to Bush's star wars program to provide an anti-theft system.

Face it, pc's suck. Mac's are better.

Plus in a photoshop test my Apple II beats a Dual Xeon 2.4Ghz by 15 seconds.

New Math?

[denjin]

Doesn't 3TFlops/300 machines
=10GFlops per machines?

Both sides missing the point

[Teahouse]

Even if the hardware costs the same, I happen to think it probably does, you are still looking at a huge performance difference.

My guess is that Dell (a Texas company) is being favored on this project to cut it's teeth in the SC world. They sought out Cray's help in putting this all together. Chances are they will get better at this as time goes by, but the performance difference is still pretty awesome.

In the SC world, performance is everything, Apple has done a good job of it at VT. Let's not get all partisan on defending Mac's (as the Apple wonks are into) or mindlessly defending Dell's offering simply because it uses Linux (as a lot of otherwise level-headed coders are doing).

Apple did a good job. Dell will get better. Accept and move on.

Sorry, wrong numbers

[deleuze]

The following just appeared on macnn.com:

Dell cluster vs. Apple cluster; redux
Tuesday, October 14, 2003 @ 12:05am

MacNN reader Dave Shroeder writes: "The University of Texas just rolled out a $38M Dell/Linux cluster that will achieve 3.7 Tflops, not even yet at full capacity. Compare that to Virginia Tech's $5.2M Apple/Mac OS X cluster that achieves 17.6 Tflops, constructed in 3 months." Several readers followed up on the pricing and cost issues involved: [updated]

"The $38M total was NOT for a single supercomputer. Please correct this information immediately. It was announced inFebruary for a total package that included:

The establishment of the new Institute for Computational Engineering & Sciences (ICES) at UT, including:
four new endowed faculty chairs in ICES at UT additional funding for the research endowment and the visiting scholars endowment in ICES
the completion of construction of the ACES building (the 4th floor) for use by ICES and TACC

and the establishment of a terascale distributed computing infrastructure at UT, hosted by TACC, including: two supercomputers at TACC (the cluster you refer to, and the other IBM system)
two massive storage systems at TACC
three leading-edge components to increase UT's networking infrastructure
increases in operations funding over five years for ICES and TACC

The original author also followed-up on the his note: "The 17.6 Tflops figure is Rpeak (theoretical max performance). LINPACK Rmax (maximum achieved performance), the measured benchmark by which rankings are judged, will be announced at a session on November 18 at Supercomputing 2003.

"It may also be worth pointing out that the $38M figure is for the 5-year life of the project at UT, while the $5.2M figure is the initial cost of the asset itself at VT, and does not include operational money. The dollar amounts aren't directly comparable.

"In fact, based on all the responses I've gotten, this story, as posted, probably isn't very accurate. It might be better to link to both of the articles, mention that the numbers are just theoretical max performance and that "real world" numbers will follow. One could imagine that Apple is bound to make a good showing in price/performance, but the price/Tflop figures are not accurate because the prices include different things."

rackmount

[oZZoZZ]

How do you guys think rackmount affects price/size of this unit?

It's no secret that rackmount components cost substantially more than tower components... my question is: does rackmount really save the space?

The G5 creates an insane amount of heat, and would require 3U and at the very least 2U... If you took a G5 and turned it on it's side, that'd be the equivilent of 3U.

They seemed to have packed these G5s in special racks (Wider than normal), to allow 3 of them stacked side by side. There also appears to be little to no room between them vertically, making them nearly as small as 3U cases, and saving the price of rails, rackmount case/psu. Also allowing easier changing of broken ones (just buy a standard one and insert), no taking off rails, making sure cases are the same, etc... saving time in the long run I think.

Personally, I'm not a big fan of rackmount, but what do y'all think?

UT Responds...

[dallask]

It seems that the comparison between the two universities is not an apples to apples comparison.

http://www.theinquirer.net/?article=12114

Re:UT Responds...

[Corydon76]

Even so, given their figure of $3.0 million for the computer, that's $810,000 per teraflop versus VT's computer at $295,000 per teraflop. The G5's are still nearly 3 times cheaper.

FLOPS...

[ErnstKompressor]

Even if one were to entertain the moronic statement that FLOPS isn't a very good performance metric(what is then?) across architectures, is FLOPS not the right performance metric to assess the power of one architecture? Are you really just denying the VA results for no good reason? Are you saying we should run SPECmarks on the VA and UT clusters? Are you really that dim? Is your existence really that threatened by the prospect that you might not be on the greener side? Is your grasp on reality really so tenuous that you must resort to calling your fellow computer-users zealots and fools? Do you actually think that the VA cluster's potential second place supercomputer status is fraudulent? Why would 5%-ers have so much sway?

Re:FLOPS...

[yomegaman]

I call you zealots because that's what you are, just look how defensive you got when I suggested that Apples aren't all that, you're practically slobbering. Look, the dual G5 is about the equivalent of a dual Xeon 3.06 in SPEC rates, and they cost about the same amount. How you can go from that to "4x the power at 1/7 the cost" without making shit up, well you tell me.

Re:Finally

[techsoldaten]

Great, everything is getting double the speed it got in 10.2...

The question that comes to mind is why did you pay for 10.2 if it wasn't ready to perform that well on the hardware you mentioned?

Apple is doing you no favors right now, and they are persuing the same path M$ did with Win 95/98/Me. Users are essentially paying to test the company's OS while they figure out how to get it to work properly. In the meantime, they are offering features and enhancements that are of marginal value at best.

The kernel optimization and network que enhancements are parts of BSD which Apple could not get working before the last release of their OS. The new finder, expose, mail app, etc. are nice looking applications, but they really don't do a lot you couldn't do before 10.3.

You declined to mention the most 'wonderful' thing Apple is claiming about the OS enhancements - a 64-bit OS(!). Mac OS X on the G5 right now is about as much a 64-bit system as the Atari Jaguar. Mac has not built a 64 bit OS, they built a 32-bit OS with portions recoded to take advantage of some 64-bit chipset features. People buying this upgrade and paying for Apple to make good on their claims.

Apple is at least a year off from living up to all their hype (which means application support is at least 2 years off), and it sucks to pay someone to make their OS work well with their system when they tout the tight integration between the two as a distinctive quality of their product. The worst part is that you will be paying a premium during this process.

I mean, yeah, the latest XP service pack is pretty much a bunch of bug fixes, but they also added USB 2.0 support, gave users new compatibility with about 200 consumer devices, and fixed numerous DHCP and networking issues. No system running XP is likely to run any faster than before the SP, but that can be considered a good thing next to Apple's program.

Re:Bad Math

[Sialagogue]

The worst kind of idiot is the one who's too busy telling you how stupid you are to see how moronic they look.

That doesn't work all that well, no real flow to it. You might want to rephrase/tighten it up a bit before you use it again.

Just a suggestion.

Re:Finally

[Trurl's Machine]

Finally, a single example of someone saving money on the purchase of an Apple!

Actually, any iBook owner is such an example.

Actually Lonestar Cost $1.9 million

[Deviate_X]

From Infoworld Dell delivers Linux cluster deep in the heart of Texas

Dell's list price of a configuration similar to Lonestar is $1.9 million, with services and installation charges expected to bring the total cost to around $3 million, a Dell spokeswoman said.

From the Inquirer: University of Texas kyboshes MacNN's cluster story.

Cost of supercomputer only part of $38 million

By INQUIRER staff: Tuesday 14 October 2003, 17:09

THERE'S MORE THAN MEETS the eye to a story published by MacNN and reported here today about the cost of a Dell cluster versus an Apple Mac OSX cluster.
See Dell Intel cluster costs 30 times more than Apple system.

Tina Romanella de Marquez, communications and development manager at the Texas Advanced Computing Center (TACC), says that the $38 million mentioned by MacNN is for far more than just a supercomputer.

She said: "The $38M total you refer to was not for a single supercomputer. It was announced in February for a total package that included:

"The establishment of the new Institute for Computational Engineering & Sciences (ICES) at UT, including:

our new endowed faculty chairs in ICES at UT
additional funding for the research endowment and the visiting scholars endowment in ICES
he completion of construction of the ACES building (the 4th floor) for use by ICES and TACC

"and the establishment of a terascale distributed computing infrastructure at UT, hosted by TACC, including:

two supercomputers at TACC (the cluster you refer to, and the other IBM system
two massive storage systems at TACC
three leading-edge components to increase UT's networking infrastructure
increases in operations funding over five years for ICES and TACC".

She adds: "There are many more things that were needed to create ICES and establish a terascale distributed computing architecture at TACC. This point was made by TACC Director, Jay Boisseau, during the Lonestar dedication ceremony. The value of the specific computer referred to was approximately $3.0 million. And, no tuition funds were used in this process. Most of the money did not even come from UT. The package included $8M in discounts and donations from about 10 leading technology vendors, and over $15M from a generous foundation." And, she continued: "The VaTech number ONLY includes the actual computer, not the cost of the building, power, cooling, people, or anything else needed to actually operate it."

So that comparison goes out the window, then.

Re:Finally

[techsoldaten]

I wasn't being a troll, I was sarcastically expressing my disbelief at these figures. The day an Apple anything costs 1/7th the price of a similarly equipped PC solution I will stop buying computers.

Seriously, consider the 1/7th figure: what does that include? Electricity? Location? Staffing? There must be something more to this...

Re:Finally

[techsoldaten]

This is straying a bit off topic, but could you explain that comment? I have price sheets in front of me from 8 different vendors that say you are wrong by a 8-14% margin.

Re:Finally

[Trurl's Machine]

If you want to match iBook's level of energy consumption, you have to go with Centrino, and that catapults you to the powerbook price region. Do "your 8 different vendors" offer 4-5 hrs of battery life, decent 3D acceleration chip with dedicated video RAM, ultrasilent fanless operation (in normal circumstances) and the same or smaller size + weight?

Re:Apple is more expenssive

[Trurl's Machine]

Well, except for Gadamer. Boy did I have a hard time with him.

Damn, I've read this sentence as "Boy did I have a hard on with him" ;-)

Re:Wow....

[Hoser McMoose]

Here's a few:

- MUCH smaller physically (1U and 2U racks vs. desktops). You can fit the entire 300-node cluster of these things in the space that you can fit roughly the same space as 54 of the G5 PowerMacs (assuming a split of about 64 of the 2U racks and the rest 1U racks).

- ECC memory. The Apple does not support ECC memory, so you can't trust your data (and, for the last time, that Deja-vu software is NOT going to make this magically go away!).

- It's cheaper. The Dell cluster cost $3.0 million for the computers, the Apple cluster cost $5.8 million for the computers. The $38 million figure included costs such as renovating the building and all the extras, while the VT cluster was a very bare-bones cost.

- You can use existing clustering software. There is VERY little, if any, software out there designed to support and maintain Mac clusters. VT has been developing most of the software themselves. For PCs there is a lot more software out there already. Of course, if you've got a bunch of grad students to do the work for "free", this isn't such a big problem :>

- Reduntant, hot-plug power supplies

- No useless hardware for the cluster. The G5 systems come with high-end ATI Radeon 9600 graphics cards, which are just a complete waste of time, money and electricity (= more money). They also come with Apple Superdrives (DVD writters).

- Faster hard drives (SCSI, potentially up to 15K RPM drives, though I can't find details on what UT bought). Drives can also be hot-swappable.

Of course, there are also some advantages to the PowerMac G5 setup:

- 64-bit capabilities. I don't know how far along their software is in this regard, but it's GOT to be better than the PSE used on PCs. This is the number 1 biggest advantage of the Mac

- More performance, at least in the theoretical flops. If they can get their software setup at all right, the Mac cluster should perform very well, particularly if they can vectorize stuff (you often can with clusters) and make use of Altivec.

- Decent resale value if they decide to break-down the cluster and sell it a few years from now. These are high-end desktop machines that can be taken out of the cluster and sold pretty much as-is. The UT/Dell cluster is a bunch of rack-mounted machines that can't really be sold. That being said, I don't know how big of an advantage this really is, typically you buy a cluster expecting to use it for quite a number of years.

- You've got a virtually limitless group of Mac zealots to do anything and everything you can possibly think of, just so that you they can get a look at this magical cluster of Macs that has been blessed by the high-holy Steve Jobs himself! :>

In short, the UT/Dell setup is a traditional cluster, the VT/Apple cluster is more of a novelty, but a novelty that shows some good potential if it could be designed properly, ie if Apple releases some G5 X-Servers. A G5 X-Server could make for some VERY nice clusters.

Price per Teraflop

[wembley]

As it turns out, the computer cost $3 million, vs. $5.2 million for the 17.6 terraflop Mac G5 cluster at Virginia Tech.

3.7 tflops / $3M = 1.23 tflops/$1M

17.6 tflops / $5.2M = 3.38 tflops/$1M

So with the Apples you get 2.75x more computing power for the same $$.
Sounds like a an easy argument to get by the Dean's office...

Re:Price per Teraflop

[grue23]

I'd be curious to see some of the other tradeoffs, such as the required rackspace per tflop. The G5's are in desktop cases at VT, I'm interested if we are going to see G5 Xserves sometime in the future that have a much better power/rackspace ratio.

I'm also curiuos how much the coolant systems were to build and to operate (and how much power both systems take, while we're at it).

I think it's great to see what Apple can do, but there's other metrics that would be of interest to universities as well.

Re:Wow....

[SilentEchos]

Easier to setup? Um no.. Look into the story a bit, UT hired CRAY super computers to set up their cluster while VT had hte volunteer help of some students.. Easier to setup..

AMD pic on article?

[Anonymous Coward]

Correct me if I am wrong. I thought that Dell only used P4 and Celeron processors, due to a deal with Intel.

Why not use Opteron systems?

[geekee]

A dual AMD Opteron system gets similar perforamnce when compared with a dual G5 system at a substantially lower cost. Plus you have more options in terms of cases, and may be able to get a rackmount solution.

Re:Bad Math

[Delphiki]

You've got to be joking. You're suggesting that SPECmarks is a better judge of a system than well... anything? SPEC marks don't reflect any actual use. If anything I'd think SPEC tests have had a negative effect, since people use those as a judge of the quality of a processor when it's really just a judge of how well a company can specialize a compiler and chip to do well on this one test.

umm... WHERE'S the OPTERON-BASED CLUSTER?!

[enigmals1]

...just wait. ;)

Imagine an x86-based cluster at less than the price of the Big Mac and faster. Intel used over 2000 CPU's at 2.4GHz. Now imagine the number of 2.4GHz AMD Opterons in a cluster. We'll see... ;)

Re:umm... WHERE'S the OPTERON-BASED CLUSTER?!

[Hoser McMoose]

There are about a half-dozen major clusters in the works using AMD's Opteron processor. Cray is building the largest, a 10,000 processor cluster that should hit 40 teraflops. IBM has a 1000+ node (2000 processor) cluster for Japan, and China is building a similar sized cluster. I've also heard some reports here and there about 256 and 512 node clusters build built by other organizations.

In short, the Opteron has been a VERY popular choice for clusters, though I don't think that any of the projects have reached the Top500 list yet.

Re:umm... WHERE'S the OPTERON-BASED CLUSTER?!

[enigmals1]

Yup, AMD is definitely nipping at Motorola's heals in the MIPs-per-GHz arena. ;)

so why is this story listed under "AMD"?

[Bored Huge Krill]

Apple doesn't use AMD, and last I checked, neither did Dell. So why is this an "AMD" story...?

Krill

I'm posting from A.C.E.S.

[davidstrauss]

I have nothing of truly insightful value to say, except that the whole A.C.E.S. fourth floor project (where they're putting the cluster) is kind of mysterious. It's still under real construction, as in glass, metal, drywall, and more. I'm not surprised they're putting the cluster in A.C.E.S., though. It's a nearly brand-new building with an easily upgradable communications infrastructure.

On a completely unrelated note, go tsunami! (UT's nearly campus-wide 802.11b WLAN)

Re:Finally

[w3weasel]

playing with trolls is surely a sign of boredom... so here goes.
Answer to question 1: 10.1 was slower than 10.2 was slower than 10.3. and Apple openly stated 10.0 was not 'production quality' software.

kernel optimization and network que enhancements are parts of BSD which Apple could not get working before the last release of their OS

read 'optimized' as 'tuned' to the Hardware Abstraction Layer of Apples hardware, which BSD was never optimized for. 10.2 featured impressive use of the processors resources, and efficient network queueing, but it is further improved in 10.3... I just don't see how it was 'not ... working'.
Apple has made zero claims regarding the performance of the new OS. They have only hyped the new features (is this where you got lost?). The RUMOR sites (rumor as in 'unsubstantiated' and 'unfounded') have provided all performance data.
If someone such as myself were to give real world examples of performance improvements in the new systems, and were legitimately using a real Developers Seed from Apple, then they would be in gross violation of the NDA. You could assume that reports are coming from people who do not have a legitimate Dev. Seed. (you can believe these reports or don't, your choice, but they are quite accurate so far).

I mean, yeah, the latest XP service pack is pretty much a bunch of bug fixes, but they also added USB 2.0 support, gave users new compatibility with about 200 consumer devices, and fixed numerous DHCP and networking issues. No system running XP is likely to run any faster than before the SP, but that can be considered a good thing next to Apple's program

I get the sense that you are acknowledging that the 'upgrade' in 10.3 goes further than what you might get in a windows SP... doesnt that refute your original argument?
10.2.3 was slower than 10.2.5 was slower than 10.2.6 was slower than 10.2.8. Speed gains were marginal, but measurable. and you really lost me by stating that no speed gain is a 'good thing' as compared to Apple.

No one is forcing Mac users to upgrade, and if you say the same about Windows, you should compare the respective EULA's of each.
I don't mind that you have an opinion but to imply that continuing to improve the software is equal to screwing the customer seems a bit daft.
If you are an Apple customer, and you think the price of upgrade is a rip, then install Debian, Suse, Yellowdog or your *nix of choice. If you are not an Apple customer, you need to come out of the closet, because your level of Applephoibia is clearly an indication of deep seated fruit-flavored tendencies.

And lets not forget

[autopr0n]

In this case, you're also building a whole office to put your computer in. (as UT built a new building to house the computers)

And the correction makes this *so* clear...

[King Babar]

So Slashdot corrects the story:

M: worm eater writes "The Register has posted a correction to the widely-reported story that a 3.7 terraflop Dell cluster cost the University of Texas $38 million. As it turns out, the computer cost $3 million, vs. $5.2 million for the 17.6 terraflop Mac G5 cluster at Virginia Tech."

Indeed. And now we can see that UT got a totally killer deal. Why, they paid only about $800,000 per teraflop while Virginia Tech, using horribly over-priced Apple hardware, paid an absurd $300,000 per teraflop of computing performance. As you can see, uh...wait a minute. Hmm...[spin, spin, spin]

Ah. So what do those Hokies think? That money grows on trees? How on earth are we going to see a recovery in the tech sector when people are only willing to pay for cut-rate supercomuter solutions that can't even run Windows natively?

:-)

Um, yeah

[autopr0n]

The $38 mill is the cost of building a brand new building, and supporting the cluster for five years (staff, power, networing, repair, ac, etc). The price on the macs is just the price of the actual machines.

The sad fucking thing is..

[garrulous]

that were always treated to a moving target with mac zealots. If the hardware is slower than x86, we get drowned in claims that the actual productivity is in the software, that is until some positive benchmark comes out touting fast mac hardware. Then hardware speed is the be all end all. Same thing happens with cost, availability software, ergonomics. Surely, I'm treating mac fans as a monolithic culture which is always misleading but cmon, give us a break. The precious mac platform would probably go a whole lot further with less slavering from the loyal adherents.

Another set of Apple G5 Benchmarks?

[fitten]

Sounds like another round of Apple Benchmarking (tm) conducted like at the release of the G5s :)

Tera, not terra

[more]

Ouch. Terraflops for teraflops. I will be waiting to see terrafeet (for astronomical distance measures) in US corporate press releases and popular science articles.

In other news

[autopr0n]

Morons learn the diffrence between a bunch of a computers, and a bunch of computers a new building to put them in and staff, mantnence, power, AC for five years...

To bad YOU didn't know

[autopr0n]

That the UT price includes the power, staff and even the building to put the computers in. Not just the individual computer cost.

apple delivers, finally!

[kraksmoka]

first, i have never heard of a university paying anything for labor. and if i recall, the VT is a volunteer project, and i for one would spare my time to play with hardware like that.

second, i have never heard labor factored into the cost of a machine (tho to call either of these beasts, just a machine is a shame, lol) but it would be reasonable for someon to calculate the TCO based on computing power and the cost of cooling, interconnect, etc.

third, the macs are using SATA which is today's technology, not ancient scsi, score one for cheaper hardware.

last, can't you just accept that apple finally hit a home run on the high end of the hardware world, for a welcome change from the stagnation of the wintel world? ? ? ?? ?

Re:apple delivers, finally!

[homer_ca]

Work study labor is subsidized by the school's financial aid. The employer (usually a university dept.) only pays about 1/4 of the student's wages. Is it too much to ask for the project to budget $2/hr for labor?

As for SATA, whoop-de-freakin-doo. Find any sysadmin who'd trade Ultra320 SCSI for SATA. The example I gave for a $4000 Dell 2650 server includes $2600 worth of Intel Xeon. The same system with dual 2.8Ghz Xeons is $2900. $100 less than a G5 tower built with (cheaper) desktop class hardware. Ok, so it doesn't have a DVD burner. I'll spot you the $100 there, and they're even again.

I'm not trying to bash Apple. They've caught up with Wintel in CPU speed, the rest of the components are commidity, and the prices are in the ballpark of high end desktops. Throw in that Steve Jobs signature hype, and it's a winning combo.

Re:apple delivers, finally!

[kraksmoka]

look man, 2/hr for labor adds up to the price of two or three machines. its like arguing the cost of the screws that hold the case of a pc together. i'm sure the accountants have a line item for it, but we aren't bean counters, nor have a line by line breakdown, lets use some nice round numbers, unless that is, we are the ones writing the checks. shoot.

all i'm sayin that's nice about SATA is that it is nice to see that apple is using the best commodity hardware out there, the new stuff, the stuff everyone wants, and wintel heads have been crowing about since it arrived. i missed that they went scsi in the article, i'm human, so sue me.

fact is, SATA will be easier to replace (compUSA or anywhere soon) and cheaper than scsi, as it is the successor to IDE and i think has a good clear shot at doing so.

yes, good product+good software+sane environment + reality distortion field = winning product

Re:he was being facetious

[justins]

Can you provide any more detailed information, links etc? It would be interesting to know if the risk is on the order of being killed by a meteorite, or what.

It's STILL more expensive

[m_niessner]

$3 million for 3.7 Terraflop = $810k per Terraflop

$5.2 million for 17.6 Terraflop = $295k per Terraflop

Which means the Dell is 2.75 times more expensive.

Re:It's STILL more expensive

[m_niessner]

You dumbass!

You have 2.0555 Terraflops per million dollars.

That means it's $485k per Terraflop.

That would mean it's still over 1.5 times as expensive.

...and 3.7 terraflop vs 17 terraflop

[ProfessionalCookie]

No one seemed to mention the price per terraflop, which is surprising 'cause:

$3.0m/ 3.7 Tflops = $810811/Tflop v.s.
$5.2m/17.6Tflops = $295455/Tflop

Why is there any question as to which was cheaper? Seems to me that the Apple cluster is the better deal and the Dell cluster still costs more than twice as much, with substantial headroom for error.

Why the hell...

[mnemonic_]

does this story use the AMD icon?

DOH!!!

[t0ny]

I guess the zealot's rush to judgement has been foiled again.

That's nothing.

[Valar]

One set of doors cost our athletic department over one million dollars. Go UT admin!

Dell states $3 million list, integrated

[mzs]

It is obvious that the $38 million figure includes more than the cluster, otherwise the per node costs would be impossibly huge even including the fiber. A coworker sent me this link that gives some concrete numbers.

Dell's list price of a configuration similar to Lonestar is $1.9 million, with services and installation charges expected to bring the total cost to around $3 million, a Dell spokeswoman said.

Since VT put the system together themselves we should really be comparing the $5.2M figure to Dell's $1.9M figure keeping in mind that Dell's number is list.

This same coworker points out that sustained performance is anywhere between 20% and 60% of peak on the top 500 list so that should be kept in mind. But he did get to do some preliminary QCD lattice simulation test runs on a dual G5 and he was getting 800 MFlops on one cpu and 1400 MFlops with both cpus on lattice sizes that did not fit in cache. This was an interesting result because the one cpu case was comprable to the best performance he has seen on x86 but the two cpu case beat the best dual cpu x86 he has seen by about 200 MFlops due to the better memory bandwidth on the dual G5.

The thing is that he has an SSE2 optimized version of the lattice code that beats the pants off of the G5 version. He expects to see a similar boast to the G5 numbers with AltiVec optimisations but cannot be sure. The trouble is that with AltiVec he can only do single precision. I am not sure which compiler he used on linux for the x86 numbers but he used the beta IBM compiler on OS X for the G5 numbers.

The G5 system is still 2.74 times cheaper

[StrawPoll]

The G5 system is still 2.74 times cheaper. The dell is $810,810.8/terraflop The G5 is $295,454.5/terraflop.

Can't be built for $5.2M

[Overzeetop]

This is a typical hype-story. Now, I'm from Tech, and we can count, but the math used is leaving a lot out of the equeation.

The press release quote from the Roanoke Times is "Tech plans to spend $5.2 million over the next five years on the computer cluster, which eventually will be housed in the university's fledgling Institute for Critical Technology and Applied Science." and is here: http://www.unirel.vt.edu/vtnetletter/sept03/featur e1.html

We'll pretend the facilities already existed and all of the people will be working for free (that's what graduate students are for anyways, right?)
So...
12000 SF of dedicated facility space (and a 24x7 staff)
Assembly, testing, and operation.
Sole use facility items (1.5MW of UPS and Diesel Generators)
170 Tons of cooling (HVAC & assoc components) ...is all free

Now, from the $5.2M (over 5 years) comes:

Power: 3MW x 24hx365.25dx5y x (1000x0.06c/kwH) = $7.89M

Oops, even if all the network, racks, cableing, custom cooling parts, and computers were FREE, we'd still be $2.5M over budget. Whoops.

This is somewhat simlar to UAB flying an amatuer radio satellite for $20,000. If you ignore the costs of everything, it always sounds cheap.

Heck, I can get you a brand new Mercedes S Class Limo for $50. That's as long as you don't count the $2000/mo lease payment for the next four years, and maintenance, or the $40,000/yr for the chauffeur (that's really operating budget, so it doesn't count toward the cost of the car).

Dell Deal Still a Gyp

[Shuh]

$3 million for 3.0 teraflop, vs. $5.2 million for the 17.6 teraflop


You don't need a calculator to know that.

Re:Wow....

[fucksl4shd0t]

Any advantages of the Dell supercomputer over the G5 setup?

UT is located in Austin, TX.

Dell is located in Austin, TX (home, anyway)

Therefore, the college just spent a fuckload of money that more than likely came from out-of-town, and some of it will stay in town. :)

Re:Finally

[fucksl4shd0t]

the network que

the network what?

MOD PARENT UP...

[ErnstKompressor]

Truth, truth, truth...

Misinformation from UT itself

[Durindana]

It's unfortunate that a number including various different ICES projects got attached to the supercomputer project alone. However, it's important to take this person's comments with a grain of salt.

Case in point: Miss de Marquez incorrectly claims Virginia Tech's total quoted cost for Apple's solution "ONLY includes the actual computer."

That's patently not true. VT's announced cost expressly included both the InfiniBand interconnects and the (massive) piped-water cooling system that, incidentally, accounts for most of the facility's projected electricity use.

Also note that Miss de Marquez labels the "value of the specific computer" $3 million. Does that number include cooling, power, the building and labor? There's no way to know, but that information certainly is available for Apple's machine. Telling.

Head on over to VT's page for more info: http://computing.vt.edu/research_computing/terasca le/

Re:Misinformation from UT itself

[GigsVT]

VT's announced cost expressly included both the InfiniBand interconnects and the (massive) piped-water cooling system that, incidentally, accounts for most of the facility's projected electricity use.

That sounds like part of "the actual computer" to me.

IT IS TIME FOR YOU TO STOP YOUR CRYIN'...

[ErnstKompressor]

Your use of the term 'da bomb' is retarded. You are retarded. You are a waste of resources. You are obviously a stunted individual, friendless, hopeless, destined for total obscurity. I look forward to not ever hearing anything about you. I have no doubt that you will eventually kill yourself once you have exhausted your limited ideas and realize that your l33t skillz don't even assure you a job at the local pretzel shack.

I'm sure your 'bomb' of an x86 with it's 'original' OS (WTF does that mean?)is real fast at downloading that bukkake porn that you fantasize so hard about, knowing that you will never find yourself in that same situation. It must really hurt inside knowing all this and still pretending that it doesn't. I sit here with my friends and we laugh at your infantile stupidity. Your opinions on these matters are always welcome because while you think that your barbs are so quick and biting -- oh that gay humor -- every AC post like yours is hours of laughter for us. So please keep them 'coming' ;)

I would return the "Shut the fuck up" sentiment, but I do really look forward to half-wits such as yourself, with their endless blathering about their PC's being 'straighter' than our Macs.

"Nimrod' is nicer than moron/retard/idiot...

[ErnstKompressor]

I was just trying to keep it a notch below rude and insensitive :)

Re:Bad Math

[Delphiki]

Intel and AMD have both used compilers which would not work in real use for their SPEC tests. You can reproduce the tests with different Fortran and C compilers, but you sure as hell won't get the same results as someone using a compiler specifically designed to perform well on SPEC tests at the cost of all else. You accuse Apple of handicapping competitors, but Apple used the same compiler for both systems. I haven't heard anyone list any actual problems with Apple's results for the G5. Just knee-jerk reactions from x86 enthusiasts who's egos would be too hurt if their precious pentium wasn't the best thing in the desktop marketplace. There were a number of articles which broke down every single one of the complaints made about those benchmarks, and showed they didn't have merit, and now that the G5's are out, people who are doing tests for real use are showing that the G5's are as powerful as anything in the x86 market.

And you seem to be under the impression I'm a Mac zealot. My primary computer is an x86 Linux/Windows box, so spare me. Do you honestly think I'm going to cry myself to sleep because some AC doesn't like Apple?

Re:Finally

[w3weasel]

queue, look it up
*sigh*

Support? This is Dell

[dbIII]

Methinks the price tag includes a lot more than the hardware costs.

Remember this is Dell we are talking about.

You don't get support, you just get to talk to some confused guy in Mombai who just started, and has to get through a huge volume of calls a day. You can't get parts within the week, you need a third party supplier that actually keeps things in stock to get your Dell parts from. Dell have reduced costs a lot by cutting inventory and support costs - however this doesn't help the customer.

If you have a system like this you have a few of your own support guys, keep spare parts in stock and order them direct from the manufacturer - not a re-seller (no matter how big they are) like Dell. ie. You run it like an industrial plant that is worth a fraction of the amount you lose for a weeks downtime.

Re:Finally

[fucksl4shd0t]

queue, look it up

*sigh*

Idiot. que en espanol = what in english

It's a joke. Laugh, or don't. Corrections not needed.

So you're actually talking about $18 Million

[Divx]

Even at $3 million, it would aparantly cost $18 million in dell products to build what $5 million in apple products cost?

Correction!

[Ohreally_factor]

The Register issued the correction that it really was the Inquirer all these years. The editors apologize for the error.

Re:Do a terraflop per buck comparison.

[Ohreally_factor]

Are we talking base10 or base2 dollars?

Re:IT IS TIME...

[Ohreally_factor]

You're thinking of Mac Eye for the PC Guy? Steve Jobs gets Bill Gates to wear black turtleneck sweaters? Jobs gets Balmer on a vegetarian diet? That would be, uh, da bomb!

Re:Finally

[w3weasel]

the network que

the network what? If you like what I said, you might like my music

Ah! now I see, from your use of mixed languages it was all so obvious! (idiot)

das Netzque que?

Se voce gostasse de che cosa ho detto ma musique

so very obvious

All that matters

[lungbutter]

so much back and forth on which is better/faster/cheaper. the only important thing to me is that Apple IS competing in this space at all. to see them price competetive (or even favorable) is simply remarkable. very interested to see how things add up in November when final results are in. and again when they upgrade to OS X 10.3...

Now that's a bargain

[draskovic]

Providing 25% of the performance at 60% of the cost, that's what I call a real bargain.

Re:And what about the Cafateria Lady?

[GreatTeacherMusashi]

hey man, dun underestimate the fine Dietrick Cafeteria ladies, truly though, VT is highly wired, especially where they're housing it, place is this close having ethernet jacks in the bathroom