Big Mac Officially Ranks 3rd

An anonymous reader noted that according to Wired, it will be announced officially on Monday the Big Mac supercomputer is the third-fastest super-computer. The article also talks about some of the amazing supercomputers in the planning stages. The sort of stuff that will make Big Mac look like that old TI-85 collecting dust in your drawer.

Comments

I'm still using my TI85....

[Anonymous Coward]

...you insensitive clod!

Kudos to the Mac

[Space cowboy]

It has to be said that Mac's haven't been famous for their speed, always pushing the "it does more", or "there are 2 procs" arguments, but this gives them some serious ammunition. Perhaps they'll even get their advert on the air in the UK now :-)

Simon

already official

[Anonymous Coward]

Clicky for the official November list

Trademark infringement

[Anonymous Coward]

Excuse me, but Big Mac is a registered trademark of McDonald's according to http://mcdonalds.com/legal/index.html

So what's going on here? Can they actually do that?

Re:Trademark infringement

[lost_it]

Virginia Tech has not (and will not) call the computer "Big Mac". BBC used the name when it first started appearing in the news, and everyone else picked it up, IIRC.

The people in charge of the cluster don't want to call it "Big Mac" because (1) they don't want a lawsuit from McDonalds, and (2) who wants to be associated with nasty, greasy fast food?

They've worked out a solid candidate for a name (it's not official yet) that isn't quite as catchy as "Big Mac", but it also doesn't have any of the downsides.

Internet distributed computing

[ergo98]

Given the basic benchmarks used to rank supercomputers, could a cluster of loosely coupled machines compete, or is the bandwidth demands for the benchmark set too demanding? I'm just curious how projects like what is detailed at distributed.net compare: 1100 dual-processor macs would be vastly outranked by the hundreds of thousands (or millions) of PCs taking part in distributed processing for various code cracking or cancer curing purposes.

Re:Internet distributed computing

[Anonymous Coward]

Hmmmm:

PowerPC 970 G5 1800 rc572: 13,400,000.00

AMD Athlon64 1600 rc572: 5,771,251.00

Intel Pentium 4 3500 rc572 4,960,583.00

Re:Internet distributed computing

[gcondon]

Okay, I took your advice and looked at the distributed.net speed statistics. I looked for the fastest PowerPC & Intel scores in each project. Here's what I found ...

PROJECT OGR:

CPU @ MHz = Speed
G5 @ 2000 = 19,180,166.00
G5 @ 1800 = 17,100,000.00
G4 @ 1250 = 13,946,216.25
P4 @ 3200 = 12,155,245.00
Xeon @ 2800 = 10,251,811.00
PIII @ 1440 = 9,570,000.00

PROJECT RC572:

G5 @ 2000 = 15,058,974.67
G5 @ 1800 = 13,400,000.00
G4 @ 1250 = 13,084,678.25
P4 @ 3200 = 4,502,730.00
Xeon @ 2800 = 3,935,299.00
PIII @ 1440 = 2,927,187.00

Of course, these numbers need to be taken with a grain of salt since there is only a few (or even one) top-end machine of each class in the statistics. However, contrary to your assertion, it appears that the PowerPCs kick ass compared to the x86s.

My understanding was that, if anything, the distributed.net algorithms unfairly favor the PowerPCs - esp. those with Altivec. I believe the Apple has used that fact in their advertising much to the consternation of many Slashdotters.

Re:Internet distributed computing

[JohnsonWax]

My understanding was that, if anything, the distributed.net algorithms unfairly favor the PowerPCs - esp. those with Altivec. I believe the Apple has used that fact in their advertising much to the consternation of many Slashdotters.

And, of course, problems in computational fluid dynamics and the like tend to also favor Altivec. Not always, but it's pretty common. As such, the value of Altivec shouldn't be tossed off as some kind of parlor trick. For these applications, it's quite relevant.

Re:Internet distributed computing

[netwiz]

yah, and there's a reason. The Altivec vector instruction set specifically provides an instruction in hardware that the RC5 routines (and most encryption in general) make heavy use of. 1GHz G4 procs kill the Intel hardware by about 3:1 (approx 9mil on the above #s)

That said, the intel hardware should rip thru properly vectorized FP code about as well (for functions it directly supports in hardware), and should kill the G5 on integer performance. Mostly has to do with the G5 having fewer integer units than the P4/Xeon. This has been pretty much proven with the SPEC scores we've seen thus far, with both sides using top-of-class compilers (none of this GCC crap).

And now, back on topic, I think that this says alot about the IBM FP hardware with regards to Intel. Intel bent over backwards for an architecture that's not much faster per clock, and given the apparent lack of clock scalability in the Itanium, they're going to be hard-pressed to keep up. Their lack of FSB bandwidth is going to hurt them as well, esp. in the server market. I know of at least two scientific apps (nothing big, just school stuff) that the Itanium chews thru faster than the G5, but only for small datasets. The guy showing me his results mentioned that when the dataset goes from 10MB to 500MB, the G5 ends up around 2-3 times faster, as the memory interface becomes a bottleneck on Intel.

This may be one of the core reasons LINPACK on hugely parallel systems brings the Itanium and G5 so close to each other.

It doesn't help that the PPC970 looks to cost about a third what the Itanium costs.

Re:Internet distributed computing

[martinX]

Don't know about that:

Apple and IBM jointly developed the PowerPC G5, world's first 64-bit desktop processor. With two double-precision floating-point units , advanced branch prediction logic and support for symmetric multiprocessing, the PowerPC G5 processor builds on previous PowerPC designs, combining an optimized Velocity Engine with a superscalar, super-pipelined execution core that can execute more than 200 simultaneous in-flight instructions. This high-bandwidth core has over 12 discrete functional units that process massive amounts of instructions in parallel.

And it has got to be the most totally buzzword compliant CPU ever built.

All from here and here.

Yes but...

[Nefrayu]

Can the Big Mac play games like the WOPR?
How about a nice game of chess?

Scant details on supercomputers...

[Spoing]

The Top 500 supercomputer list does provide the basic comparison information, though nothing like OS used or IO speed (network and storage). For that, you have to dig through each site and even then it is not easy to find. (The Earth Simulator uses SUPER-UX for the OS -- another Unix tuned to this type of task.)

That said, for what is provided, the Earth Simulator seems to be the current king by about 2x. (Corrections appreciated.)

Re:Scant details on supercomputers...

[Jack Auf]

I was also wondering about OS's used on the Top 500. I really wish they'd include the OS in the list as well. What would the OS statistics look like, I wonder?

IBM: 31%
HP: 28%
Linux: 20%
Sun: 12%
Cray: 6%
Other: 2.8%
Apple: .2%
Microsoft: 0%

Err, or something like that.

Check the #5 and #6

[Jesrad]

The Top500 site lists two competing 64bits architectures-based clusters: the Integrity rx2600, with 1938 Itanium2 at 1.5GHz (must be pricey), and an 2816 Opteron 2 GHz cluster, that achieves only three fourths of Big Mac's performance. Now that's a defeat for AMD.

Also, the VirginiaTech cluster is the only "self-made" supercomputer in the Top50 (the next one is ranked 63th, based on SunFire V60). The original #3 slipped to the 7th position because of the new supercomputers. Competition for that third place was tough !

Now where's the G5 XServe ? It was supposed to be out when OS X Server 10.3 was released.

Re:Check the #5 and #6

[damiam]

If you had read anything about this project, you would know that they have implemented their own low-overhead software error correction to compensate for the non-ECC memory. Presumably the benchmarks were done with this enabled.

Re:Check the #5 and #6

[The Infamous Grimace]

"...Then, as you have read it, you can tell the world how you do this "low-overhead software error correction to compensate for the non-ECC memory"..."

Taken directly from interview with Srinidhi Varadarajan:

Q: How do you deal with Error Correction in Memory?

A: There's a lot of traffic on Ars Technica and other places. We do failure recovery, memory doesn't report. One of the things we've noticed is that failures aren't an issue yet. The reason they can be competent is the LINPACK test, which is showing 16 digits of accuracy. We are planning on moving to ECC systems in the future. They may have to run things twice for a bit.

(tig)

Re:Check the #5 and #6

[tychay]

No, VT did not do a lot of "assembly-level hacking" one man working two months did port a bunch of code and he did use the best compiler and LINPAK on the market (Professor Goto's libraries). If LANL didn't do the same or better, I'd be disappointed.

Also you keep harping on the fact that it was "self-assembled." But then you go on to compare it to a system not provided by IBM, HP, NEC, or Cray but one provided by Linux Networx. Perhaps if VA Tech had gone to them, Linux Networx might have beat out IBM's Opteron bid of $9-10 million. But could have they gotten as low as $4.2 million--the list price from Apple?

You're going to have to face the hard reality that the Opteron may be an integer demon but the IBM 970 has it beat handily in floating point. The Rpeak of the 2Ghz Opteron (2 Gflops/s) is 1/2 that of the 2Ghz 970 (4 Gflops/s). Even accounting for the fact that the Rpeak->Rmax dropoff might be larger for the 970, that's too much to make up. Also, Virginia Tech considered the Opteron, but found that at the performance they wanted (specifically floating point performance) the systems would have cost twice as much ($9-10 million instead of $4 million) which is why the correctly opted for the 970 and which is why they're #3 instead of #6.

And that's without using Altivec/VMX/Velocity, since that unit can't do double precision add-mults.

As for heat issues. The 2Ghz 970 uses 47 watts which puts it approximately 1/2 the heat of a Pentium 4 and significantly less than the Opteron. IBM will be selling 2x1.6Ghz 970's in a blade configuration early next year and I'm sure "heat" isn't the reason for the delay. The issue with G5s in a 1U rackmount is that they won't exist until 1Q 2004 and it came down to availability. If you did any reading on the subject, you'd find that Virginia Tech's first choice was actually 970 systems from IBM, but they wouldn't be available in time--same thing happenned to the Dell Itanium 2 bid (it could have also been cost, Dell was "exploring pricing options"). The IBM Opteron bid was too high as was the HP Itanium 2 bid so they opted for Apple after the announcement. Smart move, two months of coding and several hundred pizzas later they have the #3 supercomputer. Any compromize, NCSA's gigantic P4 cluster would have beat them out.

Oh, oh yeah?

[Faust7]

The sort of stuff that will make Big Mac look like that old TI-85 collecting dust in your drawer.

Cluster a billion TI-85s together and then we'll see who's collecting dust.

THe bandwidth demands are high.

[mindstrm]

Interconnect is very important.

This is nothing like distributed.net.

For a problem that can be broken into millions of discrete, independent chunks, sure, distributed.net's model is fantastic, and works really well... (seti, folding, distributed.net, etc)

For something where you need lots of feedback from nodes, (like these benchmarks, and lots of simulation work), bandwidth is everything.

What's the difference?

[mindstrm]

I mean, I understand reasonably well the benchmarks used... but my question is this:

In the past, we always looked to the DoE or DoD for who had the fastest computers... they had stuff we could only dream of.. huge, fast clusters of funky computers we've never heard of.

Now, a university built one out of macs... and it competes with the same benchmarks.

What I wonder is, are there applications the old-style supercomputers are still better at, or has technology simply advanced since then? (Things like 10gig ethernet and ghz processors and memory busses, etc)... have we simply surpassed them? Don't just feed me some line about I/O either....

my switcher story

[caffeine_monkey]

I used to run an Intel-based supercomputer, but then one night, I was modelling a nuclear explosion on it, and all of a sudden it went berserk, the screen started flashing, and the model just disappeared. All of it. And it was a good model of a nuclear explosion! I had to cram and remodel it really quickly. Needless to say, my rushed model wasn't nearly as good, and I blame that Intel supercomputer for the fact that DARPA yanked our funding.

Its also the CHEAPEST

[goombah99]

Mac has never been known for being cheap, just a good value. Now even this seems to be fading: macs are a cheap too!

Now this system is the cheapest of the top 10. its cheaper than many it beat by a factor fo ten (more than that considering some of the building infrastructure are in that figure). Even more interesting these were stock mac at full price loaded with DVD-roms, firewire, blue tooth, the OS, etc..---not some stripped down model.

Its a good bet too that this thing is going to have lower maintainence costs and higher up-time given the macs attention to cooling, the use of high quality hard drives and power supplies, and high end memory chips. (on our cluster a tenth that size we blew 60 hard drives in the first 6 months and had to replace 10% of the motherboards.

Re:Its also the CHEAPEST

[Cecil]

I would guess that your guess is wrong.

Have you seen a G5? And do you know why your typical fans fail? Could it be perhaps because they are undersized, underpowered, cheap fans being pushed to their limits to try and brute force enough air churning around to keep your system cool? The G5s have a properly engineered case with a specific path for airflow, and abundant, high-quality fans mounted on rubber shock absorbers to dampen vibration. I suspect that these fans will have few problems. And if they do, won't it be a painful process to replace them: unlatch side panel, remove plastic airflow enclosure panel, put hand on fan assembly, slide out, slide in new fan assembly. Click, done. I wish my servers were that easy to replace things on.

Cheap hard drives? Yeah, sure, whatever. They use Seagate Serial-ATA drives. I don't know about you, but Seagate stopped fitting into my definition of 'cheap and crappy' about 8 years ago.

I have never dealt with ECC memory, so I grant you that point, however.

Re:Its also the CHEAPEST

[ZackSchil]

Have you ever even looked at a G5, seen its internal design, read the technical documents outlining airflow zones and the large, high quality, low RPM fans, the controller that measures temperature and keeps them moving only when they need to? No? Ever looked inside a generic PC box (as in technical slang and literally a box) with little to no airflow and 2 or 3 noisy fans drilling away as fast as they can the whole time trying to circulate air in and out of the little holes in the back? The G5 has been designed to keep cool with as little noise and wear as possible.

Re:Its also the CHEAPEST

[MrTangent]

"The machine is the first supercomputer based on Macs; it is one of the few supercomputers built entirely from off-the-shelf components and it cost a bargain-bucket price -- only $5.2 million. By comparison, most of the top 10 supercomputers cost about $40 million and up. The Earth Simulator cost $350 million."

The Earth Simulator is #1 on the Top 500 list as seen here. Quote taken from here.

"The $200 million (US$) computer is the fifth in the DOE's nuclear weapons simulation program, called Advanced Strategic Computing Initiative (ASCI), which allows the U.S. to keep its nuclear stockpile while complying with the nuclear test-ban treaty. Named "ASCI Q," the newest supercomputer will be capable of performing 30 trillion calculations per second.*"

Taken from here.

So, the #1 Supercomputer, the Earth Simulator, cost $350 million. The #2 Supercomputer, the ASCI Q, cost $100 million. The #3 Supercomputer, the Big Mac, cost a mere $5.2 million.

*Looks like ASCI Q got only 13.88 trillion calculations per second, not 30.

Re:Kudos to the Mac (don't forget the others)

[tychay]

Hmm, guess this means my submission a couple hours ago won't go through (dangit, Wired!)...

Here is the official press release and the list.

There is a lot of good points to note all around. The first is the G5 Terascale cluster at Virginia Tech at #3 (10.28 Tflops/s, 2200 CPU, Infiniband) is the first academic computer to break 10 teraflops/s. This extra performance was promised at Mac OS X Developer's conference last month. Not to sure if the price is a testament to Infiniband ($1.5 million cabling, cards, and routers) or the Macs ($4.2 million list).

Good thing too because in a surprise move the NCSA cluster made the list at #4 (9.82Tflops/s, 2500 CPU, Myrinet). This cluster is built using Dell's running Pentium 4 XEONs and Red Hat Linux! One subtle point to note is that they didn't get all the systems online in time (there should be 2900 CPUs, not 2500). I bet some programmer at PSC and an ex-Chief Scientist of SDSC is appreciating having a hand in edging out NCSA for #3--not to mention Apple beating Dell for #3.

The fastest Itanium cluster is at #5 (8.63 TFlops/s, 1936 CPU, Quadrics) which is looking like the odd man out boxed in by a PC based systems using Myrinet, the P4 Xeon above, and the most powerful Opteron system at #6 (8.05 Tflops/s, 2816 CPU, Myrinet). Another point of similarity:did I mention it's also using Linux?

And finally, It's easy to overlook #73, a single compute node of BlueGene/L (1.44 Tflops/s, 1024 CPU). Imagine 128 of these connected together and you have something that will easily take #1 when it's completed even if we handicap it 20-40%. As noted on SlashDot earlier, this will be running Linux.

Re:Kudos to the Mac (don't forget the others)

[tychay]

I miffed some of my links in the parent post:

Here is the reference to the ex-SDSC scientist.

Here is the link showing that the Opteron cluster is using Linux Networx.

Finally in the interest of full disclosure and to pre-empt the anti-Mac zealots, I should mention that the $4.2 million for the G5 machines is probably the education list price, because when you go to Apple Store, putting 2GB of RAM into 1100 2x2Ghz G5's will cost you $4.4 million (+ a little more for having some spare machines).

Congratulations VTech

[gsdali]

Now to get on with the research. It's a credit to them that this computer got from the drawing board to fruition in the tiny amount of time that it did. It's raised the bar for price/performance in the research computing world and hopefully many less wealthy institutions (I'm looking at UK universities especially here). At the end of the day its about the research they put into it and the results they get out of it.

I'm confused...

[Spoing]

Aren't there supposed to be comments about Microsoft here? This is Slashdot -- isn't it?

Well, to get the ball rolling, here is a query on the top 500 supercomputers using Microsoft Windows. Corrections and insight are appreciated.

Infiniband, not G5

[binkleym]

The G5 is a cool processor, but it isn't the reason the VT cluster is so fast, the Infiniband interconnect is. The LINPACK benchmark that is used to determine position on the Top 500 list depends very strongly on the latency of the network connection.

Infiniband has ~ 8-12 us latency (probably even less by now), while ethernet is an order of magnitude slower. In real-life applications it's actually worse than this suggests.

We have tested a real-life application (socorro) using both gigabit ethernet and Myrinet (slightly slower than Infiniband), and gigE took 600 seconds to finish a run, while Myrinet took 4.

VT's cluster is using the largest Infiniband network yet built (or at least announced). The previous largest Infiniband network was O(100) machines. VT could have built the cluster using Xeons, Itaniums, or Opterons and arrived at roughly the same level of performance.

Brute Force supercomputing = meh

[Boone^]

Run down the list and look at processor counts. We've got 5120 at the top (vector), but number 2 needed 8192 to get the job done. BigMac at #3 drops to 2200 and the processor counts hover in that 2000+ category. Until #19, when Cray's X1 jumps in at 252 processors.

Having a fast computer is cool and all, but if you can do it with 252 CPUs instead of 1024 (#22, P4 2.4), isn't that a win?

Besides, LINPACK doesn't stress interconnect latency and bandwidth, only cache and memory performance. When you run a "real" codes on these Mac/Xeon clusters and get 5% efficiency, suddenly the Earth Simulator (and the small Cray X1's) look good when they blow well past the 50% efficiency mark.

G5, not Xeons, Itaniums, or Opterons

[2nd Post!]

Yes, you're quite right, the networking hardware is important.

But as researched by the VT folk, the G5 is significant: It was cheaper for their needs than the Xeons, Itaniums, and Opterons of similar performance and energy consumption!

So both component choices were critical to their achieving number 3.

Can someone tell me...

[Whatah]

Why does every product apple make have its own icon ?

And yet equally, if not more, important products like amd64 don't have their own icons ?

Additionally, why does this CPU have a G5 icon? And not a PPC970 icon ?

Has slashdot sold out to apple ?

Re:Apples and Oranges

[prichardson]

Actually, you are horribly horribly wrong about two things.

You definitely could not do that with Opteron or Xeon systems. VT was in negotiations about price and delivery time with Dell and Apple. Apple beat out Dell's prices (shocking!!!).

Also, the G5 makes a great cluster computer. It comes standard with gigabit ethernet and has very easy access to parts (no screws required to install anything).

Finally, the Apples make a good cluster because in 5 years or so when they disassemble it they have 1,100 really nice desktop machines. PC's need to be upgraded more often to serve as a desktop computer (that's why Macs have awesome resale value compared with PCs).

Bad news for Itanium 2

[afantee]

The 1.5 GHz Itanium 2 costs over $3000 per chip, and even the 32-bit Xeon 3.06 GHz is about $1000, while the 2 GHz PPC 970 is about $300 or $400.  In addition, VT wants 64-bit chips, so Xeon is a nonstarter.

Excluding the Earth Simulator, the 2 GHz G5 has the highest Flops per CPU, even 5% higher than the 1.5 GHz Itanium 2 and 10 times cheaper:

#2 Alpha 13880 / 8192 = 1.69

#3 G5 10280 / 2200 = 4.67

#4 Xeon 9819 / 2500 = 3.92

#5 Itanium 8633 / 1936 = 4.45

#6 Opetron 8051 / 2816 = 2.85