Terascale Computing System Installed

lysie writes The Pittsburgh Supercomputing Center, with Compaq and the NSF, has installed the Terascale Computing System. Worldwide, it's second in power only to ASCI White at Livermore. However, it's the most powerful system in the world for unclassified research--6 teraflops per second. 3,000 Compaq Alpha EV68 microprocessors, in 750 four-processor AlphaServer systems running Tru64 UNIX."

Comments

If we could just get a Beowulf cluster of these!

[Anonymous Coward]

Running Freebsd emulating Linux. That would be cool.

A chance to run Windows now

[Anonymous Coward]

Maybe now I can have enough RAM to run Windows XP :-)

Re:Where's Linux?

[Metzli]

Why Linux? Tru64 has both good and robust clustering ability, even if it's not on par with VMS.

What's the point...

[Anonymous Coward]

..when you can just get a G4 and have supercomputer on your desktop?

battling with the speed of light

[imrdkl]

If one turns on the lights in an automobile simulator running on this machine, do they shine?

Re:Where's Linux?

[warlock]

Bah... why not Tru64, why Linux?

Obviously you've never used Digital Unix, and you are not familiar with their kick ass, highly optimizing compilers... they ain't gonna build a cluster like that to run apache+mod_php and serve crap you know, it's all about number crunching.

I was gonna say it...

[feronti]

But they're already working on building a Beowulf cluster of these. Though that sounds really wierd, now that I think about it... a Beowulf cluster of clusters... especially since each node in the cluster has 4 processors itself. Wow. Truly fractal computing.

teraflops per second ?

[Professeur Shadoko]

a flop is a floating point operation per second.
a teraflop per second would be an acceleration in processing power... not what the article means I guess

Re:teraflops per second ?

[yesthatguy]

We can start giving 0-60 times for computers...

My new desktop goes 0-60 megaflops in 0.0134 seconds! How's that for speed!

Re:teraflops per second ?

[Lars T.]

Well, 1.7 TeraFLOPS == 1.7 TeraFLOPs per second.

Re:teraflops per second ?

[T-Punkt]

"flops" is floating point operation per second. So if "flop" has a meaning at all, it would be "floating point operation" and thus "a teraflop per second" is "one teraflops" and a measurement of performance.

Top speed?

[Anonymous Coward]

So if it does 6 teraflops per second, what is its top speed, since it appears to be accelerating pretty fast?

Saying 6 teraflops per second is similar to saying 60 mph per hour

Re:Top speed?

[whopis]

It does 6 teraflops per second... but only runs for 1.5 seconds at a time... then has to restart. Everything evens out eventually

I thought the Alpha was all but dead...

[BitwizeGHC]

... good to see somebody getting some serious use out of that trusty old CPU architecture, anyway :)

Re:I thought the Alpha was all but dead...

[Relic of the Future]

Actually, as I've heard (third hand: a friend of mine is taking a computational physics course, and the professor is involved somehow, and he told them...) the PSC wasn't the only group interested in getting their hands a few hundred of these chips. If they hadn't placed their order when the did, the NSA would have gotten them instead. As it stands the NSA apparently has a few hundred on back order now... or so he thinks. If he 'knew', he couldn't say... but he's pretty sure. :)

teraflops

[Eslyjah]

6 teraflops per second

wow...6 trillion floating point operations per second per second. :-)

Re:teraflops

[Anonymous Coward]

Vroommmmmmmmmmmmm!!!!!!

Re:teraflops

[jfoust]

For what it's worth, SETI@home recorded an average of nearly 40 teraFLOPs/sec in the last 24 hours. Makes that supercomputer look positively pokey by comparison. :-) And it appears they're gearing up to handle even more data...

floating-point operations per second per second?

[bdg]

A rate of acceleration. The first computer to mature with age? Give it a year and it'll be doing 1.89e17 flops. Can you imagine a beowolf cluster of these? :P

Team fortress would be...

[Anonymous Coward]

Killer on this.

Then all the lag-bitches would ahve to complain about is their lousy ping (what you only have a T3?)

Scales like a real UNIX should

[SumDeusExMachina]

3,000 Compaq Alpha EV68 microprocessors, in 750 four-processor AlphaServer systems running Tru64 UNIX.

There will probably be a lot of people here asking "why isn't this running Linux?", without really knowing what they're talking about. First of all, Linux just doesn't have the kind of scalability that a commercial UNIX, particularly Tru64, does. Secondly, Tru64 is quite well-known for its excellent clustering capabilities, and its tight integration with the Alpha platform leads to high efficiency in computing. Finally, when you are paying $43 million for a supercomputer, you most certainly are going to be running the best software out there too, and frankly, the only reason that people out there are writing free software is that no one would want to pay for their code.

When you pay for the cost of commercial UNIX systems, you are paying for the assurance that 1) you aren't going to have stupid design flaws like the one the 2.4 kernel has in its inability to use virtual memory efficiently and 2) All of your nice new custom hardware is going to be supported, and frankly, high performance drivers for high-end hardware under Linux are sorely lacking.

Re:Scales like a real UNIX should

[Marcus Brody]

All though I agree with most of what you are saying, I think you should think about toning down the flamage:

the only reason that people out there are writing free software is that no one would want to pay for their code.

This is clearly not the only reason. There are a number of philisophical & practical reasons for free software. Furthermore, ther are numerous examples of people who are paid to write free software (e.g. linus, alan cox); and people who are paid to write propriety code (i.e. they are good enough programmers that someone is willing to pay them) in their job, but also are involved in free software projects in their own time.

Re:Scales like a real UNIX should

[Sircus]

"the only reason that people out there are writing free software is that no one would want to pay for their code."

I'm no expert on Tru64 scalability, but this level of flamebait makes this post highly suspect to my mind. Would someone who both knows something on the subject and can manage to comment without bad-mouthing the competition care to say whether this post is really +3, Insightful?

Re:Scales like a real UNIX should

[Anonymous Coward]

Don't worry. He has been metamodded accordingly.

Re:Scales like a real UNIX should

[Peter La Casse]

Linux (and *BSD) does offer some advantages over Tru64 and other high-end unices, the most notable of which is control. The big national labs (who are the primary builders of monster clusters such as this one) have the resources to add whatever features they want to the operating systems they use but they are often not allowed to modify the internals of proprietary operating systems. Also, many of these clusters run on Compaq Alpha / Tru64 systems, future support of which is in doubt. With a Free Software solution, the OS vendor will never go out of business (because there is no OS vendor.)

Re:Scales like a real UNIX should

[bconway]

Alpha was the first non-x86 port of Linux, done by Linus himself after being given a DEC machine as a gift on a trip to the US (though I don't recall if it was on loan or not).

Re:Scales like a real UNIX should

[Anonymous Coward]

Haha, you're kidding, right? First off, Compaq will let anyone see the Tru64 source that asks for it and won't give it away. All commercial UNIXes have done this for decades at universities and elsewhere. Secondly, as nice as it is to change the Linux kernel, the fact remains that it just doesn't scale like a real UNIX, and that's not a feature that you're going to add yourself.

Re:Scales like a real UNIX should

[oozer]

If you are a big enough customer you can get the source to almost anything. Many VAX customers had the source of VMS back in the day (not that you would want to read it as it was mostly written in Pascal and Macro-11 but still, if having the source gives you that warm fuzzy feeling all's the good). The same was true for large Solaris installations even before Sun 'opened' their source.

Bearing in mind that this machine was built by Compaq under contract I would find it inexplicable if the systems programmers on site did not have the source to tweak as required.

Re:Scales like a real UNIX should

[Anonymous Coward]

His idea of fun is getting together with friends and projecting the source of the Linux tree onto a big screen... they pass around a bottle of something strong and laugh at the kernel.

Sounds like real fun.. Just how fucked up can people be?

Re:Scales like a real UNIX should

[Anonymous Coward]

I agree - that's like saying that only reason people climb mount everest for free is that noone would want to pay them to climb it.

Re:Scales like a real UNIX should

[d^2b]

2 things that people have mentioned in choosing
DUnix over Linux.

1) Compilers
The compaq compilers are available under
linux.

2) SMP scalability.

This possibly is a bit of a red herring, since the SMPs are comparatively small (4 processors).

Another issue that I didn't see mentioned is
page coloring, and the VM system in general (hi Greg!). Many people find that there is a 10-20%
improvement on numerical codes.

Another issue is the familiarity of the Compaq
service organization with Tru64 vs. Linux.

Since it doesn't really cost them anything
to include the licenses, and it is probably
good publicity for Compaq's big iron sales
(remember, this was before Compaq decided to
flush all of its HPTC customer down the toilet)
why not?

Re:Scales like a real UNIX should

[Anonymous Coward]

Interesting you should mention page coloring, since Alpha is the only mainstream processor that still gains from it.

x86, ia86, SPARC and PowerPC all could care less about page coloring, thanks to more intelligent/complex cache designs.

Re:Scales like a real UNIX should

[Anonymous Coward]

But as time goes on, commodity systems will reach accpetable performance for most tasks (even scientific), so there will be little commercial incentive for maintaining these codebases.

If Tru64 can only distinguish itself from linux in such rare installations, its like doomed.

Re:Scales like a real UNIX should

[Paul Komarek]

Some of what you say makes sense, but mostly you sound like an advert. I'll pick on this line:
"When you pay for the cost of commercial UNIX systems, you are paying for the assurance that 1) you aren't going to have stupid design flaws like the one the 2.4 kernel has in its inability to use virtual memory efficiently and 2) All of your nice new custom hardware is going to be supported"

Having administered a Tru64 4.0 and 5.0 box, I can't agree with your statements about "what you pay for" when buying commercial UNIX systems. We had to upgrade because Tru64 4.0E did not support more than 8 SCSI devices on a single chain. Why on earth did we have to pay $1000 to be able to support an old SCSI standard?
We would have moved to linux, except that we have a half-terabyte of ADVFS-formatted data -- i.e. our data is "held hostage" by a proprietary file system format. If all goes well, we'll soon have 700GB of linux-readable space with which we'll rescue our data and then reformat the original array.

Oh, and let's not forget the time (before I was the admin, thank goodness) the machine was crashed by facilities to stop it from relaying spam -- turns out Tru64 ships (or shipped) with an open mail relay. linux has flaws, but at least you get the flaws for free! ;-) Oh, then there are the Tru64 network drivers for our old tulip based card . The card doesn't support full duplex, but Tru64 tried to make it support full duplex!

Now there's the reboot cycle it got into, which corrupted the filesystem. However, the disk check ran without errors, and there's nothing unusual in the logs. Tru64 has some great features, none of which we need. We're only using it because we have to. We only paid to upgrade it because we had to.

-Paul Komarek

Re:Scales like a real UNIX should

[Paul Komarek]

I didn't actually catch the model number of the machine, but I believe their using ES-40 (how do you pluralise a model number, without changing the model number?). We have an ES-40 in our lab, and I've spoke with the PSC guys about cheap (i.e. not from Compaq) memory for an ES-40. I got the impression that all of the nodes on their new big machine were ES-40s. But maybe it was a different "new big machine".

Anyway, we run linux on our ES-40 Model II, and it works great. The machine is a dream to administer (I last thought about it several weeks ago =-). However, we're not doing any parallel processing on ours, and aren't likely to anytime soon. We bought it for expandability. The ES-40 Model II will hold 32 1GB PC100 ECC wierd-physical-form-factor dimms, and up to 4 cpu cards. We like to think of the cpu multiplicity as "more machines with no extra administration". The memory is great, too -- we've got folks running 15 GB processes. Maybe the program didn't need to use that much memory; but the astronomers don't have to get fancy with their source code, and hence are saving a lot of time and effort.

You haven't lived (maybe died) until you've waited for a 4GB process to finish dumping core. Thank heavens I haven't been involved in any 15GB "accidents". =-)

I will say that our Tru64 machine (a Microway dual EV67, similar to a Compaq DS-20) has held up surprisingly well under hellish loads: load averages above 5 for a week are not rare, and I was able to recover from a load avg of 40 during an administrative accident. It will be interesting to see if linux does as well for us. However, I've got a lot of gripes about Tru64, and I'd prefer running linux even if we have to ask the users to play nicer.

-Paul Komarek

Re:Scales like a real UNIX should

[Anonymous Coward]

And for those that believe Beowulf is something so great, they'd better take a look at CPLANT, unless youd don't really like enjoying real clustering...

Re:Scales like a real UNIX should

[Apotsy]

Yes. And the same thing goes for compilers. People in Linux land actually seem to think gcc is the best or at least one of the best compilers available, but frankly, it's crap. I've seen applications get as much as a 60% speed boost simply by being recompiled with a commercial compiler instead of gcc. Gcc takes too long to compile, produces huge binaries, and can't do much optimization save for ultra-simple, textbook examples.

No doubt there are people out there who work on commercial compilers and sit around and laugh at gcc the way that HP-UX kernel engineer you talked about does.

Free software is good stuff for some things, but this idea that some people have about it being superior to everything in the commerical world is laughable.

Re:Scales like a real UNIX should

[ScumBiker]

That would be ES-40's. I think. It's been centuries since school. Back when we had to load the coal ourselves in to the furnaces.

Re:Team fortress would be...

[Anonymous Coward]

How True.

They could model all of Afghanistan being turned into radioactive slag, in less than 1 cycle.

Cool!

...and then Microsoft jumps in

[Arkoth]

Bill Gates throws a 150 million dollar payment check for the rights of running Microsoft Windows XP on the massive machine.
Leave the fastest computer alone!

Gates: You will run Windows, Resistance is Futile!

Now the following scenario is possible...

[Robber Baron]

#In the beginning was the computer.
And God said
:Let there be light!
#You have not signed on yet.
:God.
#Enter user password.
:Omniscient.
#Password Incorrect. Try again!
:Omnipotent.
#Password Incorrect. Try again!
:Technocrat.
#And God signed on 12:01 a.m., Sunday, March 1.
:Let there be light!
#Unrecognizable command. Try again!
:Create light.
#Done.
:Run heaven and earth.
#And God created Day and Night. And God saw there were 0 errors.
#And God signed off at 12:02 a.m., Sunday, March 1.
#Approx. funds remaining: $92.50.
#And God signed on at 12:00 a.m., Monday, March 2.
:Let there be firmament in the midst of the water and
#Unrecognizable command! Try again!
:Create firmament.
#Done.
:Run firmament.
#And God divided the waters. And God saw there were 0 errors.
#And God signed off at 12:01 a.m., Monday, March 2.
#Approx. funds remaining: $84.60.
#And God signed on at 12:00 a.m., Tuesday, March 3.
:Let the waters under heaven be gathered together unto one place
and let the dry land appear and
#Too many characters in string specification! Try again.
:Create dryland.
#Done!
:Run dryland.
#And God created Earth and Seas. And God saw there were 0 errors.
#And God signed off at 12:01 a.m., Tuesday, March 3.
#Approx. funds remaining: $65.00.
#And God signed on at 12:00 a.m., Wednesday, March 4.
:Create lights in the firmament to divide the day from the night.
#Unspecified type. Try again!
:Create sunmoonstars.
#And God created Sun, Moon, Stars. And God saw there were 0 errors.
#And God signed off at 12:01 a.m., Wednesday, March 4.
#Approx. funds remaining: $54:00.
#And God signed on at 12:00 a.m., Thursday, March 5.
:Create fish.
#Done.
:Create fowl.
#Done.
:Run fish'nfowl.
#And God created the great seamonsters and every living creature
that creepeth wherewith the waters swarmed after its kind and
every winged fowl after its kind. 0 errors.
#And God signed off at 12:01 a.m., Thursday, March 5.
#Approx. funds remaining: $45.00.
#And God signed on at 12:00 a.m., Friday, March 6.
:Create cattle.
#Done.
:Create creepy things.
#Done.
:Now let us make man in our image.
#Unspecified type! Try again.
:Create man.
#Done.
:Be fruitful and multiply and replenish the earth and subdue it
and have dominion over the fish of the sea and over of the fowl
of the air and over every living thing that creepeth upon the
earth.
#Too many command operands! Try again.
:Run multiplication.
#Execution terminated. 6 errors.
:Insert breath.
#O.K.
:Run multiplication.
#Execution terminated. 5 errors.
:Move man to Garden of Eden.
#File Garden of Eden does not exist.
:Create Gard.En.
#Done.
:Move man to Gard.En.
#O.K.
:Run multiplication.
#Execution terminated. 4 errors.
:Copy woman from man.
#O.K.
:Run multiplication.
#Execution terminated. 3 errors.
:Insert woman into man.
#Illegal parameters. Try again!
:Insert man into woman.
#O.K.
:Run multiplication.
#Execution terminated. 2 errors.
:Create desire.
#Done.
:Run multiplication.
#And God saw man'nwoman being fruitful and multiplying in the Gard.En.
#Warning: No time limit on this run. 1 errors.
:Create freewill.
#Done.
:Run freewill.
#And God saw man'nwoman being fruitful and multiplying in the Gard.En.
#Warning: No time limit on this run. 1 errors.
:Undo desire.
#Desire cannot be undone once freewill is created.
:Destroy freewill.
#Freewill is an inaccessible file and cannot be destroyed.
#Enter replacement, cancel, or ask for help.
:Help!
#Desire cannot be undone once freewill is created.
#Freewill is an inaccessible file and cannot be destroyed.
#Enter replacement, cancel, or ask for help.
:Create Treeofknowledge.
#And God saw man'nwoman being fruitful and multiplying in the Gard.En.
#Warning: No time limit on this run. 1 errors.
:Create good'nevil.
#Done.
:Activate evil.
#And God saw he had created shame.
#Warning: System error in sector E95. Man'nwoman not in Gard.En.
#1 errors.
:Scan Gard.En. for man'nwoman.
#Man'nwoman cannot be located. Try again!
:Search Gard.En. for man'nwoman.
#Search failed.
:Delete shame.
#Shame cannot be deleted once evil has been activated.
:Destroy freewill.
#Freewill an inaccessible file and cannot be destroyed.
:Stop!
#Unrecognizable command. Try again.
:Break
:Break
:Break
#ATTENTION ALL USERS ATTENTION ALL USERS: COMPUTER GOING DOWN FOR
REGULAR DAY OF MAINTENANCE AND REST IN FIVE MINUTES. PLEASE
SIGN OFF.
:Create new world.
#You have exceeded your allotted file space. You must destroy
old files before new ones can be created.
:Destroy earth.
#Destroy earth. Please confirm.
:Destroy earth confirmed.
#COMPUTER DOWN. COMPUTER DOWN. SERVICES WILL RESUME ON SUNDAY
MARCH 8 AT 6:00 A.M. YOU MUST SIGN OFF NOW!
#And God signed off at 11:59 p.m., Friday, March 6.
#And God saw he had zero funds remaining.

Supercomputers...

[maan]

I'm in a class at CMU with the head of the PSC...we've been having fun these past weeks, with him talking to us about this "machine". Seems their #1 objective right now is to submit the best possible score for the TOP 500. Apparently, the deadline was October 1st, but then they have some time after that to "rectify" their score...

There was a fun story apparently about a slowdown that was due to _one_ RAM dimm not seated properly... So 2999 processors were doing their job, but then waiting for the last processor to finish its job, which was taking much longer...

I've seen pictures of this beast. All I can say is: wow. So many cables, so many machines...

And apparently, they're not yet completely connected. Each box is supposed to have two connections to a "fat tree" quadrics network. Well right now they only have one... But it seems that Linpack isn't so communication oriented, so it's not too big a strain on the network.

Maan

Cheer up!

[UserChrisCanter4]

Pittsburgh Super Computing previously sold one of their old cray units on e-bay.

So, hey, maybe we'll be bidding on this bad boy in a couple of years...

Terascale Computing System Installed

[fea]

So why in the h$#% are we allowing Compaq to rid itself of Alpha processors? For those of us fortunate enough to have them (I use two at work), they are unbelievable number crunchers. Why? Why?

Re:Terascale Computing System Installed

[Paul Komarek]

At least Transmeta has shown some signs of marketing prowess. DEC and Compaq screwed up marketing the Alphas. At any rate, we've bought 7 alphas. =-)

And I don't think Compaq sold the Alpha line because they couldn't make it work financially -- I think they sold it because they wanted to make the whole company look attractive to HP. Thank goodness we still have the PowerPC from IBM. That will hold for the next 10 years, at which time Intel might finally deliver a good IA64 implementation. We've been waiting, what, 4 years already? -- with no end in sight, either.

-Paul Komarek

Re:Terascale Computing System Installed

[Paul Komarek]

Unfortunately, they didn't ask our permission. Including share holders like me. They ask me to vote for boardmembers, as if I ever cared. But when they decide to make a massive strategic change, they just do what they feel like. And selling the technology to Intel feels like twisting the knife. Mike Capellas, do you hear me? You JERK! I feel a little better, now, somehow.

At least they've still got the iPAQ. Who knows what its future is, though. Maybe we'll get lucky, and Intel will bail on IA64 and just make Alphas. I can dream, can't I?

-Paul Komarek

{sniff}

[A_Non_Moose]

Beautiful, man, just beautiful...

After reading all the specs...wow. Nice work.

"What Norman and other astrophysicists would like to learn is why star formation is so inefficient."

Hell, I could have told you that...this is "mother nature" at work, not a computer sim.
"she" does things in her own sweet time...

And look at the cooling specs...yikes. It should have been build in Alaska...outside...in winter.

Moose.

(fingers crossed this gets posted to the right forum...not like last time. Prolly serves me right...Used windows + ie + relaunch after an "internal error" + posting to /. heh)

Hey, I just noticed, maybe that is what I.E. stands for..."Internal Error"?

Re:{sniff}

[Anonymous Coward]

And look at the cooling specs...yikes. It should have been build in Alaska...outside...in winter.

something tells me that you've never sat outside the pittsburgh computer center freezing your ass off trying to smoke a cigarette on an october morning :)

-c

Re:floating-point operations per second per second

[Petrol]

So how long before this cluster converts all energy in the universe into heat?

Comment removed

[account_deleted]

Comment removed based on user account deletion

Science Editor :)

[Greta_Berger]

Let me quote the 'Science Editor':

"SYSTEM CAPABILITY: The 3,000 processors can perform up to 6 teraflops, or 6 trillion calculations per second. Virtually every man, woman and child on earth would have to perform a calculation each second to keep pace..." hmm... with my PC maybe :)

Re:Science Editor :)

[Anonymous Coward]

Hehe... that guy is smart... s-m-r-t

not just a MS slam

[Anonymous Coward]

agree with me or not, the thought is amusing of MS strong arming some organization into building a like system but with some Windows OS. I can just see the end figures then. "Capable of 6 Teraflops per second, 3,000 Compaq Alpha EV68 microprocessors, in 750 four-processor [insert system] running Windows [something]." Then you could probably read that a clean install and setup configuration (without any actual data or logic loaded) would take up about 5.7 of those Teraflops, and the 2.7 terabytes would be reduced to a useful portion of around 512 megs or so.

The funniest thing is how MS would in reality have to create an entirely new OS for this. Simply because no sane person would allow their spaghetti design on such an awesome system. Since most in the world know that when MS says 'integration' and it is actually 'embedded' I suppose that would just add to the problems with existing MS software. I don't think researchers would appreciate having to preformat any and all data they enter in to the machine to be required to be in a propietary and lawyer guarded format. IE would not be needed, but OOPS, that is EMBEDDED inside! And anytime there was an environment change, the entire machine would have to be rebooted. (besides the usual reboots from the horrid memory management and non partitioned memory systems) I can just see one tiny segment of one of millions of threads that has a divide by zero problem and suddenly the entire machine crashes!

HAHAHAHA, just imagine a protein folding program crunching away while the scientists eagerly await the results for medicine synthesis... suddenly a blue screen of death appears or some MS dictated process (why give the user the choice, what the hell does he know?) ends up destorying weeks of processing time (and lots of money) or even better, ends up giving the results as a giant MS trademark.

ok, it is a ms slam. But I have been dealing with WAY too many crashes, quirks, and lack of administrator administrating ability these past weeks with MS bloated crapware.

Re:I cannot see the point

[BigBir3d]

TRU/64 is used because Compaq Alphas are 64 bit processors, and Compaq wanted to use their own Linux distribution. This cluster is going to be used for scientific research, which is almost exclusively UNIX based, and always has been. This thing has to stay up for extended periods of time, it can't crash 500 hours into an an experiment that could require 750 hours, or more, to complete.

And, they want to cluster this with other clusters....mmm....Beowolf....

Can you...

[Anonymous Coward]

...just consider it said.

Links to more info on TCS1

[martyb]

Here are some links to more information:

• List of the Top 500
  
• Official TCS1 Status Page(but not yet updated)
  
• TCS1 User Documentation: The Terascale Computing System
  
• Slashdot article announcing Birth of a Terascale Baby (Oct. 4, 2000)
  

"teraflops per second"?

[pete0t2]

Is that measuring the acceleration of the chip?

Quake Comment

[chairmanKAGA]

I'm waiting for the standard slashdot Quake comment when discussing super computing =)

Re:Quake Comment

[Da Masta]

I wonder how this baby's stucture would survive through a 'quake measuring 8.5 on the richter scale.

ummm

[Anonymous Coward]

"Such rankings are transitory; with machines being upgraded and even faster machines planned, the Terascale could be obsolete within three years unless it is expanded"

yeah in 3 years 6 teraflops wont be shit :)

Comment removed

[account_deleted]

Comment removed based on user account deletion

Re:Where's Linux?

[Lumpy]

Why not Windows XP?

OW! Stop throwing things at me!

Some Science Editor...

[BadDoggie]

"Designers were battling against the speed of light, which limits how fast signals can travel the cables."

Umm... no. Designers were battling against the speed of electrons though a cable, signal attenuation and noise. The speed of light is that limit on the gigaflops/sec acceleration or something.

woof.

I'm off to a bar only 18 kg from my apartment for a 9V glass of beer.

Re:Some Science Editor...

[Anonymous Coward]

My impression is that the electrons move really slow, but that is not a problem, because the "speed" of a voltage change is actually the speed of the electric field through the wire, which is precisely the speed of light.

Re:Some Science Editor...

[Paul Komarek]

I suspect the speed of light problem had more to do with synchronization than with time-in-flight. That is, the difference of time-in-flight between computers is what mattered for maximum performance. For instance, it will take 10 times as long for a signal to travel 10 meters than it will for a signal to travel 1 meter.

I think it takes about 40 nanoseconds for light to travel 1 meter. A 750MHz cpu does one cycle every 4/3 of a nanosecond. In 40 nanoseconds, a 750MHz cpu has gone through 30 cycles. Because these EV68 chips are really beautiful superscalar processors, a lot of instructions are consumed and retired during 30 cycles.

If the time-in-flight between nodes is highly asymmetric, I expect it would be difficult to reasonably schedule work across all the nodes. With some machines 1 meter away, and others 33 meters, the nearest machines could get a signal 1000 cycles before furthest machines did.

Even without all the computation, you can see that the nearest machines could get 33 times more work done than the furthest, when waiting for synchronization signals. What the computations were meant to show was that a 750MHz cpu, and in particular an EV68 Alpha, can do meaningful work during this time.

Please take note that this kind of analysis is not in my field, and furthermore this post is the first time I've thought about it (though I am familiar with these chips, as I administer and use them daily). So please go easy on the flaming corrections!

-Paul Komarek

Re:Some Science Editor...

[Paul Komarek]

It just occured to me that a P4, with its super-long pipeline, would be even more sensitive to timing and synchronization problems in this sort of setting. How many stages does the P4 have, over 20? That's 20 cycles after receiving the "go!" signal before you get an answer out the business end of the P4.

I'd think this fixed-overhead incurred for each synchronization would have quite an effect for computational loads that required a lot of synchronization. Thus a cpu with a shorter (dare I say "reasonable"? ;-) pipeline would have an advantage on such loads. For instance, I'd like to nominate the EV68 Alpha. ;-)

It would be great if someone who knew what they were talking about corrected me, since I really don't know what I'm talking about. It would be really great if this person wasn't as lazy as me, and could compare actual pipeline lengths instead of dredging up old and unreliable memories.
-Paul Komarek

Re:Some Science Editor...

[flegged]

True. Electrons in an average electrical potential (say 5V) move through copper at a few centrimetres per second. By contrast, the speed of propogation the electrical field is c (300, 000, 000 metres per second), 9 orders of magnitude faster. It's the speed of the field that matters, not the electrons themselves.

Practical Uses..

[Lumpy]

The most practical use I see for it is the pleasure of pissing off Compaq by using a processor they decided to abandon... and 3000 of them no less.. That is priceless in it's own right, I would love to see their reasons for using the Alpha over the other platforms.

I would love to see some of the particle simulations this puppy could crunch, we might even see some accurate weather simulations finally..

I just wish there was a way to get that kind of power available to the universities.. Could you imagine what grad students could do if they didnt have to write a 20 page thesis and description of what they would want to run on the system? some of the best discoveries are the middle-of-the night AHA! sessions that need to be ran at that moment.

Re:Practical Uses..

[oozer]

You can bet that if grad students had access to such hardware the very first thing they would use it for would be to get the highest team score on Genome@Home.

Re:Practical Uses..

[Anonymous Coward]

PSC is run by Carnegie Mellon University, Pitt, and Penn State, IIRC. I'm only a sophomore at CMU and i've had accounts on two of their machines, already; one of them being the Cray T3E. So, right, students do have access to these machines.

Re:Practical Uses..

[PapaZit]

Actually, the PSC started as a joint venture between The University of Pittsburgh, Carnegie-Mellon University, and Westinghouse. (Trivia: their BITNET nodes were all prefixed with CPWSC for Carnegie Pitt Westinghouse Supercomputing Center: CPWSCA, CPWSCB, etc.) Practically, this meant that Westinghouse supplied the machine room space, CMU provided the staff space and handled administrative stuff (like paychecks) and, well, Pitt students and professors could work at PSC and deal with Pitt's payroll system instead of CMU's.

The PSCs networking group spun off into the National Center for Network Engineering (NCNE) when PSC proper lost it's NSF funding (due to sheer administrative stupidity, IMO). The PSC provides the internet connection to CMU, Pitt, and Penn State, but they have no other real affiliation with Penn State.

They're pretty liberal about giving out research accounts, so long as you're somehow affiliated with one of their funding sources. Currently, this mostly means that you have to be an academic of some sort in Pennsylvania (including students) or you have to be doing networking research, particularly for Internet 2.

Wow!

[Junta]

I know, in a few years 6 teraflops will be nothing, but just read the Slashdot post, we are talking about 6 tera floating operations per second per second, with that sort of acceleration, imagine the terapflops it can acheive in moments. And here I thought processing power would always have to move at a contstant speed, no wonder this thing is a big deal. It may be only second best to start with, but in just a few seconds, it will beat the pants off the most powerful system :)

Cool!.

[tcc]

"COOL!!!" that's the reaction when you hear a cluster solution from alpha...

"HOT!!!" is the reaction you'll get in 2 years from now when you'll see the same thing comming from intel.

:)

Linux stability on an Alpha Cluster

[mestreBimba]

While I was working for the INEEL (Idaho National Engineering Lab) the guys in the visulization and supercomputing center built a beowulf cluster of about 40 Dec Alphas running Linux, I don't remeber what flavor. They had some stability issues with the Linux kernel for the Alpha, and if my memory serves correctly the performance was also a little off what was expected, which was also traced to the Linux kernel.

Linux

[Anonymous Coward]

http://www.ncsa.uiuc.edu/News/Access/Releases/0109 11.Titan.html

Processors

[Anonymous Coward]

The EV68 is old news.

I want the EV69.

Mmmmya, that'd be good.

1 Calc/sec. per Person on Earth?

[LuxuryYacht]

SYSTEM CAPABILITY: The 3,000 processors can perform up to 6 teraflops,
or 6 trillion calculations per second. Virtually every man, woman and child
on earth would have to perform a calculation each second to keep pace.

Unless the population of the Earth has increased by 1,000 times recently, every person would have to perform a calculation every 1ms to keep pace.

Re:I cannot see the point

[T-Punkt]

> TRU/64 is used because Compaq Alphas are 64 bit processors, and
> Compaq wanted to use their own Linux distribution.

Err, Tru64 as it's called now, is no Linux distribution. It's Digital/Compaq's version of Unix for the Alpha architecture (or "AXP" if you prefer that). It had the names "Digital Unix" and "DEC OSF/1" before it was renamed to Tru64.

And please: Don't feed the trolls...

Linux/Tru64

[Anonymous Coward]

Actually it comes down to...

a) What Compaq supports.
You don't spend 30 million and get no support.

b) Compilers.
Compaq's Alpha Linux compilers are several revs behind their compilers on Tru64.

c) There is a known 10-15% performance hit between Linux and Tru64 on the same hardware on identical code. This is mostly due to item "b" above and some kernel issues.

Interrestingly a Intel based Linux cluster was bid against this machine and the NSF awarded this box.

Of course all this probally happend before the Alpha announcement.

All of that Tru64 is great for this and that is bull. Remember these are just 4 way boxes with an interconnect. The OS is not scalling. The programmers do the scaling...

Many of the Tru64 cluster tools are very bad.
You would be very scared if you knew...

Should be number #2 on the top 500 list I would think.

Not to mention

[epseps]

DEC/Compaq's kick ass tech support. If this much money is going to be blown on a supercomputer, you are going to want the experiece of a company that set the standard in tech support and service...instead of going to alt.os.linux and post "750 Node Cluster won't boot"

I need to get out of my cube

[Anonymous Coward]

I've been working at Westinghouse as a Computer Engineer for the last month and I had NO idea what was going on three floors below me. Guess where I'll be Monday morning?!

great another supercomputing center that doesn't..

[Anonymous Coward]

what do these supercomputer centers do with all that computing power ? crack pgp ? waste more money ? run more mars landing simulations ? figure out how the earth was formed ?

Re:rate of speed

[Anonymous Coward]

Bet you cringe when a cop says, "high rate of speed".

Re:Where's Linux?

[Paul Komarek]

As far as the compilers go, the DEC compilers and fancy math libs are available for linux (thanks for everything, Maddog!). Many of the optimizing features of the Tru64 version of these compilers are available on the linux version.

It seems like there is still a small performance delta in my experimental results (on our own Alphas, not on their cluster!), in favor of Tru64. But I can't be sure about this, and the delta wasn't large.

Thus my conclusion is that 1) they liked something about Tru64 besides the compilers, and/or 2) Compaq liked having their name on the OS running the cluster, and gave them a good deal.

Somebody else talked about support from Compaq. I expect the good folks at PSC know nearly as much about these machines as the Compaq engineers do. Furthermore, linux is a supported OS for these machines taken singly, though I didn't see "750-member ES40 cluster" in the support options... ;-). I expect that if you buy 750 Alphaservers from Compaq, you won't have to worry about support. =-)
-Paul Komarek

only won use

[rednuhter]

However, it's the most powerful system in the world for unclassified research--6 teraflops per second. 3,000 Compaq Alpha EV68 microprocessors, in 750 four-processor AlphaServer systems running Tru64 UNIX

SETI SETI SETI SETI SETI SETI SETI SETI SETI SETI
SETI SETI SETI SETI SETI SETI
SETI SETISETI SETI c where i am going with this .....?

big time social question needs answering

[Anonymous Coward]

mainly, why do I see mention of a "Hispanic Heritage Month"? Are we not a modern and enlightened society that can celebrate specific cultures any ol' time? Why must we be flooded with every company, govenment agency and so on with very hypocritically specific cultures? I don't notice a German Heritage Month, or a Scottish and/or Irish Heritage Month. I don't notice an Indian Heritage Month (as in India), a Russian or just Slavic Hertitage month. No Korean, Japanese, Chinese or just some other PC over(and incorrect)generalization called Asian Heritage Month.... see the problem here?

Let me inform you a bit more. Do you have ANY idea how much of your tax dollars are being spent for this stuff? How about how many of your tax dollars are being spent specifically for 'oversight' to force the American subjects to conform? Then we have all the special-self-interest groups that strong arm and coerce businesses and individuals into 'honoring' these specific months.

I guess what most don't know (they know very well, but do not CARE) is that this crap makes people incapable of seeing Americans, but now they see an African/Indian/Asian/etc American. Sounds and feels like 'sub people' Americans. Notice how it is known to most conscious people that American is defined as White Male? Notice a problem with that? Well, guess what? That problem exists because of YOU! Because you insist in labeling, and more specifically labeling others of different skin color and genetics as sub-Americans. Just like when you see the 'Its a Black thing man, you wouldn't understand'. Well, you are correct, I don't understand. But then later that same person actually accuses me of seeing them as different?!?!?! OF COURSE, I SEE YOU AS DIFFERENT! You called it out to my attention. Before that, you where just another person. This country confuses me. SOOOO much opportunity, but so many lazy hypocrites that would rather sit on their butts and bitch about perceived discrimination instead of working hard and MAKING SOMETHING OF THEMSELVES.

Have your computer solve this mystery?

Late reply...

[cr0sh]

Disclaimer: I am not a kernel programmer, nor a compiler programmer.

Maybe your friend (and his friends) need to get off their butts, quit passing the bottle, and help make the Linux kernel what it really could be. Same thing goes for people who "laugh" at the inefficiencies of GCC.

Why is it that they have to sit around, drink beer, and laugh at code on a big screen? That sounds as pathetic as a bunch of beer-gut guys watching football, instead of out there playing it.

Contribute! That is what is needed.

However, I bet I know the reason why your friend can do nothing but laugh - he probably sold his soul and signed an NDA. Sucks to be him.

Re:Late reply...

[cr0sh]

Yes, it is shocking that there are people who are all for themselves or for money, and none for others. That is shocking, and sad.

I understand your argument about kernels for large systems vs. small systems, and how the needs of one may cause issues with the other, if implemented (or not implemented, depending on the direction). Perhaps in this case, the kernel needs to be forked into a parallel dev effort, one side for PCs, etc - the other dedicated to larger systems.

I don't believe I have blind devotion - I use what I feel is best for my abilities - right now this is Linux. I have given thought to BSD as well, and also wondered and prodded about on various niche OS projects - but Linux seems to be the most viable, in that I don't have to worry about my hardware becoming quickly obsolete because of an OS change, and I don't have to worry about not being able to find and run a piece of favorite software because the OS no longer supports it. BSD allows this too, and I like its slower "rev" cycles - whereas Linux seems to be a frothy mess, everything being updated all the time - but it isn't something I worry about much.

I just feel it is better to give back - because I know in the end others will give back to me. It has worked for me for a while now, in many areas of my life (not just the open source community). At the end of the day, I know what I have done to help has made a difference for others. Sometimes, I am even told that it has. This to me is better than any amount of money someone could give me for my work.

Re:Late reply...

[cr0sh]

You are right - the debate will never be settled.

What do I do for a living? I am a software developer for a Phoenix company. Our stuff isn't open source, probably never will be (but who knows?). I don't work 90 hours a week - my time is my time. I will put in extra hours when it is the right thing to do, but I won't do it just for the hell of it (ok, sometimes I do put in extra hours - you get into that "mode", where time just flows, and code is flowing - great state to be in).

I don't think being paid to do a job is selfish - but if that is all you do with your life - working, getting money, being paid, never giving back - yeah, that is selfish. One could state that he did give back - to his employer - by working a ton of hours, but he didn't give back to the community. That is selfish. He didn't just learn programming on his own. He had teachers. You know it, he knows it, and I know it. I give back because of all the people I learned from typing in code from magazines and books from when I was a kid. I give back because of the numerous examples I have found online about ways of doing things. I give back so someone else may learn from me, and teach others along the way.

But I do this on my own time. Not my employer's, my own. I do this because I love computers and software and coding - not because of these things can potentially make me money - but because of the worlds they have opened up for me. The insights, the freedom, the knowledge, the friendships - all of it!!! These things are things I cherish - and I wish to give others the chance to share in the same ways and feelings I have tread and experienced.

It seems like today companies and people only want to make money, let no knowledge out, never truely give back. But should they succeed, they will simply be causing their own ultimate demise, for where will the new information creators come from? The schools? Perhaps - but what about those who don't go, or can't afford, higher education? Should they be denied these things? Should they not be able to program computers, render 3D graphics, or build their own OS, should they so desire?

The corps are saying "Yes! THEY MUST BE DENIED!" in their mad rush to censor and restrict the flow of all information - not just copyrighted information (which I don't have a problem with, were it not that copyright extends forever anymore - ie, Sonny Bono/Disney Copyright Act, DMCA, SSMCA, etc). They want to even stop libraries, the internet itself. They are killing themselves and don't even know it, nor care.

Don't cut out Open Source yet - it isn't over. The bubble burst because of bad investment decisions, investment by VCs who would take a business plan written on a barroom napkin. They were stupid, and arrogant. Many great projects have benefitted from the open source and GPL philosophies. Maybe they haven't made money yet because they haven't found the right business model. Maybe they won't find the right business model. But they should at least try, especially now in these more "sane" (and I could argue less sane as well) times.

As far as my beliefs in OSS - could I write an operating system as you say? No, I could not - not because I couldn't learn how to do it - there are plenty of ways to learn how, and I am sure I could learn it if I was so inclined. I don't believe I could do it, for the same reasons that Linus didn't create the entire Linux kernel himself today. He created and released, from the first release, a very basic kernel. Improvements by others rolled in, and he incorporated them steadily, along with his own improvements. The thing kept getting bigger, until it is at where we are today. Kernel creation and design by near anarchy, is what it is. So no, I couldn't do it myself. I would be willing to bet, though, with a proper plan and good software design, and the release of a simple kernal that followed that design, you might be able to amass enough people to continue with development. The problem would be getting enough people who have access to the same kind of large scale arhitecture, which may be where and why this kind of development would fall apart. Of course, if you could get a company to "donate" a large dev machine to work on, it could be done. I believe IBM (or someone) actually has done this. Whether they did it for altruistic reasons or not is another issue...

Comment removed

[account_deleted]

Comment removed based on user account deletion

Comment removed

[account_deleted]

Comment removed based on user account deletion

Comment removed

[account_deleted]

Comment removed based on user account deletion