Cray XT-3 Ships

imagine a... by Anonymous Coward · 2004-10-25 19:35 · Score: 5, Funny

single node of those.

Re:imagine a... by Anonymous Coward · 2004-10-25 19:56 · Score: 0

or a single node in a beowulf cluster of beowulf clusters... wait a tic, that would be a beowulf cluster!
Re:imagine a... by catch23 · 2004-10-25 21:05 · Score: 1, Interesting

Man a single node is $20,000?? I could build a pretty good opteron system for $2000.... so the other 18 goes to the interconnect? $2 million is a lot for only 200 processors.
Re:imagine a... by Anonymous Coward · 2004-10-25 21:33 · Score: 5, Insightful

*rolls eyes*
When you have a single CPU, designing the system to be pretty fast is easy. There's no major contention to deal with.
Two CPUs? Slightly harder, but reasonably straightforward. You don't see a 2x improvement in speed over one CPU, but it's around 1.95x, give or take a bit.
Four CPUs? Now you're starting to see less improvement ... probably around 3.2x, because of all the contention issues.
Sixty-four CPUs? You'll be lucky to get a 50x speed up over a single CPU.
When you get to 200 CPUs, the issue of access to shared memory and other shared resources becomes critically important. It's also an issue that most computer buyers don't need to worry about, because they don't have 200 CPUs in their system. This means that you have a lot of highly specialised research going on, and relatively few buyers to spread the cost of that research over.
Two million for a 200 CPU box which has low latency, low contention, and solid reliability is not a lot at all. You might not buy it. That doesn't mean nobody will.
Re:imagine a... by crimsun · 2004-10-26 00:35 · Score: 4, Informative

It's not just hardware: the amount of non-parallelizable code in parallel applications impacts scalability most tremendously.

The upper bound on speedup is generally Amdahl's law. Plainly, the efficiency approaches zero as the number of processes is increased. Generally we consider the major sources of overhead to be communication, idle time, and extra computation. Interprocess communication is considered negligible for serial programs in this context (we consider message passing). Idle time ends up contributing to overhead, because processes idle awaiting information from others. Extra computation is virtually unavoidable at some point; for instance in MPI's Single Program Multiple Data model, each process in tree-structured communication other than the root is eventually idled prior to the completion of computation, and each process determines IPC at some point based on rank.

There are notable exceptions to Amdahl's law, however; Gustafson, Montry and Benner wrote about such in Development of parallel methods for a 1024-processor hypercube, SIAM Journal on Scientific and Statistical Computing 9(4):609-638, 1988.
Re:imagine a... by ant_slayer · 2004-10-26 01:31 · Score: 2, Informative

My apologies, but I couldn't help but think that you'd be *really* lucky to get 50x out of 64 CPUs. Examine the following:

1 CPU @ 1.00x -> 1.00 / 1 = 1.000
2 CPUs @ 1.95x -> 1.95 / 2 = 0.975
4 CPUs @ 3.20x -> 3.20 / 4 = 0.800
64 CPUs @ 50.0x -> 50.0 / 64 = 0.783

Pop that into an OpenOffice.org spreadsheet and look at the graph.

That is not linear, in fact, it's non-linear in the direction that *helps* more and more processors. If the decline from 4 CPUs to 64 CPUs is a mere 1.7% efficiency compared to the 17.5% drop from 2 to 4, then, by golly, I'm going to cram hundreds of CPUs in there and see it tail off. Hello amazing performance.

Instead, reality is that the dynamics change. You can't evaluate "equivalent performance" to a single processor system. There is no reasonable metric with which to do so.

-Ant Slayer-
Re:imagine a... by thegnu · 2004-10-26 02:09 · Score: 1

Isn't there a point at which spending what we now consider an excess of cpu cycles on directing the flow in processes is more efficient? My oh-so-simplistic analogy is the jump from 2D to 3D graphics took a while, but it became more efficient to create frames from renderings of a massive 3D construct than a frame-by-frame drawing.

So, sure, there's more waste at the current point in the spectrum, but won't there be another curve in the opposite direction at some point? I think just the fact that 5 years ago, 5 processors was less efficient than now is an indication of things to come. There are constantly incredibly brainy people talking about the wall we're pushing up against, and we still haven't hit it.

Are we at the point where we have dedicated process processors (doing load balancing and only load balancing)? Keep in mind I know nothing about the subject at hand.

I'm fairly young, so my exposure to the advancement of computing technology is limited, but I haven't seen the indication that we're hitting a wall really. It might take some reevaluation, but when was that a bad thing?

The Bush Administration sez: "When yer president!"

--
Please stop stalking me, bro.
Re:imagine a... by Craig+Davison · 2004-10-26 04:36 · Score: 1

Come on, he was obviously pulling the numbers out of his ass. Like those people who keep saying "99%" when they mean "the vast majority, but I don't have any hard numbers beyond that".

--
Hands in my pocket
Re:imagine a... by Anonymous Coward · 2004-10-26 06:24 · Score: 0

Hmmh. Your estimates for scalability are actually rather optimistic. The general rule of thumb is that you might get something close to sqrt of number of CPUs (2x output for 4 CPUs, 4x for 16), meaning 200 CPUs would roughly be up to 16x faster than a single CPU. That's assuming general use, not tasks and algorithms designed for optimal parallelism, of course.
.... Same rule of thumb, I think, also applies to corporations... one with 10000 people can do maybe up to 100x times what a single skilled individual would. And that's why big corporations are such overhead-prone bloated dinosaurs.
Re:imagine a... by Anonymous Coward · 2004-10-26 10:03 · Score: 0

Oh I can imagine one. 400 MHz bus shared among 200 processors means the memory bus looks like 2 MHz to each one. Kinda slow. I guess that's why people either a) build clusters, b) limit SMP to about 16 processors (maybe 32), or c) use ccNUMA (coherent cache non-uniform memory access) to run up to about 512 processors. NASA built a computer with 10240 processors where 20 x 512 processor machines are connected together. ccNUMA solves the prosessor/bus latency issue (on Linux of course, Microsoft doesn't have anything in this space). On a cluster, each processor only has access to it's local memory. All other data is passed by a (usually) high speed network, with 1Gb/s as the minimum, (gigabit ethernet) or Myrianet (3.4Gb/s) or infiniband, hypertransport, 1394b, etc.

Comment removed by account_deleted · 2004-10-25 19:35 · Score: 2, Funny

Comment removed based on user account deletion

How big is it? by rooijan · 2004-10-25 19:36 · Score: 3, Interesting

I read the article (okay, so I kinda read it :-) ) and it has the speed and specs to be a geek's improvement on sliced bread. But how big is it, physically?

The article doesn't appear to mention its dimensions, and I'm curious to know what kind of space you need to install this baby. Anyone got any idea?

--
Daar is nie 'n lepel nie

Re:How big is it? by Anonymous Coward · 2004-10-25 19:41 · Score: 4, Informative

Dimensions (cabinet): H 80.50 in. (2045 mm) x W 22.50 in. (572 mm) x D 56.75 in. (1441 mm)

Weight (maximum): 1529 lbs per cabinet (694 kg)

http://www.cray.com/products/xt3/specifications. ht ml
Re:How big is it? by commodoresloat · 2004-10-25 19:42 · Score: 1

The weight is 1529 lbs per cabinet (694 kg). Imagine lugging that up to your 5th floor walkup apartment....
Re:How big is it? by Max+Romantschuk · 2004-10-25 20:00 · Score: 1

Sibling posts already gave the spec, but I believe the size of a system like this is rather insignificant.

If you're paying 2 million and upwards for a thing like this you probably can afford an appropriate space with appropriate climate control.

(OK, so some people cram ${car_price * 10} worth of HIFI equipment into a ${small_japanese_car}, but I doubt anyone would want one of these installed in their closet...)

--
.: Max Romantschuk :: http://max.romantschuk.fi/
Re:How big is it? by Anonymous Coward · 2004-10-25 22:50 · Score: 2, Funny

Dimensions (cabinet): H 80.50 in. (2045 mm)
Wow... for the first time in my life, I couldn't picture 80 inches, but I could 2 meters. I think there may be hope in the metric system afterall.
Re:How big is it? by Seanasy · 2004-10-26 04:09 · Score: 1

The 2000 processor version (10 teraflop peak) going to Pittsburgh Supercomputing Center is about the size of a living room (12'x28'). Compare that with PSC's current HP machine (~3000 processors, 6 teraflop peak) which is the size of a basketball court.
Re:How big is it? by afidel · 2004-10-26 16:12 · Score: 1

Actually as IBM is trying to prove with Blue Gene/L thermal density is a limiting factor in compute power. Also roughly 2/3rds of the cost of a supercomputer is operating budget which includes electricity both for the devices and ~3x as much for cooling, plus the cost of the facilities upgrade. In fact I know several academic institutions that have turned down one generation old supercomputers that would have been free because the facilities upgrade and operating costs were too high for their budget.

--
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.

I'll pass for now. by mrjb · 2004-10-25 19:37 · Score: 3, Funny

This is only the XT-3. I'll wait for the Pentium-3-4.

--
Visit http://ringbreak.dnd.utwente.nl/~mrjb/growingbettersoftware to download your free copy of the book

Re:I'll pass for now. by Pleione · 2004-10-25 20:08 · Score: 2, Funny

Don't you mean "AT/ATX"?

we're getting closer... by nilbog · 2004-10-25 19:37 · Score: 5, Funny

A few more years of advances like this and we might have a machine capable of running Longhorn!

--
or else!

Re:we're getting closer... by metlin · 2004-10-25 19:51 · Score: 3, Funny

Ahh, now that's what I call an optimist.
Re:we're getting closer... by NanoGator · 2004-10-25 20:05 · Score: 2, Funny

"A few more years of advances like this and we might have a machine capable of running Longhorn!"

A few more years of computer advances and this joke will still be modded funny!

--
"Derp de derp."
Re:we're getting closer... by Pleione · 2004-10-25 20:10 · Score: 1

Oh, quiet you...it was funny...sort of...well not really but you know what I mean?
Re:we're getting closer... by NanoGator · 2004-10-25 20:26 · Score: 2, Insightful

"Oh, quiet you...it was funny...sort of...well not really but you know what I mean?"

It was funny like a year ago. Now it's as overused as an SNL skit.

--
"Derp de derp."
Re:we're getting closer... by relaxmax · 2004-10-26 00:44 · Score: 1

You say Longhorn... I say games, OS's, VM's, Office stuff, Audio/Video utilities, yada yada yada.
Gimme back those PC-XT days when we had to boot the computer of a 5.25" floppy, followed by Digdug or Pacman. Even 0 B-KB-MB-GB HDD was enuff...
** sigh **

--
Love all, Trust few, Follow one.
Re:we're getting closer... by provolt · 2004-10-26 00:56 · Score: 4, Insightful

Ah the joys of youth.

Back in my day we spelled "enuff" without the 'f' character and it was good enough for us.
Re:we're getting closer... by drinkypoo · 2004-10-26 04:26 · Score: 1

A few more years of computer advances and this joke will still be modded funny!

A few more years of computer advances, and this joke will still be based around a kernel of truth.

--
"You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
Re:we're getting closer... by Tony-A · 2004-10-26 07:28 · Score: 1

The hardware giveth.
The software taketh.
That's the long and the short of it.
Hornwise, that is.

$2 million for a computer? by commodoresloat · 2004-10-25 19:37 · Score: 3, Funny

It better have a lot of good games. How many mouse buttons does it have?

I can't believe people complain about the price of iMacs....

Re:$2 million for a computer? by Maniac-X · 2004-10-25 21:47 · Score: 1

It's a supercomputer. You use it for taking over the wor- I mean, things like rendering a feature-length CG movie. Or doing the special effects for Star Wars ep. 3 _

--
(A)bort, (R)etry, (I)gnore?_
Re:$2 million for a computer? by Klar · 2004-10-25 22:18 · Score: 2, Funny

How many mouse buttons does it have?
Please.. it doesn't have any.. it just *knows* what you want to do before you *know* what you want to do..

--
Boxing Equipment Reviews
Re:$2 million for a computer? by Short+Circuit · 2004-10-26 00:33 · Score: 0

Your hands just seem to...sink...into the control panel. After that, you're mental interface with the computer will be good enough to cause an addiction.

--
tasks(723) drafts(105) languages(484) examples(29106)

real FPU operations by Barbarian · 2004-10-25 19:38 · Score: 4, Interesting

How are the Opterons at standard FPU operations in double precision? SSE2 and friends are nice, unless you have to make compromises in your simulations.

I ask, because I remember that the Athlons beat the pants off the Pentium 4's in FPU operations, so all the benchmarks were rewritten to use SSE2.

Re:real FPU operations by jmv · 2004-10-25 19:44 · Score: 4, Informative

Opterons beat the pants off the Pentium 4s in x87 (i.e. old) FPU operations. If you want to get good performance, you need SSE/SSE2. Both for AMD and Intel. For pure SSE, the Pentium 4s beat the Opterons mainly because of the clock speed, but for multi-processor systems, the hyper-transport and all more than makes up for that.

--
Opus: the Swiss army knife of audio codec
Re:real FPU operations by bersl2 · 2004-10-25 19:52 · Score: 1

I seem to remember it being the other way around.

Anyway, it depends on how you're using the floating point numbers: the standard 387 FPU instructions are faster, but the superscalar operations are more efficient when used in their intended role of vector calculations.

Or so I've heard. YMMV.

(BTW: nice sig...)
Re:real FPU operations by Pleione · 2004-10-25 20:13 · Score: 1

Isn't the P4 a generation behind the Opteron? What about 3DNowPro!, doesn't that stand for something?
Re:real FPU operations by ceeam · 2004-10-25 20:14 · Score: 1

Yeah, yeah, SSE... Nice, if you can manage your data properly aligned and don't mind manual assembler coding (even intel's compiler does so-so job at vectoring). Still, even then you don't get trigonometry, for instance.
Re:real FPU operations by Dogers · 2004-10-25 20:24 · Score: 1

I think 3dnow pro doesnt really include enough "extra" stuff to be worthwhile. It's mostly made up of the SSE instructions..
http://www.amd.com/us-en/Processor s/ProductInforma tion/0,,30_118_756_3734%5E3738,00.html

--
I am a viral sig. Please copy me and help me spread. Thank you.
Re:real FPU operations by jmv · 2004-10-25 21:02 · Score: 3, Insightful

Both SSE and 3DNow! get you (in theory, at best) two adds and two multiplies per clock cycle, even on an Opteron. So yes, just because of the clock, the P4 beats the Opteron in the case of pure (no memory/cache access, no depencency, nothing else) float operation. Now, in real life, you sometimes spend longer waiting for the data than computing with it and that's how the Opteron quite often comes out on top, especially for multi-processors.

--
Opus: the Swiss army knife of audio codec
Re:real FPU operations by jmv · 2004-10-25 21:07 · Score: 5, Interesting

Couple facts about SSE:
1) You can use it in scalar mode, in which case it's almost like x87, only a bit faster because:
a) It doesn't use a braindead register model (stack)
b) On P4, you can do a mul and an add in parallel with SSE, but not with x87
2) You can use SSE intrinsics. It's not as easy as "normal" programming, but easier than assembly and almost the same speed.
3) Unaligned access is possible. It's slower than aligned access, but overall better than non-vectorized code.
4) Trig is so slow that SSE/x87 doesn't matter (unless you write approximations, in which case SSE will also be faster).

--
Opus: the Swiss army knife of audio codec
Re:real FPU operations by Maniac-X · 2004-10-25 22:01 · Score: 1

http://www.tomshardware.com/cpu/20030422/opteron-1 9.html Looks to me like the opterons pretty much beat the pants off of barton 3000+'s in most FPU ops, but a few are only by a slim margin. Maybe that has something to do with the fact that there's two opterons, and only one barton. Its probably better to compare the dual opterons to the dual xeons... in which case the opterons did 2-3x better, in everything but mflops. But that probably has to do with the xeon's having almost twice the frequency of the opterons. (the xeons were 3.06GHz, and the opterons were 1.8GHz)

--
(A)bort, (R)etry, (I)gnore?_
Re:real FPU operations by Anonymous Coward · 2004-10-26 04:52 · Score: 0

Except the funny thing is fpmath=sse seems to be a strong way to HURT performance IRL. At least in several cases on both P4 and Opteron with gcc.

Check out acovea.
http://www.coyotegulch.com/products/acovea/acovea_ 4.html

"Some processor-specific options still do not appear to be a major factor in producing fast code.
Much to my surprise, I have yet to find any consistent evidence that options like -mfpmath=sse improve program performance. Thus Acovea bears out my personal experience, though it does not explain why so many people continue to suggest that I should use -mfpmath=sse to generate floating-point code. If someone could suggest a good "-mfpmath=sse", I'd appreciate seeing it."
Re:real FPU operations by jmv · 2004-10-26 06:15 · Score: 1

Yes, I've been experiencing that too while optimizing Speex. The main reason I think is that the gcc folks have been working on optimizing x87 code for more than 10 years, while SSE is pretty recent. Actually, until around 3.3, -mfpmath=sse would simply generate bad code on Speex. Anyway, I guess it just means that more work should be put on improving scalar SSE code, not just dumping the idea.

--
Opus: the Swiss army knife of audio codec

Heat... by Gurezaemon · 2004-10-25 19:38 · Score: 1, Interesting

I can't even begin to imagine what sort of heat this thing would put out, or what sort of power requirements it would have.

Re:Heat... by rctay · 2004-10-25 23:11 · Score: 1

These type systems either use floor ducting on a AC system designed for multiple cabinets/rack, or a dedicated freon cooling system. In the days of mainframes you would have a dedicated chiller system just for the computer room. The air and power was routed in by a modular floor grid system.

Just the name brings back memories by Dancin_Santa · 2004-10-25 19:42 · Score: 3, Informative

In this day and age of very fast computers and clusters built in our basements, there sometimes comes along a story that whispers of the computing age of days long past. Cray is one of those names that can drop a jaw just by the mere utteration of the name.

The name is synonymous with speed and power and the unwillingness to cut corners in order to shave a few dollars off the final product. When you buy a Cray, you know you are getting top of the line hardware.

It looks like Sandia wants to build the fastest supercomputer in the world by clustering a few of these monsters, and I have no doubt that they will. Looks like more fun articles about this in the future. :-D

There are two prominent applications for these machines. The first is nuclear weapons simulation. Personally, I don't see the point to that. The other application is in weather prediction. By feeding in current weather variables into a well-written model, a supercomputer is able to predict to a large degree of accuracy the future weather. Such an application will always be welcome.

I think I'm going to have to fire up the old ][e, the nostalgia is killing me!

Re:Just the name brings back memories by joib · 2004-10-25 21:59 · Score: 4, Informative

There are two prominent applications for these machines. The first is nuclear weapons simulation. Personally, I don't see the point to that. The other application is in weather prediction.

Oh, please. Buy a clue, will ya? There's lots and lots and lots of applications that use supercomputers, or could use if they were more affordable. A few examples from the top of my head:

Materials science, that is ab initio simulations, moldyn, you name it. This alone probably uses > 50 % of all supercomputer cpu time in the world. By comparison, weather prediction and nuke simulations is small potatoes (or shall we say, the simulations as such are big, but the number of people engaged in weather prediction or nuke simulation is really small compared to all the supercomputing materials scientists).

CFD, the automobile and aerospace sectors are big users.

Electronic design.

Seismic surveys, the oil industry uses lots and lots of supercomputers to find oil deposits.

Biology. Gene sequencing, moldyn simulations of lipid layers and whatever.

Climate prediction, somewhat related to weather prediction. Official purpose of the Earth Simulator.

All of the examples above could easily use almost any amount of cpu power you can throw at them. The only thing that stands between a lot of scientists and improved understanding of the world is computing power.
Re:Just the name brings back memories by LiquidCoooled · 2004-10-25 21:59 · Score: 2, Funny

There are two prominent applications for these machines.

Wrong! There is a third, more used application: Solitare.

Even super computer coders have to wait for results.

I also asked this recently, but didn't get a reasonable answer, do these beasts have screen savers? if so, Are they just blackout type, or busy 3d rendered whizbang super cool ones "Just because we can"?

(I realise you may not be able to answer that, but someone might)

--
liqbase :: faster than paper
Re:Just the name brings back memories by Anonymous Coward · 2004-10-25 22:43 · Score: 0

I also asked this recently, but didn't get a reasonable answer, do these beasts have screen savers?

You didn't get an answer because it's a stupid question, and it is a stupid question because you seem to think that a computer like this is just a really big version of your home computer. Never the less, the controlling machine is usually a workstation class machine, which is might run a screensaver if the administrators decided to. It would probably be running *nix, so it would probably run one of the many X screen savers.
Re:Just the name brings back memories by LiquidCoooled · 2004-10-25 23:35 · Score: 1

No, I asked this because this machine cannot be active and at 100% cpu usage all the time.

During the times of no activity what does it do?

Now, since you answered my question (running on x allows any of the screensavers), I will have to thank you Mr AC. :)

A secondary follow on question now springs to mind, how quickly would the SETI saver run on one of these boxes, has anybody benchmarked it?

--
liqbase :: faster than paper
Re:Just the name brings back memories by ocelotbob · 2004-10-25 23:55 · Score: 1

I also asked this recently, but didn't get a reasonable answer, do these beasts have screen savers? if so, Are they just blackout type, or busy 3d rendered whizbang super cool ones "Just because we can"?
3d screen savers, of course. In fact, this entire universe is actually a screen saver simulation running on one of these boxes. Just hope that no one decides to move the mou

--
Marxism is the opiate of dumbasses
Re:Just the name brings back memories by capmilk · 2004-10-25 23:57 · Score: 2, Funny

During the times of no activity what does it do?
It creates random noise that is then fed into the Seti project so our computers have something to do in times without activity.
Re:Just the name brings back memories by fitten · 2004-10-26 01:49 · Score: 1

I dunno... many of the HPC machines I've seen use batch schedulers and do, in fact, run at near 100% CPU utilization for weeks on end. When you have 1000s of scientists from all over the world all queuing up jobs to be run and each job takes a while, the machine tends to stay busy. We even had jobs that took multiple days to run even when running on 64+ CPUs (all of them busy at 100% for as long).

Many of these machines at the national labs (Sandia, ORNL, WES, etc.) are actually used by scientists all over the USA, if not the world, so it isn't like the machine only runs the jobs that the folks at the site need run.
Re:Just the name brings back memories by Atzanteol · 2004-10-26 02:15 · Score: 1

During the times of no activity what does it do?

Saves energy running idle? This thing takes a lot of power to keep running, and wasting it on a screensaver probably isn't very smart. I'd be surprised if X windows were even installed on this thing. This isn't your home computer!

--
"Ignorance more frequently begets confidence than does knowledge"

- Charles Darwin
Re:Just the name brings back memories by droleary · 2004-10-26 02:30 · Score: 2, Funny

There are two prominent applications for these machines. The first is nuclear weapons simulation. Personally, I don't see the point to that.

Well, when you nuke the site from orbit, you do want to be sure don't you?
Re:Just the name brings back memories by flaming-opus · 2004-10-26 02:48 · Score: 4, Insightful

Actually, there is no reason to cluster a few of these. If you have a 2000 node xt3 (or t3e, paragon, blue-gene, cm5, insert mesh-structured mpp here) and a 4000 node xt3, you stick them together and make a 6000 node xt3. But that's just picking nits.

Curiously the xt3 IS about shaving dollars off the price. If you go read the origional whitepapers on the system, they go through EXTENSIVE cost-return analysis. They studied their (then-) current generation of cluster systems, as well as future linux/solaris/aix clusters, and rejected them as (interestingly) FAR TOO EXPENSIVE, once the administrative costs are factored in. They then looked at, and rejected, cray's vector solution, the X1. They then decided that the (amazingly) most cost effective solution was to underwrite cray's product development cycle on a wholey new product. Basically they asked for an update to the system they already had. (asci-red i.e. intel paragon++) Nobody was building such a thing. Since cray had a really strong similar product in the 90s. (T3D, T3E) the department of energy asked them to create an update. Some designs never die.

What I'm most interested in is the reliability. One of the biggest difficulties in the T3D engineering cycle was dealing with memory failure. red-storm is going to have 10,000 processors. Lets assume each has 2 banks time 3 dimms (chip-kill) of memory. That means there are 10,000 x 6 x 18 = 1 million+ memory chips in the system. IF 1/100th or a percent of these fail, that's still a lot of memory failures. How well are faults isolated? That's the big question for systems this big.

I'm also a little wary of cray's use of lustre. I've used lustre before, as well as other cluster-FSes. While I'm not aware of other filesystems that will scale to 700+ i/o nodes, I'm not confident in lustre. It's an immature product at best. (I don't mean to disparage the people working on it, it's a neat architecture, but it's a hard problem, and I'm not sure it's ready for prime-time.)
Re:Just the name brings back memories by LiquidCoooled · 2004-10-26 02:59 · Score: 1

Excellent. Thanks for the insight.
It certainly makes sense.

(As you can probably tell, I haven't had any experience of big rigs)

--
liqbase :: faster than paper
Re:Just the name brings back memories by caveat · 2004-10-26 03:10 · Score: 1

...nuclear weapons simulation. Personally, I don't see the point to that.

Would you rather we resume testing to insure the continued viability of the stockpile? I agree, humaity would be better off without the things, but if we're going to keep them around we might as well be as sure as we can that if we do ever have to use them, they're going to work; personally, I'd much rather see the DOE doing things like this that have trickle-down benefit instead of vaporizing small islands in the South Pacific.

--

Facts do not cease to exist because they are ignored. - Aldous Huxley
Re:Just the name brings back memories by Anonymous Coward · 2004-10-26 04:18 · Score: 0

Yeah, weather. Pair a bunch of these together, and they'll change the freakin weather pattern so they can in turn predict what the effect of their exitistance has on the weather!

That's some power. No doubt.
Re:Just the name brings back memories by Anonymous Coward · 2004-10-26 10:16 · Score: 0

Nope. Good try but no cigar. If you want way too much speed, you need to look at IBM's BlueGene/L machine. They use a custom version of the power processor (not that I don't like Opteron, it's hypertransport technology comes right from DEC's Alpha). IBM's preliminary machine is a mere 8192 processors, performing at 8.66 teraflops, and ranked as the 4th fastest machine in the world. The completed machine will run at 360 teraflops (work to be finished in 2005) will lead the list. They already have orders for another one. Go to www.top500.org for more details.
Re:Just the name brings back memories by psetzer · 2004-10-26 10:40 · Score: 1

During times of no activity, they find something for it to do. Typically, if you've got a fast enough computer, there's people lining up just to get their hands on the thing for a few milliseconds. You use something like a workstation to talk to it, and it spends all of its time running FORTRAN or C.

--
"Anyone who attempts to generate random numbers by deterministic means is living in a state of sin." -- John von Neumann
Re:Just the name brings back memories by Inthewire · 2004-10-26 10:45 · Score: 1

"There's more than just speed"
-- Seymour Cray

--

Writers imply. Readers infer.
Re:Just the name brings back memories by ScrewMaster · 2004-10-26 13:53 · Score: 1

Yes, Corporal Hicks, indeed you do.

But the real question I want an answer to is: can this thing play Duke Nukem 3D Atomic Edition? On a really really big screen? Now there's a weapons simulation for you.

--
The higher the technology, the sharper that two-edged sword.
Re:Just the name brings back memories by Anonymous Coward · 2004-10-26 15:56 · Score: 0

I'd be surprised if X windows were even installed on this thing. This isn't your home computer!

While I don't think it currently runs on the XT-3, X windows runs just fine on the X1. There is even a copy of xclock that comes with the X11 software.
Re:Just the name brings back memories by d474 · 2004-10-26 19:02 · Score: 1

My angle on this issue of nuclear testing/simulation is that we (USA) tested these weapons for decades both in reality (South Pacific/Nevada/New Mexico)and in more recent decades in supercomputer simulations. How much more testing does it take to come to the conclusion that when you drop a big F'n bomb on a target that the target and everything around it for miles are totally eviserated?

After all, when the proverbial shit hits the fan and these weapons are used in any level of escalation, the subtle minutiae of precise details involving effective yield, fallout, etc., aren't really going to concern us - we'll be dead or dying anyway. Interesting how much money we are all willing to sacrifice to study the forcasts of our own self-imposed demise.

The world is what you make of it.

--
Authority questions you. Return the favor.

software by virtualone · 2004-10-25 19:42 · Score: 0

what kind of operation system runs on this beast?
did they write something froms scratch or did they just write some kernel patches
insmod torus_cpu.o

--
Only morons moderate based on a sig.

Re:software by Coryoth · 2004-10-25 20:04 · Score: 5, Informative

what kind of operation system runs on this beast?

UNICOS is usually a safe bet. In this case the specs say UNICOS/lc, which is made up of "SUSE(TM) Linux(TM), Cray Catamount Microkernel, CRMS and SMW software"

I'm not entirely clear how to interpet that, but I think it runs as follows: It runs the Catamount Microkernel as the kernel, and uses SUSE for everything else (so we have SUSE Linux, without the Linux - all of a sudden that GNU/Linux stuff starts to make sense). The CRMS is their interconnect management and monitoring software, and SMW is the System Management Workstation - which I'm guessing is their administration frontend.

It's worth noting that that's some pretty serious software there (because Cray has a lot of experience dealing with large systems) - you can bet that the management and monitoring software is some very serious stuff.

This thing is to a beowulf cluster what a dual G5 PowerMac is to homebuilt PC system running Linux From Scratch. It's going to work flawlessly "out of the box" with a smooth and polished interface that lets you get done everything you want to do simply and easily. You can of course make your home built PC with LFS work just as well, it's just going to take you an awful lot of effort.

Jedidiah.

--
Craft Beer Programming T-shirts
Re:software by Anonymous Coward · 2004-10-25 20:28 · Score: 0

The current "Cray" guys obviously are great at what they do. But most of that experience has not been around as long as many people think.

Several years back the original Cray was bought by SGI. Then a couple years ago, a Seattle based startup company called Tera with their own unique technology bought the Cray name and renamed their company.

I'm not sure how much technology transfer went along with the purchasing of the name "Cray".
Re:software by Anonymous Coward · 2004-10-25 20:39 · Score: 0

More detail on the history of Cray here:
http://en.wikipedia.org/wiki/Cray_Inc.
Re:software by Anonymous Coward · 2004-10-25 21:27 · Score: 0

Perhaps it runs Linux (kernel and all) on the service nodes, and the catamount kernel on the compute nodes?
Re:software by marsu_k · 2004-10-25 23:39 · Score: 1

It runs the Catamount Microkernel as the kernel, and uses SUSE for everything else (so we have SUSE Linux, without the Linux - all of a sudden that GNU/Linux stuff starts to make sense)
So should we call it GNU/Catamount?
Re:software by Anonymous Coward · 2004-10-26 00:27 · Score: 0

According to TFA, Linux runs on the login-PE's (processing elements), to give a GUI for the operator. The other PE's, those doing the work, run UNICOS.
Re:software by flaming-opus · 2004-10-26 03:00 · Score: 3, Informative

This split microkernel architecture has been in use for a long time on big mpp systems like the paragon and the t3e. The software base (catamount/linux) is new, but the design is old.

catamount is the kernel that runs on the compute nodes. IT's a tiny kernel that packages up the OS service requests, and sends them, over the interconnect, to an OS or I/O node, which does the real work of the operating system. catamount is a descendant of PUMA, which came from Cougar. These are heavily derived from work done at caltech. (I believe CMU, and one of the UTexas schools also played a role, but am not sure). The idea is that the microkernel is small and unobtrusive, and it gets the hell out of the way so the application can use the CPU as much as is possible.

The OS and I/O nodes run linux, and provide services to the compute nodes. This is probably, but it could just as easily be running as a user-space daemon on the OS node. (Though you might have to do some mem-copys that way, which would lower performance)

NOTE: Though these nodes take advantage of some of linux's features (like the lustre file system) they do NOT necessarily implement these features for the system as a whole. They probably provide a minimal set of features necessary for the sorts of problems that the xt3 runs. All the scheduling work that has gone into more recent linux kernels is of little use, as the compute nodes have their own scheduler, probably more closely tied to the batch dispatcher than to the linux kernel. To say that the system runs linux is true, but a little misleading. It's a very different linux than what runs on my desktop, and it's used in a very different way.

So......the cost compared to? by ericdano · 2004-10-25 19:42 · Score: 0

So, how does this compare to running Apple's Xserve? Bang per buck? Heat? Space? Etc etc....

--
It's either on the beat or off the beat, it's that easy.
I moderate therefore I rule!
--

Re:So......the cost compared to? by the_2nd_coming · 2004-10-25 19:52 · Score: 2, Informative

X-serve clusters would be cheaper, but I think that Cray has the edge n the interconnect tech. So, you need massive bandwidth in the system, get the Cray. you need next best bandwidth at a low price, get the Xserve cluster.

--

I am the Alpha and the Omega-3
Re:So......the cost compared to? by Anonymous Coward · 2004-10-25 19:53 · Score: 0

Answers:

1. Better.
2. One library of congress.
3. Great movie.
4. Sure is a lot of it.
Re:So......the cost compared to? by Coryoth · 2004-10-25 20:16 · Score: 4, Informative

So, how does this compare to running Apple's Xserve? Bang per buck? Heat? Space? Etc etc....

There's not a lot to compare. We're talking apples and oranges. It's like asking to compare a PowerMac G5 with a bunch of PC parts scattered on the floor as desktop machines. Sure, you can put the PC together, load it with Linux, tinker with it to get everything working, etc. but that's a fair amount of work compared taking the PowerMac out of the box, plugging it in, turning it on, and having everything work perfectly.

Read the specs, particularly with regard to the interconnect, system administration, and hardware and software reliability features. This thing is seriously engineered to be massively parallel system with top of the line hardware and software to support and maintain that, as well as extremely impressive reliability features.

Jedidiah.

--
Craft Beer Programming T-shirts
Re:So......the cost compared to? by Anonymous Coward · 2004-10-25 20:52 · Score: 0

Sure. Someone spending $2 million++ would be very concerned about the space it occupies and the heat it generates. It is quite hot working on my HomeCray but the speed means I can read all of slashdot in a few seconds.
Re:So......the cost compared to? by Lussarn · 2004-10-25 21:29 · Score: 1

So what you say is that not even the macintosh can touch the 30000 proc cray, but it has the second place pretty much secured. And to a better price. Dream on.
Re:So......the cost compared to? by adzoox · 2004-10-25 22:05 · Score: 2, Informative

You say it's comparing Apples to Oranges but its not really ...

The VT Supercomputer specs vs the Cray specs page you pointed to:

CRAY 460 GFLOPS per cabinet (96 processors @ 2.4 GHz)

Apple - if my math is right - 420 GFLOPS (100 processors @ 2.0Ghz)

The new specs for the specialized VT Supercluster are pretty impressive.

Their throughput and interconnect is most likely weaker - but still VERY strong with fiber channel.

--
Yell & scream & rant & rave... it's no use... you need a shaaaave ~ Bugs Bunny
Re:So......the cost compared to? by Coryoth · 2004-10-25 22:28 · Score: 1

You say it's comparing Apples to Oranges but its not really ...

Oh, why is that?

The VT Supercomputer specs vs the Cray specs page you pointed to...

Right, so if you compare Virginia Tech forking over money not just for Xserves, but also for all the interconnects, the cooling, setting up the system, developing maintenance, monitoring and administration software etc. to buying a Cray and plugging it in they have almost comparable performance?

If I buy a cheap AMD CPU, cheap motherboard etc. overclock it, install a Linux From Scratch system on it over a couple of weeks, and get a desktop running (that all the Apple fans deride), that's comparable to buying a G5 PowerMac and plugging it in, because, you know, the performance specs are similar, and you can fiddle the Linux desktop so it looks like OS X...

Cray is to supercomputers what Apple is to desktop machines. If you want to argue that a bunch of PC parts is comparable to an Apple desktop, go for it.

Jedidiah.

--
Craft Beer Programming T-shirts
Re:So......the cost compared to? by ozbird · 2004-10-25 22:29 · Score: 2, Funny

There's not a lot to compare. We're talking apples and oranges.

No, we're talking Apples and Crays... Didn't you read the post before replying? ;-)
Re:So......the cost compared to? by Anonymous Coward · 2004-10-26 00:24 · Score: 0

Fibre Channel, that's what, 2 Gb/s? That's 266 MB/s (peak) per channel. I don't know the topology.

The Cray XT3 has 7.6 GB/s (peak) between each and every uniprocessor node. In a 3D grid topology (6-ways from every node).

Expand the Big Mac to 5000 CPUs and see if it still keeps up. Also make sure to benchmark something else beside Linpack.

And check after 5 years which one has had easier maintenance.
Re:So......the cost compared to? by Anonymous Coward · 2004-10-26 04:12 · Score: 0

They paid 10.2 million at VT and rank above MANY Cray offerings!
Re:So......the cost compared to? by Jeremy+Erwin · 2004-10-26 04:21 · Score: 1

What's this about fibre channel? System X uses Infiniband as its primary communications fabric.

Infiniband for communications fabric

twenty-four 96-port switches with 10/20 Gb fabric (depending on duplex)

fat-tree topology

choke point is the bus: it is faster to send data across the room than the motherboard. Half the nodes can talk to the other half at the same time: 1.25GB/s

Ultra low latency fabric (less than 8s)

Each card has two ports, they use one.

source
Re:So......the cost compared to? by the_2nd_coming · 2004-10-26 04:34 · Score: 1

you idiot!!!! I am talking about clustering. if you clustered 30,000 CPUs of XServes together, you would get a really good bang for the buck because it would come in way cheaper than the cray or comparable systems.

reading comprehension must have been low priority at your school.

--

I am the Alpha and the Omega-3

hmm . .. by TheScorpion420 · 2004-10-25 19:43 · Score: 0

Its 2:43 am CST with 12 comments and its already slashdotted, damn and I was really looking forward to reading that . . .

--
If you pay your taxes you support terrorism!

cray by Exter-C · 2004-10-25 19:43 · Score: 1

Cray was at one time the fastest system.. then it went belly up.. now its back.. lets hope its here to stay for the long term. With some of the technology flowing through it would be a great thing for the industry.

~If only HP would keep the Alpha on and continue the development rather than crush one of the only real risc procs.

Re:cray by Anonymous Coward · 2004-10-25 21:12 · Score: 5, Interesting

Cray never went "belly up". It was acquired by SGI around 1997 or so, then divested and merged with Tera, who renamed the resultant entity "Cray Research".

Although it's true that Cray was not growing strongly before the SGI buy-out, it was not failing either. It could have kept running quite happily for many years, but in the bizarro-world of Wall Street, a company which is not growing is dying. I so love it when economists use biological terminology for corporations. In Wall Street's thinking, the only healthy growth would be a cancerous tumor.

Anyway....

The whole SGI-period of Cray is actually quite fascinating, and I suspect the true story will never be fully known. Lots of SGI engineers had their non-Cray technology branded with Cray marketting names, most egregiously LegoNet becoming CrayLink. Lots of Cray folks - aka. Crayons - felt that the core of their company was gutted by an SGI operation which didn't care for the extreme high-ends of HPC.

One rumor I heard, from a well-placed source, is that the Cray merger with SGI was primarily arranged by the USG. The intelligence services have huge investments in both company's products, so the merger between them made sense. I was told that as a quid-pro quo, the USG had an in-principle agreement to continue purchasing Cray gear to provide enough revenue inside SGI to keep both Cray architectures alive. However, certain parts of SGI felt that the US government didn't live up to their agreement, negotiations to rectify that weren't successful, and so SGI management defunded significant aspects of the Cray engineering work.

Also, FYI, Cray is one of those companies which will never totally go "belly up" anyway. Given the sensitivity of the work which they did, their support databases alone are full of sensitive and/or classified information. Should the company cease trading, it would be acquired by a shelf company whose sole function is to ensure this data would remain private. That's been the fate of almost all of the now-defunct supercomputer and high-end graphics companies who formerly supplied the defence and intelligence market.
Re:cray by Fred_A · 2004-10-25 21:52 · Score: 2, Funny

It seems to be really lacking in the blinkenlight department though.

What good is a supercomputer without blinkenlights ? They just don't make them like they used to...

--

May contain traces of nut.
Made from the freshest electrons.
Re:cray by Anonymous Coward · 2004-10-25 22:48 · Score: 0

That's outragous! I demand they add a panel of blinkenlights, two reel to reel tape drives which spin back and forward and a clacking teletype! I'll take a glass tty which makes teletype noises as it slowly prints each character to the screen, at a pinch.
Re:cray by flaming-opus · 2004-10-26 03:26 · Score: 1

Pre-SGI cray needed help, badly. They just had too many product lines for a post-cold-war supercomputing company.
They had the C90-to-T90 transition, which they were doing almost completely by themselves. Even IBM had given up on ECL. T90 was fast, but it was really really expensive. 32 processors sharing a flat memory is just too daunting, even without caches.
The J90 was profitable in the end, but took a lot of capital during the design work. Furthermore it detracted from the T90's transition to IEEE fp.
The T3D was a really cool product, but the engineering costs were huge.
The 6400 was a really great product, FOR SUN microsystems. Why did cray think they could sell a database server better than IBM or sun could?

It's not that they didn't have a pot to piss in, it's that they tried to piss in 25 pots all at once. Lo and behold, they got their feet wet. Shock and amazement.

It's too bad that sgi didn't use the cray division very well. They had a really good opportunity to blend some amazing technologies. Cray had the old, wool-suit engineers who had tackled hard problems for a long-long time. SGI had the wild breed of california technologist from the early RISC days. The sum of the parts was huge. Alas the combined whole was a disaster.

Cray today looks like they are doing cool things. xt3 is pretty neat. I'm curious to see how they combine it with the x1 vector stuff. Hopefully their three current product lines can share a lot of technology in future revisions. They just need to avoid getting too big for their britches. The supercomputing market is just not big enough to support anything too massive. Especially when you have to go head-to-head with IBM.
Re:cray by joib · 2004-10-26 04:53 · Score: 1

Cray today looks like they are doing cool things. xt3 is pretty neat.

XT3 certainly is a very interesting architecture. However, I do wonder how it will succeed economically. It's trying to squeece in on a pretty crowded market, with both clusters on the low end and blue gene very shortly providing superior density at very competetive prices.

I'm curious to see how they combine it with the x1 vector stuff.

Crays roadmap indicates that they plan to share tech as much as possible between the X1 and the XT3 families, to the point that sometimes in the 2010 timeframe the only difference will be the cpu boards. Everything else will be the same, interconnect tech, software, etc. Even going so far that you could have different cpu boards in the same system!

Hopefully their three current product lines can share a lot of technology in future revisions.

Do you mean XD1 as the third line? Personally, I think XD1 is somewhat of the black sheep. As I see some significant overlap between XD1 and the XT3, perhaps they simply bought Octigabay to avoid them meddling in their future XT3 market?

At the moment, I think Crays trump card is the X1 family. Most users have apparently been very pleased with the performance of the systems on real world codes. Also, the only competitor in the vector business is NEC, so they can probably sell X1:s with higher margins than the XT3. Soon they'll have their X1E ready, with 3 times the compute density compared to the X1. That is, the X1E will be within a factor of 3 of the compute density of the Blue Gene! The NEC SX-8 as such seems like a very competetive cpu to the X1E cpu, but NEC has an appalingly low density of only 8 cpu:s per cabinet whereas Cray crams 128 in a single X1E cabinet.
Re:cray by Anonymous Coward · 2004-10-26 05:04 · Score: 0

Also rumor has it certain people in SGI wanted to get into politics and used Cray as leverage.

What is definitely true is after the merger alot of Cray resources were sold off to companies like Celestica. So much for custom packaging plants and the like!

SGI also built a new building for making Origin systems even as it sold off and then leased back the offices.
Re:cray by flaming-opus · 2004-10-26 05:08 · Score: 1

Octiga bay is certaintly an odd buy, but it doesn't seem to get in the way of the big guns. It really is more of a linux networx sort of product, with a similar level of sophistication. They can keep the division largely seperate and be happy, I would think. The cray name adds credibility to the product, and it addresses a market segment that isn't served by the current cray offerings. Obviously the institutional investors agreed with you, and dropped cray stock 20% after the acquisition. I definately agree that it's a bit of a black sheep, but maybe that's okay.

I think the X1 is an amazing machine. However I don't think it's true that X1 is only competing with NEC. Most big codes have been ported from vector archs to MPP or cluster code. If you read the comparisons Oak Ridge did, they are comparing X1 to altix and IBM p-series. The X1 does very well compared to scalar systems, but they also have to compete on cost. They need to keep up a critical mass of sales to pay for the custom engineering.

They also need to do something about the terrible, terrible scalar performance of the MIPS-derived X1 cpus. While it would be cool to have a mix-match system with some vector boards and some scalar boards (presumably the vector boards have multiple parallel interconnect networks), I'd rather see an X1 with an opteron as the scalar processor. (Not that this seems reasonable, as the scalars in the X1 can access the vector registers in 3 cycles, or something like that. totally impossible even on hypertransport)
Re:cray by Anonymous Coward · 2004-10-26 10:32 · Score: 0

If only HP would keep the Alpha on??? Sorry, gone! AMD aquired the Alpha hypertransport technology from Compaq after Compaq acquired DEC (AMD put an updated version of hypertransport into the Opteron). Besides being a very fast bus, hypertransport also allows up to 8 processors to be stuck together without "glue" chips --hence the name Opteron. HP then merged with Compaq, and sold the remaining parts of Alpha to Intel (a while ago now). Itanium3 is what the old Alpha team has been working on.
Re:cray by Anonymous Coward · 2004-10-26 16:18 · Score: 0

Also, FYI, Cray is one of those companies which will never totally go "belly up" anyway. Given the sensitivity of the work which they did, their support databases alone are full of sensitive and/or classified information. Should the company cease trading, it would be acquired by a shelf company whose sole function is to ensure this data would remain private. That's been the fate of almost all of the now-defunct supercomputer and high-end graphics companies who formerly supplied the defence and intelligence market.

Not true. The SPR database is accessible to many employees who do not have clearances and may not be US citizens.

How big it is by commodoresloat · 2004-10-25 19:45 · Score: 2, Informative

from TFA -

Dimensions (cabinet):

H 80.50 in. (2045 mm) x W 22.50 in. (572 mm) x D 56.75 in. (1441 mm)

Sorry to reply twice but I forgot this detail.

Any future plans? by Anonymous Coward · 2004-10-25 19:45 · Score: 0

I'm impressed by the technological achievements of Cray and what they have contributed to the industry, but more importantly, when is the Extreme Edition coming out?

You don't have to begin to imagine by commodoresloat · 2004-10-25 19:47 · Score: 3, Informative

You could just read on the spec page: Power: 14.8 kVA (14.5 kW) per cabinet. Circuit Requirement: 80 AMP at 200/208 VAC (3 Phase & Ground), 63 AMP at 400 VAC (3 Phase, Neutral & Ground) Cooling Requirement: Air Cooled, Air Flow: 3000 cfm (1.41 m3/s) Intake: bottom, Exhaust: top.

Re:You don't have to begin to imagine by fbform · 2004-10-25 21:13 · Score: 5, Interesting

More interesting is this spec:

Acoustical Noise Level: 75 dBa at 3.3 ft (1.0 m)

For comparison, that's roughly the same as an average vacuum cleaner when you're operating it, or maybe a good-sized pickup truck passing you in the next lane.

And remember, this value is *per cabinet*. You have to do a weighted sum over all the cabinets in an installation to get a true dB level. I wonder whether the maintenance people will have to use noise-level exposure limits for this baby.

And here I was, complaining about the quiet whine of my PC's fan.

--
Time flies like an arrow. Fruit flies like a banana.
Re:You don't have to begin to imagine by pchan- · 2004-10-25 21:26 · Score: 4, Interesting

Power: 14.8 kVA (14.5 kW) per cabinet.

that's amazing. how did the cray guys get a kilovolt-ampere that is not equal to a kilowatt? just goes to show you the power of fast interconnects.
Re:You don't have to begin to imagine by Anonymous Coward · 2004-10-25 21:36 · Score: 0

I believe they are used differently, W or kW is usually normal usage, where as VA or kVA i used for the startup power.

Electric motors have much higher VA rating than W.
Re:You don't have to begin to imagine by Anonymous Coward · 2004-10-25 21:46 · Score: 1, Informative

it is quite possible for kVA != kW, it all depends on the relative phase of the current and the voltage...

14.5kW would just be the real component of power, i.e. heat dissipated
this leaves - using pythagoras' theorem - an imaginary component of power of 2.96kVA
Re:You don't have to begin to imagine by wronskyMan · 2004-10-25 21:46 · Score: 5, Informative

Disclaimer: IANACEBIATAPEC (I Am Not A Cray Engineer But I Am Taking A Power Engineering Course)
It's fairly common to get a KVA !=KW.
Overall power used by a load is expressed as S=P+jQ, where P is the "real" power and Q is the reactive power (capacitive/inductive from motors, fluorescent lamp ballasts, etc).

While the "units" of S, P, and Q are power=voltage*current, S is generally expressed in VA, P in W, and Q in VAR(volt-ampere reactive) to differentiate the variables. Because the magnitude of S=sqrt(P^2+Q^2), S will always be greater than or equal to P (in this case, 14.8kVA=sqrt((14.5kW)^2+(+-2.965kVAR)^2)

--
--- You shall know the truth, and the truth shall make you mad- Neal (not Cowboy) Boortz
Re:You don't have to begin to imagine by Anonymous Coward · 2004-10-25 22:12 · Score: 0

You pay the power company for kVA
you use kW.
So the GOAL is to make kva=kw.

Easy with a light bulb, difficult with a power supply or motor under varying load.
Re:You don't have to begin to imagine by ch-chuck · 2004-10-25 22:44 · Score: 1

you can even buy power factor correction power supplies if your Q gets too big, as the power company does charge for imaginary power.

--
try { do() || do_not(); } catch (JediException err) { yoda(err); }
Re:You don't have to begin to imagine by ppz003 · 2004-10-26 00:33 · Score: 1

OSHA requires hearing protection for those continuously exposed to 85 dBa and above. So, not quite.
Re:You don't have to begin to imagine by ZigMonty · 2004-10-26 01:48 · Score: 1

Look up the term power factor. This system has a power factor of 14.5/14.8 = 0.98, which is surprisingly good for a computer, IIRC.
Re:You don't have to begin to imagine by HawkingMattress · 2004-10-26 03:02 · Score: 1

Not that noisy, we have two HP ML370 here that makes at least as much noise as your typical vacuum cleaner (didn't measure it, but still...). (They have debian running , and hp fan control stuff will only work on redhat or suse, so they're always blowing at full capacity for now. And there are around 15 fans in there). Still, they're just very high availibility pcs, so it wouldn't have shocked me to hear that the crays make as much noise as a jet engine...
Re:You don't have to begin to imagine by Anonymous Coward · 2004-10-26 04:07 · Score: 0

Holy shit. That's some power requirement.

My Lodge and Shiply lathe takes just a little bit more juice (running, not startup)--that rack takes almost 20 horsepower continiously. I wouldn't want to have to pay th bill for that, yikes. Better get a freakin nuke plant to run 30,000 cpus.

I'm curious, though why they chose 400VAC, when most all instutions have 480v 3phase, and 277 volt lighting. Dosen't make that much sense to me, requiring another transformer and all.

Opterons and PowerPC together by Henriok · 2004-10-25 19:48 · Score: 5, Interesting

It seems that the XT-3 not only use Opteron processors but they also use PowerPC 440 co-processors from IBM to off load inter-processor communication from the main computing CPUs. Quite an interessting set up.

The XT-3's biggest comptetitor in this segment must be the BlueGene/L type super computer made by IBM. The processors in Blue Gene/L is a custom built dual core version of the PowerPC 440 with built in high speed interconnects.

Just like IBM have a finger in all the future game consoles, they seem to have a finger in several of the next generation super computers also. Nice going IBM.

--

- Henrik

- when the Shadows descend -

Re:Opterons and PowerPC together by Shinobi · 2004-10-25 20:03 · Score: 2, Informative

No. The biggest competitor to the XT3 will be machines like the NEC SX-8, their own X1 family or the IBM p690's. They are all shared memory systems, while the Blue Gene family is not. And therein lies a whole world of difference.
Re:Opterons and PowerPC together by evilviper · 2004-10-25 20:50 · Score: 2, Insightful

Just like IBM have a finger in all the future game consoles, they seem to have a finger in several of the next generation super computers also. Nice going IBM.

It's not that they're the best thing since sliced bread, it's mainly that all their competition went down the chute for one reason or another.

HP/Compaq/DEC was the king of supercomputers. Now they're only supporting their formerly glorious products, with practically nothing new comming to replace it.

Sun seems to really be sitting on their ass.

Intel was trying, but screwed the pooch with Itanium/Itanic.

SGI was a competitor, but they've just faded out.

Motorola could compete if they put some effort into it, but they've been out of it for some time.

etc.

--
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Re:Opterons and PowerPC together by pchan- · 2004-10-25 21:42 · Score: 2, Interesting

let's see what you're missing:

* first, sgi still makes and sells supercomputers, they are far from faded. they also own cray (or did).
* tandem, bought by compaq, we all know what happened there.
* hp sells a superdome once in a while. but nobody seems excited about their itanic systems.
* sun, rotting with their out of date cpus.
* fujitsu is doing well in the supercomputer market.
* nec is also successful.
* ibm, of course.

and you mentioned motorola? you're joking, i hope.

the largest purchasers of supercomputers in the world - national labs and the nsa, like to buy american hardware. they've always had a hand in keeping the industry afloat. notice that the big labs tend to round-robbin their supercomputer vendors so that they buy a machine from each vendor.
Re:Opterons and PowerPC together by Anonymous Coward · 2004-10-25 22:25 · Score: 0

Only the XT3 isn't a shared memory system. It is a cluster.
Want proof? The Opteron can't address more than 2TB of physical memory.

A big competitor would also be the Altix, and clusters of Altixes. The I2 CPU is nearly twice as powerful as the Opteron on FP intensive HPC codes.

Of course, the POWER5 is more powerful again, but so far no POWER5 supercomputers (no doubt there will be soon).
Re:Opterons and PowerPC together by Anonymous Coward · 2004-10-25 22:29 · Score: 0

4096 Itanium's giving a linpack score of 19940 (4.9 per CPU) versus 2560 Opterons giving 8061 (3.1 per CPU) say you don't know what you're talking about.

The I2 is many years old and it will probably still be ahead of the Opteron by the time the I3 is introduced.

Now go back to bragging about your FPS.
Re:Opterons and PowerPC together by Anonymous Coward · 2004-10-25 23:22 · Score: 0

Compare the cost of the Itanium to the Opteron.

Now go back to ignoring the realities of price/performance.
Re:Opterons and PowerPC together by Anonymous Coward · 2004-10-25 23:46 · Score: 0

I'm sure SGI is much happier to have big caches offload some pressure from its interconnect, afterall it is looking to put 2048 CPUs on a single system soon.

Oh, and they're already using upwards of 8TB on their 512 CPU systems, which is already quadruple what Opterons can even physically address.

And the better RAS features of the Itanium 2 are probably a good idea for SGI's sort of systems.

Now you go back to your FPS.
Re:Opterons and PowerPC together by Dusabre · 2004-10-26 00:14 · Score: 1

etc.

Sounds like an opening for...

Google!

Now that I think about it... they have massive experience with huge data systems!
Re:Opterons and PowerPC together by flaming-opus · 2004-10-26 02:28 · Score: 1

Basically this is the same off-load as blue-gene. One processor with a MPI off-load engine. In the case of blue gene, the main cpu is another 440, while xt3 uses a much stronger opteron. (of course the IBM solution is less expensive, and much denser).

The real difference in this system is the high bandwidth shared memory. Blue Gene has hardware support for shared memory, but the software appears to be strictly MPI based. (at least in the first revision, and according to what I've read, this may be preliminary data only).
Re:Opterons and PowerPC together by Wesley+Felter · 2004-10-26 03:30 · Score: 1

Neither the XT3 nor BlueGene/L have shared memory. The XT3 supports Cray's "SHMEM" API, but that's not really shared memory as we know it.
Re:Opterons and PowerPC together by Anonymous Coward · 2004-10-26 04:13 · Score: 0

Sun was one of the 3 companies chosen to be in the DARPA supercomputing project, along with IBM and Cray...not exactly 'sitting on their ass'.
Face it, if it isn't bad news about Sun, Slashdot rarely posts it..hence...you don't know about it. But of course, conventional Slashdot wisdom gets moderated as "Insightful" because of the huge about of sheep-like groupthink that goes on here.
Re:Opterons and PowerPC together by Anonymous Coward · 2004-10-26 12:51 · Score: 0

The p690 is a simple multiprocessor system, which for most applications is optimal at about 16-32 processors. A business system basically, running database apps.

Why do you think that competes with something scalable to 30,000 processors?
Re:Opterons and PowerPC together by Bert64 · 2004-10-29 05:51 · Score: 1

And according to the last benchmarks i saw, the Alpha EV68 and EV7 chips, despite no development for years, beats the itanic at the same clockrate..
Ofcourse, HP are trying to de-emphasize the Alpha and won't be publishing the benchmarks showing what it can truly do..
As for huge multiprocessor systems, the interconnect has a HUGE impact on the overall performance, especially linpack benchmarks

--
http://spamdecoy.net - free throwaway anonymous email - avoid spam!

STD Linux Cluster? by the_2nd_coming · 2004-10-25 19:49 · Score: 0

I certainly do not want to get near that thing, STDs will rot your johnson off.

--

I am the Alpha and the Omega-3

Wow! by pyro+jackelope · 2004-10-25 19:54 · Score: 1

$2 million? Oh, sure...I'll take like...5 of those. They'll go good with the particle accellerator and mock hubble telescope I have in my backyard =). Anyways, this sounds really cool. A person could crunch so many numbers with one of those, or perhaps play a game of solitaire?

--
28:06:42:12 - That is when the world will end...

Re:Wow! by Maniac-X · 2004-10-25 22:08 · Score: 1

I wonder how much the 30,000-odd CPU model would cost. Imagine the solitaire on that!

--
(A)bort, (R)etry, (I)gnore?_
Re:Wow! by Anonymous Coward · 2004-10-26 00:47 · Score: 0

It runs SuSE Linux, so why not play UT 2004 or Doom 3?

(Okay, it runs Linux only on the "Login Elements" for user interfacing; the actual "Processor Elements" run a lean mean Catamount microkernel.)

Sic transit gloria mundis by oakad · 2004-10-25 19:54 · Score: 2, Insightful

It seems like Cray is not capable of sustaining its heritage. Buying cheap AMD processor and connecting them with customized HT interconnect is not enough to build a machine capable of record-breaking single-task performance, old Crays exhibited. When one could be sure with Cray XMP that he has the best machine money can buy (with outstanding scalar and vector abilities), new Cray is just another loosely-coupled AMD cluster. Thanks god it's not a NEC clone (at least).

Re:Sic transit gloria mundis by Anonymous Coward · 2004-10-25 20:49 · Score: 0

the process seems to have started already:
Cray SX-6 System
Re:Sic transit gloria mundis by Anonymous Coward · 2004-10-26 00:56 · Score: 2, Informative

It's not a customized HT interconnect. There's a dedicated SeaStar router chip that connects via HT to the uniprocessor Opteron + RAM node, but the actual fabric connecting the SeaStars is proprietary (each SeaStar connecting to six others via 7.6 GB/s interconnects, forming a 3D grid fabric expandable to 30K+ nodes).

That's why they use mere 100-series Opterons: they need only one HT link per CPU. Because the whole is not based on HT interconnects.

Really, loosely-coupled cluster my ass. This machine *is* capable of record-breaking single-task performance. Read the product pages again.
Re:Sic transit gloria mundis by Durinia · 2004-10-26 02:21 · Score: 1

Cray does have its own fully custom system still, its called the X1 - a highly scalable Vector machine (thats far from a NEC clone...)

I'd link it, but the site is down...

yum!! by commodoresloat · 2004-10-25 19:55 · Score: 1

From the spec page:

Operating System:

UNICOS/lc--Components include SUSE(TM) LINUX(TM), Cray Catamount Microkernel, CRMS and SMW software

I remember installing SuSE linux long ago on a ppc. The first thing I did once I got X running was fire up the gimp and doctor their logo so it said "Welcome to DuDE Linux...."

To answer your question, it looks like they've patched SuSE to run on the Cray Catamount Microkernel. Since there's no way in hell I'm going to buy one of these for my modest word processing and web browsing needs, I'm too lazy to look up what that actually is, of course.

Anyway, I run OS X these days. When I need Cray computing power, I don't need to spend no $2 million. I just fire up Desktop Cray and go to work!

The first test of the new Cray by teamhasnoi · 2004-10-25 20:02 · Score: 3, Funny

they simulated a woman who posts to Slashdot and is waiting for her Centris running PearPC on Debian to boot OS X.

Strangely, it took roughly a week. The second test was a simulation of the moderation results of this post.

It received a +5 Funny, which puzzled researchers, as it is currently modded -1 Offtopic.

Damn you Schroedinger!

MP performance overhead by Durzel · 2004-10-25 20:03 · Score: 1

I was under the seemingly somewhat misguided assumption that multi-processor systems incurred a non-trivial performance overhead from their application - i.e. a dual processor system doesn't technically perform at 200% the speed of the equivalent single because of the inherent overhead in the MP architecture and implementation. From memory generally speaking it's more like 90% (or even less).

Wouldn't this overhead rise exponentially as you added more processors to the point where - unless the machine had a compartively small form factor (vs an equivalent number of Blade servers for example) - it would become a Law of diminishing returns?

Re:MP performance overhead by Anonymous Coward · 2004-10-25 20:35 · Score: 1, Informative

It doesnt work that way because the applications this thing will be running will be highly specialised for the architecture.

The algorithm will decompose the workspace so as each node can compute its own slice of the problem almost independently.

The is only a slight communicatiosn overhead depending on what application is run. This is also offset a bit by the supposedly fast interconnect hardware.
Re:MP performance overhead by Big+Mark · 2004-10-25 20:39 · Score: 3, Informative

If Crays were built the same was as desktop dual-proc machines, then yes, the multi CPU overhead would cripple it. Fortunately, it's designed completely differently - e.g. they use PowerPC chips to handle almost all of the inter-processor communication.

You can't really compare something that can hold thousands of CPUs to something powered by Abit that can hold two, anyway. It's like comparing apples and a strange bug thing with tentacles.

Nuclear Simulations by dynamic_cast · 2004-10-25 20:09 · Score: 1

The point is that if you simulate the effects of age on a warhead you don't have to do any realworld testing. It does no good to have a nuclear arsenal that is not capable of being used.

Re:Nuclear Simulations by Jacer · 2004-10-25 21:15 · Score: 0

I'm of a mind that it does no good to have nuclear arsenal.

--
--fetch daddy's blue fright wig, i must be handsome when i release my rage
Re:Nuclear Simulations by October_30th · 2004-10-25 21:27 · Score: 2, Insightful

you don't have to do any realworld testing
I admire your positive outlook on the prospects of simulations, but as an experimentalist, I find this "soon we won't need experiments at all" (see Rev. Mod. Phys. 64, 1045-1097 (1992), for instance) attitude very dangerous. Simulations and models, even at the first principles level, should never be trusted implicitly. They only sure way to tell how nature works is via experimentation.
I can sort of understand simulating nuclear explosions, but simulating the aging process of a warhead doesn't make that much sense to me - unless the simulations are accompanied by direct observation of the (accelerated) aging of a warhead.

--
The owls are not what they seem
Re:Nuclear Simulations by klorentzj · 2004-10-25 23:07 · Score: 1

The reason for simulating nuclear reactions instead of directly observing the effects is because of things like the nuclear test ban treaty.
Re:Nuclear Simulations by October_30th · 2004-10-26 01:11 · Score: 1

Are you talking about nuclear reactions as in "fission/fusion explosion" or as in "radiation induced damage to surrounding materials". Both explosions and the aging of warhead can be simulated. Both processes involve nuclear reactions and, as far as I know, only the former is subject to the nuclear test ban. I don't see how investigating radiation damage to materials, for instance, would constitute a breach of the treaty.

--
The owls are not what they seem
Re:Nuclear Simulations by klorentzj · 2004-10-26 06:28 · Score: 1

I guess I was talking about fission/fusion reactions, although I have to say that I hadn't really considered radiation damage.

The extent of my knowledge on this is that I spent a summer working on supercomputing at Los Alamos, and this was the explanation I heard for why simulating nuclear explosions was necessary.

Not having a security clearance, I didn't get any more details than that. But, I have to agree with the original post. At least in terms of how the research grants are justified, nuclear simulation (due to restrictions from the test ban treaty) and weather/climate modeling are two major applications for supercomputing.

I didn't pursue the field at the time, because I wasn't really interested in either of those applications. So, I would be very interested to hear more about other applications that you know of for supercomputing. (I'm talking about high-end massively parallel number-crunching...I would put things like computational genetics in a different category, because they are really more of a database problem)

I'd also imagine that there are applications that are not tractable now, but will be when the horsepower is increased. What are these applications of tomorrow?
Re:Nuclear Simulations by dynamic_cast · 2004-10-26 13:48 · Score: 1

Well, the idea is you don't have to do any testing...

Personally I think we oughta pop one off every fourth of July or so. Only way to be sure.

newfangled buzz. by colaboy · 2004-10-25 20:12 · Score: 1

Yes, all that mumbo jumbo about CPUs and interconnects is fine, but what we really want to know is how fast it can run Mac OS X under PearPC... Can anyone tell us an XBench score?

Re:newfangled buzz. by adzoox · 2004-10-25 21:36 · Score: 2, Interesting

Ha - well I'm sure the guy behind CherryOS will have a press release that it runs The Mac OS at 30 Terahertz.

--
Yell & scream & rant & rave... it's no use... you need a shaaaave ~ Bugs Bunny
Re:newfangled buzz. by MarcQuadra · 2004-10-26 12:36 · Score: 1

This is ging to become the computing equivalent to 'about the size of a volkswagen beetle' isn't it?

I can just hear it:

NEW SUPERCOMPUTER RUNS OS X AT 2.7x NATIVE SPEED UNDER PEARPC!!!

--
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails

Leather seats? by gspr · 2004-10-25 20:16 · Score: 1

Do they have leather seats for the operators like the 1980s models did?

Anyone know the IP? by Lord+Kano · 2004-10-25 20:24 · Score: 1

My Distributed.net stats could use the boost.

LK

--
"Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano

AMD gets about... by BrookHarty · 2004-10-25 20:25 · Score: 2

So 96 processors, AMD gets about 144K per PE node at 1500 per cpu, or does Cray get a discount?

Also, a 30,000 cpu complex, AMD must be making a tidy sum.

Re:AMD gets about... by Big+Mark · 2004-10-25 20:42 · Score: 1

I highly doubt Cray are paying retail prices for these CPUs. I'd imagine they buy them wholesale from AMD directly, and probably a further bulk discount on top of that.
Re:AMD gets about... by mikera · 2004-10-25 20:51 · Score: 1

AMD may even be gving Cray an extra discount/kickback for the publicity value - not uncommon for this kind of cutting edge stuff.

My old company managed to get some seriously expensive enterprise software for just 10% of the retail price because we were able to convince them that having us as a client would be a publicity coup for them....
Re:AMD gets about... by Anonymous Coward · 2004-10-26 01:20 · Score: 0

Two corrections:

1) A PE in XT3 parlance is just one Opteron plus RAM. They have deliberately made the nodes uniprocessor, to ensure bandwidth and to simplify programming for it.

A node also includes the SeaStar chip that is the actual interconnect (6-way) junction. This has one HT link for the Opty.

2) Due to this uniprocessor nature, they use only 100 series Opterons (with have just 1 HT link enabled). These are cheaper than the 200 or 800 series chips, and retail well under a grand apiece.

But agreed, AMD must be making a tidy sum, although the PR could be even more important. I wonder why they haven't made a big song and dance about this already -- corporate servers is a crucial target market for them, and they still need all the street credibility they can collect...

It's not like many of these machines will be built, as it was co-developed by Cray and the buyer. More probably, this existing machine will be expanded into 30K+ nodes, and that's about it.

But it looks like AMD didn't have to pimp their warez for this very hard -- the XT3 product pages explain how they wanted x86 code compatibility -- so they are probably getting a nice profit margin out of every chip. Not like Cray can bargain much a this point when delivery is starting... and not like the XT3 buyers really care about a couple hundred more or less per CPU, when a 200-processor machine costs $2 million already...

Not even trying... by F'Nok · 2004-10-25 20:26 · Score: 1, Funny

Using AMD's Opteron processor, it scales to a total of 30,580 CPUs.

They must be in a rush to market... Clearly the target was to have 32,768 CPU's.

I mean, what the hell am I going to do with only 30,580 CPU's?!

Re:Not even trying... by jimicus · 2004-10-25 20:59 · Score: 1

Run Duke Nukem 3D on Longhorn?
Re:Not even trying... by LiquidCoooled · 2004-10-25 22:03 · Score: 1

I think you would have a hard time running Minesweeper on LongHorn with "only" 30,580 cpus!

Dont even consider doing 3d work until you get over 65535 processors.

--
liqbase :: faster than paper
Re:Not even trying... by Zzootnik · 2004-10-26 01:01 · Score: 0

'640 processors is all the CPUs anybody would ever need on a computer.'

--Gill Bates

--
Sig currently under construction. Mind the gap....
Re:Not even trying... by Anonymous Coward · 2004-10-26 05:20 · Score: 0

die

just ask Hank Dietz! by Anonymous Coward · 2004-10-25 20:38 · Score: 0

http://www.aggregate.org/

Intersting note by floydman · 2004-10-25 20:46 · Score: 2, Interesting

from their Tech.sheet they are using the Luster file system

This is the first time i see a shipped linux with this file system. Now the intersting part is that lusterfs is made for linux clusters, but this monster is not a cluster... any body can shed some light?

--
The lunatic is in my head

Re:Intersting note by flaming-opus · 2004-10-26 03:05 · Score: 1

It's not a cluster.....

well, sort of.

There are thousands of compute nodes, all of which get i/o services from dozens, or hundreds of i/o nodes. These i/o nodes run linux, several instances of linux. Basically the i/o nodes ARE a cluster, though not a compute cluster, and not necessarily a symmetric cluster. The i/o nodes run lustre in very much the same way that a cluster system would (though they can take advantage of hardware features not present on commodity clusters).

The real difference in this system is that nodes are not peers, as they are in commodity clustes. Each node has only one function, and the software is tuned to provide that one function very effectively.
Re:Intersting note by Anonymous Coward · 2004-10-26 12:27 · Score: 0

Yes, I can.

The Lustre service side runs under Linux, in the service partition. Much is fairly normal for it.

The compute node OS, though, is an evolution of that used on ASCI Red. It's known as catamount --
Sorry folks, it's not Linux in the compute section, just the service section.

For the compute partition, just like Red, catamount has no provision at all for I/O. Worse, we could not steal the IO architecture or infrastructure from Red. Red's IO architecture heavily leverages an SSI service section and, no, Red Storm does not have that advantage.

The solution we chose may be found at http://sourceforge.net/projects/libsysio and is an implementation of a POSIX-like VFS in user space. Underneath that, the Lustre client code was hiked out of Linux, altered to run w/o kernel services and a glue layer added so that it may be driven by the sysio library mentioned above.

$2 million for a 200 processor system ? by Professeur+Shadoko · 2004-10-25 20:59 · Score: 1

I can't find this price anywhere.
And it seems _really_ low.
I would expect a price at least twice higher.

Ok, $2 million is starting price, but on Cray's website they say the configuration can be as "small" as 96 CPUs.

So it's maybe $2 million for 96 CPUs.
(Still fairly cheap for a Cray, if you ask me)

Re:$2 million for a 200 processor system ? by Anonymous Coward · 2004-10-25 21:59 · Score: 0

Its cheap becuase they are using almost off the shelf opertons and ppcs cores.

Previous expensive Crays had custom made vector processors.

Mirror Links by Agret · 2004-10-25 21:00 · Score: 1

In case original page gets slashdotted: http://www.cray.com.nyud.net:8090/products/xt3/ind ex.html

--
Have you metaroderated recently?

Re:Mirror Links by Zoolander · 2004-10-26 01:17 · Score: 0

Cray can't be slashdotted, because they make teh m0st l33t b0x0rz!!!111!!!oneoneone
At least that's how things should be...

--
Meep.

Re:My new dream toy by Guppy06 · 2004-10-25 21:06 · Score: 3, Funny

Maybe if you included promises of free iPods...

But what it doesn't have is... by Anonymous Coward · 2004-10-25 21:20 · Score: 0

NUMAlink IV.

If you want an even faster, cheaper machine, buy an Altix: http://www.sgi.com/products/servers/altix/

A happy customer. (Seriously)

Re:fucking death labs by Zog+The+Undeniable · 2004-10-25 21:38 · Score: 1

Ah....for tactical nuclear weapons simulation, no doubt. And we thought General MacArthur was bonkers.

--
When I am king, you will be first against the wall.

The math for a comparable Xserve system by adzoox · 2004-10-25 21:50 · Score: 0, Redundant

If the math is right for a 200 CPU system for $2 million ...

That would mean $10,000 per CPU

That would mean 100 Apple XServes

But for the comparison, let's just go with 200 Apple X Serves giving 400 CPUs total

Let's throw in 100 Apple XServe RAIDs fully loaded

1 Xserve G5 Dual 2.0Ghz = $3000

Multiply by 200 = $600,000

For cost savings we'll go with (20) 5.8 terrabyte RAID for $13,000 each

[Instead of 100 1 Terrabyte units for $6000 each]

=$260,000

Total = $860,000 for total system

Throw in approximately $120,000 for setup and integration (as comes with Cray purchases)

NOW double that - what a value for 6 teraflops calculation [extrapolated from Virginia Tech SuperComputer scores]

What a value!! Plus you have money left over to pay the tax!

--
Yell & scream & rant & rave... it's no use... you need a shaaaave ~ Bugs Bunny

Re:The math for a comparable Xserve system by joib · 2004-10-25 22:10 · Score: 4, Insightful

What a value!!

That is, until you throw a tightly coupled problem at it and the Cray is 10 times faster because it has much better internode bandwidth and lower latency.

And, you forgot to count the cost of the InfiniBand interconnect that the VT cluster used? That's a couple grand per node.

Bottom line, apples and oranges. If your applications is easily parallelizable (i.e. doesn't require much communication between the nodes) you'd be stupid to piss away your money on a "real" supercomputer instead of a cluster. And vice versa.
Re:The math for a comparable Xserve system by Anonymous Coward · 2004-10-25 22:16 · Score: 0

The most recent info:

Virginia Tech G5 supercomputer achieves 12.25 teraflops
October 25 - 23:34 EDT Virginia Tech today announced that its rebuilt "System X" supercomputer cluster, which consists of 1,100 Xserve G5s, is now operating at 12.25 teraflops--an increase of almost two teraflops over the original system's 10.28 teraflops. "Virginia Tech will learn of its new ranking when the list is unveiled in November of this year at SuperComputing 2004 in Pittsburgh," said Srinidhi Varadarajan, the lead designer of the system. "We expect to do well." Last year, the university's system ranked third among the world's 500 fastest supercomputers.
Re:The math for a comparable Xserve system by selderrr · 2004-10-26 01:23 · Score: 1

And vice versa

If your application is not parallelizable, the supercomputer pisses away on you ?

--
When will I end this grieving ? When will my future begin ?
Re:The math for a comparable Xserve system by joib · 2004-10-26 01:49 · Score: 1

If your application is not parallelizable, the supercomputer pisses away on you ?

Only in Soviet Russia.
Re:The math for a comparable Xserve system by SuperQ · 2004-10-26 02:39 · Score: 2, Informative

You're leaving out a lot of stuff necessary to make a cluster:

#1 RAM: $3000 for the G5 cluster node includes 512mb ram. Most places demand atleast 2gb ram per CPU, we require 3GB ram per CPU in all new system purchases. This brings the node price (from apple.com) to $6500
200x $6500 = $1,300,000

#2 Racks and power: Each rack can hold about 32 machines (without getting way to hot/dense) for 200 nodes, this would be about 7 racks.
7x $1200 = $8400

#3 Interconnect: No HPC system is usefull without an interconnect. An 80 node myrinet system was $250,000, so at $3125/node you're looking at:
200x $3000 (estimate) = $600,000

#4 Networking: you need a network switch and cabling to connect all the nodes... gige is a must these days. Let's say we go cheap with HP ProCurve 2848 Layer2 managed for $3300 each we need 7 of those, one for each rack cabinet.. with trunking we can get 4gb back to a central switch. not too bad. Say we add $10/cable for pre-made patch cables, (length averaged) that's about $2250 in cables.
7x $3300 + $2200 = $25,300

#5 Disk: You quoted a bunch of XservRaid's without any kind of apple care.. with IDE raid.. I'm not going without some kind of support on it. Oh wait.. 1 file server is NOT enough to handle 200 nodes of HPC.. and apple doesn't have a clustered filesystem. You're going to have to go with Linux/Intel with RedHat GFS for that one (yes, there are other options, but I know GFS)
Say we do 4 XserveRaid's with applecare:
4x $16,000 = $64,000
We also need for dual whatever intel machines: (i'll be nice and include F-C cards in the price)
4x $3000 = $12,000
We also need a F-C switch to link all the nodes:
SanBox 8 port $5200 and 8x SFP modules $750 = $11,200
I'll pretend like we don't need GFS software support, but most places would want it. (it's another $20,000 or so, but eh.. we want cheap solution)
Disk total comes to: $87,200

Price so far: $2,020,900

And that doesn't even include setup!
Re:The math for a comparable Xserve system by Anonymous Coward · 2004-10-26 02:48 · Score: 0

actually he had $120,000 extra + some change left over in his figures
Re:The math for a comparable Xserve system by flaming-opus · 2004-10-26 05:15 · Score: 0

you're right that most estimates ignore labor, and discounts, and are completely unrealistic. Virginia Tech has the advantage that the Big Mac is just a technology demonstation, and doesn't actually do any real work.

Xsan, is apple's san filesystem. $1000 per node + $500 for a dual port FC card. A bargain by anyone's standard. Most big clusters, however, don't use cluster filesystems. I don't understand why not, it seems like a great idea. But they don't.
Re:The math for a comparable Xserve system by SuperQ · 2004-10-26 16:01 · Score: 1

I guess I didn't know about Apple's Clustered FS.

We have clustered filesystems on all of hour HPC systems. We have GFS on our "small" 80 node linux cluster, 3 GFS+NFS servers to the nodes.

We use GPFS on our Power4/AIX systems, and we have CXFS on our SGI Altix systems.

scary thought by nounderscores · 2004-10-25 21:58 · Score: 1

how would you accelerate the aging of a warhead?
make the plutonium pit out of plutonium mixed with plutonium decay products, layered to match the cross-section of a genuine old warhead pit?

make the explosive pannels that are supposed to all go off perfectly semmetrically out of aging, unstable, unreliable explosives?

make the wiring exclusively out of decaying cables, which have the insulation falling off?

Hmm. better dismantle and scrap these guys as soon as any one part begins to go.

All the real life testing on even slightly aged warheads was done in the 1970s, and by the french in the 1990s. They now keep the models in sillico, and don't let the warheads age past the point where the model has been shown to hold by recycling them for components.

I really don't want to be anywhere near an artificially aged warhead.

Re:scary thought by Anonymous Coward · 2004-10-25 22:41 · Score: 1, Funny

how would you accelerate the aging of a warhead?

Just give it a couple of little baby warheads to look after.

Re:My new dream toy by Anonymous Coward · 2004-10-25 22:38 · Score: 1, Insightful

Gamer's dream machine? This is a supercomputer, not a game console. I bet this thing doesn't even have a graphic card.

700kgs, 75dB and 14kW... by Alkonaut · 2004-10-25 23:09 · Score: 4, Funny

...Sadly I think that beats my Volkswagen on all three

What?? no usb? by Anonymous Coward · 2004-10-25 23:17 · Score: 1, Funny

Where do i plug my 128 usb pendrive to store
results from my computations... :)

Finally ... by Zurd3 · 2004-10-25 23:29 · Score: 2, Funny

We'll be able now to install Gentoo in just a few days !

Cool! by Dorsai65 · 2004-10-25 23:35 · Score: 1

Doom3 here I come!

--
--- Asking inconvenient questions for over 30 years...

At last by SFSouthpaw · 2004-10-25 23:47 · Score: 1

Now I can run every game under MAME @ full speed.

--
---southpaw

No, what stands in the way is price by Moraelin · 2004-10-25 23:48 · Score: 3, Interesting

The real problem that stands between scientists and them having lots of shiny toys is funding.

E.g., yeah, having a 30,000 CPU super-computer to simulate your gene model on would be nice. Forking over half a billion for it, well, it's suddenly not that nice any more.

Having one of those to simulate an electronic circuit, now that would probably rock. Again, paying half a billion for it, suddenly isn't that attractive.

The real question isn't how nice a toy you'd like to have, it's ROI. (Unless you work for the government, and just have a budget you _have_ to blow on stuff, whether you need that stuff or not.)

And in that context, you'd be surprised what you _can_ do with a lot less expensive toys.

Having Cray's custom interconnects sure is impressive, but for a lot of problems they're not even needed any more. _That_ is what killed Cray.

Most RL problems are not really the kind described as "_one_ huge indivisible data set, that you have to process in _one_ huge batch process." They're more like "we have this process with a small data set that we have to run 100,000,000 times." Most design problems or biology problems are really of that kind: run the same thing 100,000,000 times with different parameters.

And as Seti@Home or Folding@Home proved, a helluva lot of those don't really need _any_ kind of shared memory or fancy interconnects. The real ticket is noting that instead of accelerating the batch run 200 times, you could just split it into 200 smaller batches ran on 200 single-CPU machines.

The super-computer solution costs 2,000,000 just for the machine alone, while the 200 PCs solution costs 200,000 or so. I.e., 10 times cheaper. Better yet, the 200 PCs solution is also far cheaper to program. (Anyone can program a non-threaded batch app.) _And_ for that kind of a problem the 200 PCs solution would actually finish faster, since it has no contention issues whatsoever.

Again, that's what really killed Cray and the super-computers. They're techologically impressive, they're a geek's wet dream, but... for 99.9% of the problems out there they're just not worth the price any more.

--
A polar bear is a cartesian bear after a coordinate transform.

... Back in my day .... young whippersnapper by ebooher · 2004-10-25 23:52 · Score: 4, Interesting

So come on, ante up. How many remember being awed at the mere sight of old Crays back in the day? Like the Cray-3? I remember the first time I saw a Cray .... thing was in an anti-static environment. To access it, one had to pass through an airlock and be "decharged" or "depolarized" etc. Basically they some how charged the air to get rid of static electricity. Then you had this system that was running *in* liquid! Take that "Oh I'm so cool cause I have a l337 haX0r water cooled CPU" overclockers

They (Cray) were so proud of this accomplishment that the upper portion of the cabinet was some kind of plexiglass so you could see the fluid as it moved, and moved wiring and what not with it. Very surreal feeling, almost like the thing was breathing.

And what about the Cray-1? Wasn't that a true testiment to 70's *art* and sculpture? The thing looks like some kind of freaky bus station bench with it's odd red and white panels and black base. Though, I don't know if they all looked like that, maybe you could get them in other colors?

Ahh .... those were the days.

--
"Genius may shine aloof and alone, like a star, but goodness is social, and it takes two men and God to make a Brother."

Re:... Back in my day .... young whippersnapper by flaming-opus · 2004-10-26 02:31 · Score: 1

well, when the computer costs 10-15 million dollars, you can afford to spend twenty thousand on making it look really cool.

compared to ASCI-red, the system that red-storm is replacing, xt3 looks increadible. Yes it's a long row of rectangular racks, but at least they are stylish racks. Intel built asic-red in beige box style. Oh well. function over form I suppose.
Re:... Back in my day .... young whippersnapper by mrdogi · 2004-10-26 02:36 · Score: 2, Informative

Then you had this system that was running *in* liquid!
Before that was the Cray-2 (a.k.a World's most expensive aquarium")? In case anybody's interested, I believe they used Fluorinert as the liquid, as it wouldn't swell the PC boards, short anything out, or cause anything to corrode.
A note, the Cray-3 was created by Cray Computer Corporation of Colorado, whereas the Cray-1 was made by Cray Research of Wisconsin. In ~1990, Seymore wanted to start working on computers using gallium arsenide instead of silicon, since they could switch faster. Cray Research didn't want to try anything so revolutionary, so Seymore headed to Colorado with a group of people and started CCC. Unfortunately, they apparently made exactly one Cray-3, then folded.
Seymore Cray was quite the Übergeek.
Re:... Back in my day .... young whippersnapper by Anonymous Coward · 2004-10-26 07:36 · Score: 0

And what about the Cray-1? Wasn't that a true testiment to 70's *art* and sculpture? The thing looks like some kind of freaky bus station bench with it's odd red and white panels and black base. Though, I don't know if they all looked like that, maybe you could get them in other colors?
Ahh, yes, I'll buy a Cray-1, but only if you sell them in teal.
Re:... Back in my day .... young whippersnapper by ebooher · 2004-10-26 10:31 · Score: 1

Well .... it's actually pretty funny that you would mention that. After my own statement about what color the Cray-1 was I decided to run around the net like a mad hen ..... which is where I started finding pics ..... nothing like seeing a Cray-1 in Lemon Yellow. Anyway ....

From: http://www.spikynorman.dsl.pipex.com/CrayWWWStuff/ Cfaqp3.html#TOC19

Could you choose the colour of your Cray machine ? In the early days of the company yes, there was even rumour of a cowhide covered XMP delivered to a Houston oil company. As time went on colouring your computer was dropped as a customer option. Well almost - when there is that much money changing hands, if enough fuss is made, the exterior panels would revisit the paint shop. This did not apply to Els which were all black and red. Well almost - one customer which had just upgraded a pair of YMPs (one green, one blue) for a C90 and an EL did manage to get the EL painted a rather fetching sky blue colour.

As for XMP/EA, sn501 an internal contact reports "We lobbied to have it done up in denim (like the denim Jeans) & have a little red Levis tag attached. ... Management was not amused & it never happened."

One second user customer did have a bit of a surprise when their second user C90 arrived in a lurid deep rose/pink colour. The top of the C90, being a convex shape happened to sit just a couple of feet under a set of strip lights and resulted in lovely pink glow over a whole section of the machine room. Rumour has it that the colour matched the previous owners, girlfriends' nail varnish.

The Bell Labs Cray (XMP) was a wallpapered IC design.

--
"Genius may shine aloof and alone, like a star, but goodness is social, and it takes two men and God to make a Brother."

Of the customers not listed.... by hcob$ · 2004-10-26 00:12 · Score: 0

I'm sure the NSA and CIA prolly bought about 4-5 each :)

--
Cliff Claven
K.E.G. Party Chairman
Founding Leader of: Koncerned for Egalitarin Governance

Yeah, it's gotta be awful by thegnu · 2004-10-26 00:24 · Score: 2, Funny

Last time I bought a Cray super-computer, I was kicking myself for weeks about the 2 million dollars I wasted.

Next time, I'm just gonna build a beowulf cluster out of 200 overclocked AMD Barton 2500s. I shall NOT be suckered again!

--
Please stop stalking me, bro.

Re:Yeah, it's gotta be awful by minus9 · 2004-10-26 01:29 · Score: 2, Funny

I've got a bag full of thinnet coax you can have, complete with T pieces, you might have to find your own terminators though.
Re:Yeah, it's gotta be awful by charleste · 2004-10-26 02:44 · Score: 1

While at one of those jobs that I'd have to kill you if I told you what I did (even though I didn't know really), they had a nice "old" Cray on display in the lobby. They had got a Cray 2, so they were "obsoleting" the Cray. My thought: Awesome fishtank! Put a lavalamp in the middle, pull the boards and seal it up. It's already plumbed for hoses (liquid cooled). If only I had like $100,000 at the time to buy military surplus.... (sigh)

I saw a demo of one... by Frank+of+Earth · 2004-10-26 00:29 · Score: 1

... and when you turn it on, a crackly computer generated voice says, "Would you like to play a game?"

--
Live web cams

In other news... Bush declares war on.. by Frank+of+Earth · 2004-10-26 00:32 · Score: 0, Troll

.. Sandia National Labs for it's possession of WMD's or Weapons of Mass Division

--
Live web cams

Re:In other news... Bush declares war on.. by transatlantique78 · 2004-10-26 01:08 · Score: 0

Bush doesn't even know what a division is.

--
You are finite. Zathras is finite. This... is wrong tool.
Re:In other news... Bush declares war on.. by Anonymous Coward · 2004-10-26 02:19 · Score: 0

wow thats funny, really it is hilarious.

a joke about GWB's intelligence, how do you ever come up with that kind of material.

oh wait, you are an idiot (and more of a moron than he is)
Re:In other news... Bush declares war on.. by Anonymous Coward · 2004-10-26 02:27 · Score: 0

Buddy if you get this worked up every time someone cracks a joke about Bush then you'll send yourself to an early grave.
Re:In other news... Bush declares war on.. by narcc · 2004-10-26 08:16 · Score: 1

They confiscated my sliderule and calculator at the airport...called them weapons of math instruction...

--
Required reading for internet skeptics

I could be the king of.. by Frank+of+Earth · 2004-10-26 00:34 · Score: 0, Offtopic

Seti@home with one of these things!

--
Live web cams

Internode bandwidth - cheap solutions? by Lorphos · 2004-10-26 00:50 · Score: 0

The Internode Bandwidth seems to be the major selling point of this machine, what alternatives are there for supercomputing on a budget?

Is there anything better than a few gigabit ethernet cards in every box (and still affordable!)?

Re:Internode bandwidth - cheap solutions? by SuiteSisterMary · 2004-10-26 01:27 · Score: 1

'Super-computing on a budget' is kind of an oxymoron. You either pony up, or you're not super-computering; you're parallel processing on a big cluster.

--
Vintage computer games and RPG books available. Email me if you're interested.

Cray and blinkenlights by rdunnell · 2004-10-26 00:50 · Score: 1

Cray never really put much in the way of blinkenlights on their systems.

Of course, most models look like obscure modernist furniture or art exhibits, some of the older ones had windows that allowed you to see the liquid coolant, etc. But they definitely did not have a lot of flashy light thingies. Then again, do you need them? It's a Cray! You're supposed to be stunned and look at it in silent reverence. It's like telling a Mafia boss that he has a nice suit but he needs more jewelry.

Thinking Machines, well, that was different. Too bad it didn't do more, but it was covered in LEDs.

hybrid system with multiple kernels by Dink+Paisy · 2004-10-26 01:01 · Score: 3, Informative

From the documents, it looks like it runs Linux on the management nodes and Catamount on the compute nodes. The idea is you can do what you like with the general purpose nodes, but for the compute nodes, you run a lightweight operating system that has low overhead, minimal services and predictable scheduling. BlueGene/L works the same way; it runs Linux on the management nodes and a custom operating system on the compute nodes. Compute nodes likely provide scheduling for only the number of threads that run on the node, communication through MPI and some proprietary API, and basic debugging facilities. Compute nodes probably lack normal OS services like network, disk, or even a console.

--

Whoever corrects a mocker invites insult;
whoever rebukes a wicked man incurs abuse.
--Proverbs 9:7

It's probably more than one rack, though. by chipmeister · 2004-10-26 01:08 · Score: 1

Cray, and most other large system manufacurers, usually put four CPUs on a single board. Each board plugs into a backplane. I'm just guessing here, but you can probably only fit 256 or so in a single cabinet.

Too bad... by TheVidiot · 2004-10-26 01:15 · Score: 1

Too bad Doom3 can only use the first processor, leaving the other 199 for eDonkey and SETI....

But... by _undan · 2004-10-26 01:16 · Score: 1

... will it run PearPC fast enough?

Ever wonder why.... by Gentlewhisper · 2004-10-26 01:29 · Score: 1

They didn't do it using Intel Xeons? =)

--
Online backup with Mozy, sounds like Ozzie, but more!

are you sure you remember seeing the Cray 3 ? by bmajik · 2004-10-26 01:45 · Score: 2, Informative

Because, IIRC, that was the one that they were only building one of, and when the govt cancelled the order, thats when Cray Research went under.

--
My opinions are my own, and do not necessarily represent those of my employer.

Re:are you sure you remember seeing the Cray 3 ? by ebooher · 2004-10-26 02:13 · Score: 1

Well, anything is possible. It may not have been the 3 I saw. Though I seem to remember being able to see over it, and I think almost every other Cray was at least 6 foot tall. But I don't know, it could have been recessed into the floor. I was quite a bit younger at the time, obviously. I was more focused on the tour guide guy showing off the fact that electronics were in liquid. At the time I thought if it was wet, it conducted electricity, so that kind of blew my mind.

I'd love to find an old Cray, like maybe a C90. Something big enough to be retrofitted ... man talk about cool factor plus 50. If a geek can build a bar/fridge out of an old VAX just think what a Cray could do. Obviously it wouldn't be economical to actually use the thing as a computer anymore. Too expensive. But what a garage fridge the thing would make.

--
"Genius may shine aloof and alone, like a star, but goodness is social, and it takes two men and God to make a Brother."
Re:are you sure you remember seeing the Cray 3 ? by Durinia · 2004-10-26 02:25 · Score: 1

wrong company - Cray 3 was being built by "Cray Computer", a different company that Seymour started after he left the original Cray Research. He's most likely talking about the Cray 2, which even came with its own Waterfall
Re:are you sure you remember seeing the Cray 3 ? by ebooher · 2004-10-26 03:32 · Score: 2, Interesting

Looked around on the net, as well as a couple other /.'rs here, and someone posted a link here to a 2 and I found a pic of a 2 with the waterfall system that was mentioned by another person, and I must accept defeat within the loosened strands of my unraveling mind.

It was indeed a Cray-2 that I remember so vividly. Nevertheless, still an extremely exotic machine. Very much the Ferrari F40 or McLaren F1 of super computing. You've seen pics, maybe even seen one at a car show, but you know you'll never be allowed to touch one. It had as much class as an Italian sports car too.

I find myself wondering how many /. geeks it would take at what $$ amount to colocate a community Cray somewhere ...... be like going in with 100 friends to buy a Ferrari though. Who gets to keep the keys?

--
"Genius may shine aloof and alone, like a star, but goodness is social, and it takes two men and God to make a Brother."
Re:are you sure you remember seeing the Cray 3 ? by ebooher · 2004-10-26 10:38 · Score: 1

Again, kind of answered my own question here. Check out http://www.cray-cyber.org/general/start.php/ for a hosting provider in Germany that happens to have a Cray Y-MP EL online for external users with free accounts. Though, you do have to tell them what you want to use the account for and what not, so if you tell them to root kit their l33t 5up3rc0mpu73r they may not let you play with their toys.

--
"Genius may shine aloof and alone, like a star, but goodness is social, and it takes two men and God to make a Brother."

So how is this by Anonymous Coward · 2004-10-26 01:45 · Score: 0

related to an IBM XT? It's ages since I used one!

My dumb math. by JoshRoss · 2004-10-26 02:06 · Score: 1

That works out to $305,800,000.00 (before taxes) for the 30,580 proc version. What a deal! $10k per processor. We are reaching Sun(TM)* territory.

*Sun is a registered trademark of some company.

You can get to their website? by Roadkills-R-Us · 2004-10-26 02:30 · Score: 1

Obviously they aren't using one of these for their webserver. Or if they are, they need something more than a modem for their internet connection!

Re:You can get to their website? by rooijan · 2004-10-26 03:33 · Score: 1

I happened to see the story quite early - I got in, read it and got out before the sound of a million slashdotters could be heard approaching on the horizon.

--
Daar is nie 'n lepel nie

No one does trig with x87 anyway. by Ayanami+Rei · 2004-10-26 02:57 · Score: 1

You're supposed to use lookup tables, recurrence relations and interative refinement to get the precision you need when you need it.
The only time you should be using fsincos (SLOW) is when you need to build a table or populate variables accurately before a loop.

--
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON

Was it the 1a or 2 maybe? by caveat · 2004-10-26 03:05 · Score: 1

1a
2

--

Facts do not cease to exist because they are ignored. - Aldous Huxley

Active / 100% CPU usage vs. Screensaver. by Ayanami+Rei · 2004-10-26 03:06 · Score: 1

Errr...
No computer in existance runs a screen saver because the CPU usage is less than 100%. Desktop computers run screen savers to prevent a still image from burning onto your monitor or LCD (or to lock the terminal if it thinks you've left). This has everything to do with no use of mouse or keyboard, not CPU usage.

When any computer has nothing to do in a timeslice, it generally calls the HALT instruction, which puts it into a low power state until a timer interrupt or something comes along and wakes it up to do something else.

This is why CPUs that are idle generally take less power than one running Prime95 or 128 of them rendering a 3d movie.

--
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON

Sandia's data sheet on Red Storm by scoobrs · 2004-10-26 03:11 · Score: 1

Here's all the theory and architecture information you might want on Red Storm including a discussion of how Amdahl's Law is wrong. They add overhead due to communications to the equation and make some very interesting proposals about scalability. http://www.lanl.gov/orgs/ccn/salishan2003/pdf/camp .pdf

Their bottom line to the research: "A well-balanced architecture is nearly insensitive to communications overhead. By contrast a system with weak communications can lose half its power for applications where communications are important."

Conclusion: "For most large scientific and engineering applications the performance is more determined by parallel scalability and less by the speed of individual CPUs. There must be balance between processor, interconnect, and I/ O performance to achieve overall performance. To date, only a few tightly-coupled, parallel computer systems have been able to demonstrate a high level of scalability on a broad set of scientific and engineering applications."

--
-Those who would give up essential liberty to purchase temporary safety deserve neither. -Ben Franklin

Re:Sandia's data sheet on Red Storm by Mistah+Blue · 2004-10-26 04:08 · Score: 1

I think this can be applied to the CPU's themselves. My very unscientific observation has been that a 1.4GHz Pentium-M spanks a 2.8GHz P4 on one web based application I work with. I have to say I am very pleased with the 1.4GHz Pentium-M and feel no burning desire to upgrade my Latitude D-800 to a faster one. It is plenty fast for me, and I suspect it's power draw is less than the faster ones.

Also, eye for expandability... by Ayanami+Rei · 2004-10-26 03:22 · Score: 1

By "limiting" themselves to the 1xx opterons, they could move to 2xx or 8xx later on. It'd then be easy enough to connect 8 or 12-way by hooking up more interconnects per board, adding a second CPU on each board, etc. etc. by utilizing the additional HT links.
And then when the dual cores come out... hehehe.

--
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON

Wall Street by Moraelin · 2004-10-26 03:25 · Score: 2, Informative

You have to understand though that the stock market's expectations have nothing to do with whether the company is doing well or not.

Surrealistic point in case: at one point 3Com had a lower market value than the Palm daughter-company. Basically if you subtract the value of the Palm shares, the whole rest of 3Com was actually worth a _negative_ value for the stock market.

And we're talking divisions which were making a tidy profit. Yet they were apparently worth a _negative_ number.

No, it's not a joke. Roll it around a bit in your head to fully grasp how completely sad and idiotic that is. Real profits, real assets, worth a negative number of dollars. Stupid.

Or at the other end of the spectrum you have Microsoft whose stock market value is _way_ above the value of its assets. Without paying any dividends or acquiring much in the way of long term assets, people just flocked to drive the price up and make Bill Gates rich. Basically to give their money to Bill Gates and not even get a Windows CD in return.

The thing is, however, the stock market value has _nothing_ to do with a company's value or profits. The value of a share is only worth as much or as little as people want to believe it is. It is like Monopoly (the board game, not MS;) money: if tomorrow we decide that the blue bills are worth 10% more and the red bills are worth 10% less, who's to argue with that.

The _only_ reason the stock market on the whole goes up is basically because yearly people dump more money into it. Basically it goes up just because people want to believe it's going up, and put their money where their belief is.

And the way those values fluctuate, now that just has to do with hype and greed.

The stocks worth buying are those who'll make you a profit: typically meaning they'll raise in value. The stocks worth selling are those who don't.

Except with no intrinsic value it becomes a game of guessing what the other lemmings will buy (driving the price up), and what the other lemmings will sell (driving the price down.)

One thing that makes lemmings buy is the prospect of growth. Hence, hype is good. Hence, yes, shares in a cancerous tumor would sell like hot cakes and rocket sky high in price.

Hence, conversely, shares in a company which doesn't grow or otherwise cause more lemmings to buy, are not worth holding on to. Because they won't bring a profit. If Microsoft truly plateaued and didn't pay dividends either, regardless of how much profits it made at that point, its shares would plummet. Because between holding onto a share in MS that doesn't bring a profit, and investing in some startup that grows quickly, the second promises more of a ROI.

Now that's all a bit of an over-simplification.

Of course, there are other factors. Like just paying dividends to give people a reason to hold onto your shares even without massive hype and growth. (See why MS started doing that when its market explosion slowed down.) Or like fraud: "analysts" just telling lemmings what to buy, and thus drive up the price of the shared owned by the "analyst" and his/her clients. Etc.

But as a quick intro to the madness of the stock market, it will have to do.

--
A polar bear is a cartesian bear after a coordinate transform.

The SSI is configurable... by Ayanami+Rei · 2004-10-26 03:34 · Score: 1

... that is a bunch of "service nodes" will be lashed together into an SSI. These all run their modified SuSE. So if you have two I/O nodes and two login nodes and 4 Network nodes, that might be an "8-way" SUSE image. They communicate with other lashed-together service nodes with Luster. The compute nodes are part of a huge loosely coupled SSI running the microkernel... which are submitted jobs by one the SSIs running on the service nodes.

Someone correct me if I'm off the mark here.

--
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON

Not really true, but true. by tod_miller · 2004-10-26 03:36 · Score: 1, Redundant

Yep, system design is harder, but if you have custom hardware and special programming, you can litterally run all these bitches at full steam.

If you need some inter process traps and doohickies, then you have to try and program code which is harder than licking you own tonsils going via your rectum.

Also, you have to realise that each one of these 200 processors have about 512kb of cache in total, and you can write a bloody good program using 100gb of CPU cache, especially if it is massively parrallella..lell...isable....

Now this reasearch may become prevalent as we try and run consumer based parellel processors, with multiple cores, as lithography sucks, and nanotech is going nowhere because of grey goo panic and Dexters Lab, and Smalle Pox.

Or so I heard.

--
#hostfile 0.0.0.0 primidi.com 0.0.0.0 www.primidi.com 0.0.0.0 radio.weblogs.com

Of interest to Cray-3 info by ebooher · 2004-10-26 03:43 · Score: 2, Interesting

Cray-3 memories by Steve Gombosi From a comp.unix.cray posting

Graywolf ("S5") was installed at NCAR. Like all NCAR supercomputers, until fairly recently, it was named after a Colorado locale.

This was the *only* Cray-3 shipment, installed in May 1993, the machine was a 4-processor, 128 Megaword system.

Two problems in the Cray-3 system were uncovered as a result of running NCAR's production climate codes (particularly MM5): a problem with the "D" module causing intermittent problems with parallel codes, and an error in the implementation of the square root approximation algorithm which caused incorrect results for certain data patterns (kinda like the Pentium divide bug ;-) ). These were rectified and replacement CPU modules were installed, although I can't remember the date.

The machine ran NCAR production until CCC folded in March, 1995. Since NCAR never paid for it, at some point we reduced the CPU count to 2 and let the machine run essentially unattended. I'm not too sure when that happened, although it marked the end of my regular commuting between Colorado Springs and Boulder.

There were a total of 7 Cray-3 "tanks" constructed. S1-S4 were single "octant" tanks (the smallest that could be constructed) which accomodated up to a 2 processor/128MW configuration. S5 and S6 were two-octant tanks. S7 was a four-octant tank which we used as a software development and benchmarking platform. S6 was chiefly used for system testing.

S1-S3 were diverted to Cray-4 testing once the Cray-4 project built up steam. S4 was diverted to the quite possibly suicidal Cray-3/SSS project after S7 became available (S4 was previously our software development machine).

For those of you who have Cray-3 posters lying around (by the way, I took all the photos on that poster as well as the Cray-3 and Cray-4 brochures and all the annual reports except the first two):

1) The big photo is of S5
2) Seymour is leaning on S5 (and you have no idea how hard it was to get him to hold still that long while wearing a suit...or to talk him into that particular pose)
3) The two "cooling system" photos are S6
4) The hand holding the module is mine ;-)

Cray-3 modules were 4x4x0.25 inches in size. Each module consisted of a multi-layer "sandwich" of PC boards (69 electrical layers), with 2 layers of 16 1x1 inch stacks. The stacks were the circuit boards containing the actual circuits (GaAs for logic, SRAM for memory modules). There were 16 bare GaAs chips mounted to each side of a logic stack. I think there were 12 bare SRAM chips on each side of a memory stack (the logic chips were square, the memory chips were rectangular).

--
"Genius may shine aloof and alone, like a star, but goodness is social, and it takes two men and God to make a Brother."

It was a customer requirment. by chipmeister · 2004-10-26 04:02 · Score: 1

Cray built this system to spec for a large customer who demanded the use of the Opterons. They decided to market it sonce they were going to build it anyway. If that customer had not requested this i doubt Cray would have built it that way. look for iformation on Red Storm.

Think on this by phorm · 2004-10-26 04:29 · Score: 1

A 3Ghz single CPU machine with 1GB of RAM will still about 2-5 minutes or more to render a single frame of a raytraced animation

Let's say for a moment you have a 88 minutes for something like "Toy Story 2." Now, let's say it runs at 30fps. So we have about 88min * 60sec/min * 30fra/sec = 158400 odd frames

Now, 158400 frames * 5min/frame = 792000 minutes to render on my 3Ghz machine

Now go with the concept that a complex image at very high quality might take longer to render, but even at this rates we're looking at
13200h
550d
1.5 YEARS to render an animated movie. Now, obviously there are things that require more power. Think tactical simulations with AI, galaxy simulations with huge numbers of factors, etc etc

Some of the above need results in a timely manner, your 3000Ghz P4 isn't going to cut it, even your business class server isn't going to cut it, and your clustered business servers while perhaps having the power/resources might not be able to do things in the time needed.

For general-computing there's a PC, for blow-your-mind power there's Cray...

Depends on the code by Dog135 · 2004-10-26 04:56 · Score: 1

That all depends on what you're doing. Lets say you're using it to render a movie. Lets say one frame takes 5 hours to render and 1 minute of data transfer (round trip). Not counting the dedicated computer for pushing the data to the nodes, you get pretty much a 1x increase in speed for each node you add. At 5 hours (300 minutes) per frame, you can have up to 300 nodes without a drop in performance. After that, you'd have to sacrafice a node for pushing data.

So in short, it really depends on what your "work:traffic" ratio is.

--
"That's so plausible, I can't believe it!" - Leela

Just imagine the game. by Anonymous Coward · 2004-10-26 05:00 · Score: 0

Someday, someone has got to make a game for one of these Supercomputers. Just imagine the possibilities!

Re:My new dream toy by wolrahnaes · 2004-10-26 05:08 · Score: 1

I bet this thing doesn't even have a graphic card.

Oh, well that's fine...I'll just devote a few hundred processors to software rendering.

Something tells me that 256 Opterons working together could smoke a GF6800Ultra

--
I used to get high on life, but I developed a tolerance. Now I need something stronger.

Shipping ... but are any systems deployed? by RageEX · 2004-10-26 06:58 · Score: 1

I guess they'll be setting their sights on SGI, which currently has the 'fastest computer in the world' bragging rights (well not officially, gotta way for the Nov. top 500).

Folding@home by d3ity · 2004-10-26 07:39 · Score: 1

So, how long do you guys think this thing would take to solve all the workoad on folding@home, and find intelligent life via seti@home?

Doom3 by tpillon · 2004-10-26 11:54 · Score: 1

But will it run Doom3 on super high quality settings?

--
--Do Not Write In This Space--

How HyperTransport interconnect is different ... by BitMan · 2004-10-26 15:10 · Score: 1

Power never offered before in a commodity interconnect, HyperTransport is changing how systems are designed.

Sys Admin 2004 November: Dissecting PC Server Performance

--
-- Bryan "TheBS" Smith
Independent Author, Consultant and Trainer

MOD PARENT UP! by Anonymous Coward · 2004-10-26 17:51 · Score: 0

Come on, it's a Space Robots allusion!

We are here to protect you from the terrible secret of space, anyone?

260 comments