SGI Installs First Itanium Cluster At OSC

Troy Baer writes: "SGI and the Ohio Supercomputer Center (OSC) have announced the installation of the first cluster using Itanium processors. The system consists of 73 SGI 750 nodes, each with two Itanium 733MHz procs and 4GB of memory, connected by Myrinet 2000 and Ethernet. Software includes Linux/ia64, SGI's ia64 compiler suite, MPICH/ch_gm, OpenPBS, and Maui Scheduler."

Comments

Re:Slashdot

[it's a culture thing]

The story has been out for months and people obviously haven't been paying attention or at least have failed to see just how concerted this effort is. This technology isn't new, but it's just a small part of the whole if you know where to look.

The new systems sound great, but they're tiny compared to what it's going to be like when the GRID is up and running.

What the story fails to mention is that this system is likely to be connected to the other GRID environments in the States and the new ones in Europe at which stage you wont be talking about just 4 super computer centres but nearly a hundred, each with several Tflops of processing power and a few petabytes (10**15) of storage.

I would suggest that to put this in the proper perspective you also look at IBM's contract to do the same to 4 sites in the Netherlands, the UK GRID which has 9 sites, the German one which I dont know much about but is fairly advanced and the CERN DataGrid. These are all interconnected with the same people working on several at a time.

Or you could have a look at the top500, find all the supercomputers in Europe & the US which aren't classified or commercial and then figure out what their combined processing power is. You should then have a fair idea just how much processing power there will be in a couple of years time 8)

Now back to the Particle Physics experiments ...

I'll swap you

[green pizza]

He gets to play with Linux/SGI clusters now, I'm stuck with Alphas & an O2000 in a back room somewhere.

You can come admin these clusters and I'll go work with your Alphas and Origin 2000.

Re:fascinating

[sharkey]

Hmm. Doesn't say WHICH company.

OT: Music Box for IRIX?

[BigBlockMopar]

Hey everybody,

I've had exactly two epiphanies in my life. One was the first time I saw an Amiga, back in 1985. I knew I would have one.

The other time, it was 1993 or 1994. I was installing a video projection system into a hotel meeting room. The computer to which I had to connect the video projector was a funky little purplish pizza-box with the name "Indy".

I fell in love.

I still don't have an old Indy or Indigo - or even any UNIX workstations - so far, I've kept my cravings in check with Solaris 8 x86, FreeBSD and Linux.

There was something running on that Indigo that day. I don't remember it very well, but I remember the name: Music Box. In the window, there was a picture of a woman, and cartoon musical notes would fall out of her mouth as eerie but beautiful music played in the background. For several hours while I aligned the video projectors in the room, it was on the screens, 10.5x17 foot image of an IRIX X session, with that eerie and hypnotic image and sound. It freaked out my boss, too. :)

It looked like something you'd just leave on the screen of your computer to amuse passers-by. Anyone have any idea what it was, or whether it's available for anything but MIPS/IRIX? A Google search didn't prove too helpful.

Re:OT: Music Box for IRIX?

[irix]

The Indigo's and Indigo 2 Impact-series were purplish.

Well, not exactly. The Indy is a blue pizza box. The Indigo2 was a much larger turquoise/grren box, and the Indigo2 Impact (which had tons of problems, BTW) was the same box in purple.

I should know - I have an Indy and an Indigo2 ;)

Re:SGI Sucks ( read on )

[Anonymous Coward]

Impressive but overpriced big iron.

Sorry, but there's nothing overpriced about the Origin 3000 family. I saw a quote for a 16-processor O3400 with 16 GB of RAM; the bottom line at list price was around $500,000.

That seems totally competitive to me.

And since I'm posting anonymously anyway, you might be interested to know that SGI is planning to release a new product any minute now. (It was announced to the developer and integrator channels this week). It's a four-processor (MIPS, natch) server in a 2 RU package. They're calling it the Origin 300.

But the cool part is going to be an interconnect product codenamed Sprouter. It lets you take 2 to 8 Origin 300 systems with 4 procs each and connect them using NumaLink (formerly CrayLink) into a single system image of up to 32 processors.

At, it's projected, half the price of a 32-processor Origin 3000 system. And for my kind of programming anyway, single-system-image beats the pants off that Myrinet stuff.

The O300 has 2 66/64 PCI slots, so that's enough expansion to let you attach your basic I/O devices like fibre channel RAIDs and high-speed networking and stuff. Each server comes with USCSI3 built-in, if anybody still uses that stuff. ;-)

Not everybody needs a medium-scale single-system-image IRIX machine, but I personally do a lot of ImageVision library programming. And ImageVision, being multithreaded at the core, loves big CPU counts. So for me, and people with needs like mine, it's going to be a very cool fall.

Seti@home

[goonies]

I bet they only want to beat the sith out of the top10 seti@home users by sending in packages every 10 minutes.... ;-)

two words:

[green pizza]

imprecise exceptions

IA64 has pretty clean and easy asm, but debugging is a complete nightmare. It's certainly better than x86, but give me MIPS64 or MIPS R1X000 any day.

Re:two words:

[stripes]

imprecise exceptions

That's kind of a pain (and at least the IA64 can be forced to do percise exceptions). Debugging something that the compiler has software pipelined will be a giant pain. Your source code will look like a simple loop, but the machine state will show you in the several times (i=7, 8, 9, 10 all at once...but not quite as far into the loop for some values).

Re:IA64s are kickass...

[VAXman]

Are you new to computers? The memory you get from malloc isn't actually allocated until you access it. There is no translation for the pages at first, so the first access gets a page fault at which point the OS maps a PTE to it. And, no, you could not achieve that with your first 5GB hard drive because most likely you were running IA-32 which supports only 32 bit linear addresses. On Linux a process has a maximum of 3GB of memory so that's the maximum you would be able to allocate (subtracting the tiny overhead from malloc, as well as the other code and data the process has).

a plumber? like this guy...?

[green pizza]

http://www.overclockershideout.com/customer/MTP/

Overclocker kiddies will do anything for an extra 8 frames-per-second.

Now here's a scary thought... how about the overclockers team up with cluster builders??

Re:IA64s are kickass...

[Espen Skoglund]

It's not only a matter of optimizing for using fewer registers. If you're coding loops for instance, you would most likely want to use the register rotation stuff to do software pipelining of your loop: register r32 becomes r33 from one iteration to the next, register r33 becomes r34, and so on (same thing for predicate and floating point registers). Using this technique you can do some really kick ass optimization of your inner loops. Only problem is that coding the thing and debugging it will do your head in. Now, you would want the compiler to do this stuff for you, but gcc does not support it yet (the SGI Pro64 compiler does, though).

This was one example of low level optimizations, another one is giving hints to different branches (both target and outcome of branch conditions). This is also best done by the compiler (at least the branch target hints), and works even better if you can supply the compiler with profiling information. You can also give data prefetch hints and specify which cache level different prefetch data should go into.

Another example of when you might need to do asm is when you do SMP. The reason being that different load and store instructions are given semantics of how the are to behave in a multiprocessor environment: you want acuire semantics on this load, release semantics on this store, fence semantics here, undefined semantics there, etc. I can't see how the compiler would be able to generate correct assembly in this case (unless it is modified so that you can attach some new attributes to your variables and types).

Then there is this whole plethora of floating point stuff that I won't mention because I don't know shit about it.

Hmm, reading your post again I see that I didn't really answer your question, and most of my ranting about doing asm coding ended up with the conclusion that having the compiler do all the nasty stuff is probably better anyway. I guess I'd better shut up now.

bah

[green pizza]

give me a 4D/35, IRIX 4.05 on tape, and leave me alone

Wrong logo, Wrong idea

[green pizza]

Slashdot is *still* using the old cube logo, rather than the new "sgi" logo. Sure the new logo sucks and the old logo is quite cool, but it's time to move on. The old days are long gone. Like the rest of the 'new' SGI, there is nothing special about the SGI 750 Itanium box, it's the same box with the same Intel reference board that HP, Dell, and others are selling.

SiliconGraphics has left the building. The "hip new" SGI is here. Quit using the old logo, it reflects a much cooler company that no longer exists.

Re:Wrong logo, Wrong idea

[BitwizeGHC]

Hey, never give up hope! Remember how Borland changed their name to the laughable "Inprise" some time ago? They're back as Borland now. Companies want to please their customers, and if enough want some part of the old SGI back they'll get it.

Re:Wrong logo, Wrong idea

[fgodfrey]

Err, actually, you won't need a P brick unless you need more than 5 PCI cards. One of the features we are kicking around (read "maybe someday we'll do this but don't go telling anyone that we're committed to it") for system partitioning is the ability to boot a partition I/O less from another partition. You would then be able to NFS mount a root file system over the cross partition communication layer (network interface "cl0").

Re:Wrong logo, Wrong idea

[InsaneGeek]

Whoops got my bricks mixed... was thinking of the pdu brick.

Re:Wrong logo, Wrong idea

[InsaneGeek]

Would you consider the following link new & cool???

http://oss.sgi.com/projects/linux-scalability/do wn load/mips128.out

It's a link to a running *single image* (i.e. not a beowulf cluster) 128 proc Linux system on a mips box. When was the last time you saw Dell, HP, Gateway, do that with a Linux system? This little cluster in the post is not where SGI is going with their systems (as you said anybody can do that) but are moving with Intel numa with high speed numalink interconnects that are much faster than standard Myranet (their cross bar is in the gigaBYTES).

The real interesting part that I see (if they can live long enough for Intel to release Mercede) is the system partitioning and it's modularization. Need 2 more procs for your database but all your CPU slots are full, well plug in another "C brick", you won't have to worry about running out of CPU slots in your frame, everything is a component, you won't have to do another fork lift upgrade. Also with their partitioning I can purchase a 100 proc system, partition it into twenty 5 proc systems all within the same frame I don't have to pay for all of the overhead for 20 different frames and space for expansion in 20 of those frames, because all I need to do is plug in another "C brick" and give two of the boxes two more CPU's without ever having to have bought the headroom to begin with.

Whoa getting a bit long, SGI really has some cool stuff going on right now. If they could only market themselves out of a paper bag they wouldn't be in the bled-dry situation they are in. Personally I think the best thing for them would be to be bought by someone with big pockets who can market a product properly.

Re:Wrong logo, Wrong idea

[InsaneGeek]

In my example you have already purchased the I-brick, possibly the R-brick (depending upon how large you are), if you are using XIO the X-brick, etc. All you are doing is adding some more CPU's to existing systems in the 3000. Which before if I had 32 procs in a Sun 6500, you were done you had to do essentially throw our your 6400 and do a forklift upgrade to an E10k; with SGI's 3000 you are at CPU's you don't run out of headroom, you never have to get rid of your original investment. I can start with a 4 proc system and scale it up to a 1000 proc system and never have to throw out hardware to get there because you don't run out of headroom (there is a theoretical max I will admit but it's a limitation in interconnect speed).

Anyway yeah, you have to buy the I-brick, P-brick & C-brick to get a system running and each system will need to have at least an I & P brick; but to add to an existing system no problems.

5.86 Gflops per processor

[Perdo]

5.86 Gflops per processor (We can assume marketing fud value) at $3,300 Compared to the Athlon 1.4 with a peak of 1.37 Gflops (benched value) for $145. Keep in mind the nifty PCI card that has 4 G4 processors on it. That would make for 92 processors at 1Gflop each in just one box.

Unfortunately, this seems to mark Intel's latest attempt to push an overpriced, substandard product at us. The P4 was crippled from the begining and is only just now begining to show any promise. The PIII at 1.13 and 1.2 Ghz is finally available 8 months after the recall of their failed 1.13 processor. Even their purchase of Alpha from compaq seems to be just stock propping because the original creators of the alpha are now working for AMD. The reason Compaq was willing to sell in the first place is the second generation alpha has been subjected to over three years of delays because they simply did not have the engineering talent to improve a ten year old design.

The talented engineeers are working for AMD, built the athlon and are working on the sledgehammer.

Before anyone jumps to Intel's defence, like they need defending as long as they are the 800 pound gorilla, keep this in mind:

Craig Barret warned "This was a year of record annual revenue and earnings; yet, slowing economic conditions impacted fourth quarter growth and are causing near-termuncertainty,". He was faced with AMD going from 10% market share to 34% market share in a year. Wall street took barret's word as gospel that the entire market was in decline and not just Intel's market share. Intel is a market bellwether so we all got laid off. Just so Intel would not have to admit that AMD had a better product. Nasty business. Intel does not have a great product and they are reckless with their power.

Re:5.86 Gflops per processor

[Anonymous Coward]

think again, Apple current top-end dual 800Mhz PowerMac G4 runs AltiVec code at over 5GFLOPS. My older dual 450Mhz can run up to 3.2GFLOPS... And don't forget, the G4 has a successor due pretty soon now. I think these guys will find the pleasure of being the first will be tempered by eventually having the worst price/performance 64bit cluster in academia.

Re:SGI Sucks ( read on )

[daviddennis]

About three weeks ago, I got one of those SGI Insider newsletters in my email, with an unbeatable offer for one of their 1600SW monitors. I really wanted an Apple Cinema Display, of course, but for $550 plus tax and shipping, it sure was tough to beat; they're still about $900 on eBay.

I placed my order straight away, and it still has not arrived. The reason? During this crisis period for the company, where everything is falling apart, they are apparently changing their sales systems to use the latest version of Oracle. During this transfer, their ordering system is entirely manual. They haven't lost my order (yet), but they did take many more orders than they had available machines for, and they claim to still be straightening out the mess.

I lost even more faith in SGI when I noticed their "configure your SGI system" web page was Windows-based. For shame, SGI!

Part of this is, of course, the kind of weird degradation that occurs at any big company. But I can't say I have any faith in the company that makes that kind of blunder. Changing your ordering systems to the extent that operations are severely impacted is just plain stupid; I've never managed a major business, but if I were, the minute I had problems of that severity, I'd go straight back to the old system and send developers of the new one back to the drawing board. Elementary, surely?

Oh, and even though I am surely a beneficiary of a great monitor deal because of this, they should have dumped all the monitors on eBay, ten or so at a time. They would have made, maybe $ 800 per monitor instead of $600. That's a pretty respectable chunk of change right there.

D

These numbers are meaningless

[RelliK]

P4 Xeons have been available for several weeks now; we've had a dual P4 machine on site for about a month.

OK, I was not aware of that, but tell me where I can actually buy them. I looked at Dell's site before I posted. If Dell doesn't have them, nobody else does.

The NAS Parallel Benchmarks would seem to indicate you are wrong.

You pointed to the single (type of) benchmark that makes P4 look good, though not because of P4's virtues. P4 has two channels of RDRAM giving it 1600 * 2 = 3200 MB/s of memory bandwidth. The Athlon machine in the benchmark you pointed to had a single channel of DDR DRAM, giving it 2128MB/s of memory bandwidth. Fluid Dynamic simulation is one of the _few_ things that can really take advantage of greater memory bandwidth. Therefore P4 wins this round by riding on Rambus's coat-tails. Prove me wrong. Point me to the same benchmark where both P4 and Athlon have the same type of memory (both support SDRAM now, you know).

Re:SGI Sucks ( read on )

[LinuxParanoid]

What went wrong?

I know this is a rhetorical question, but having once spent a lot of time thinking about how to advise them before and during their fall, I'll give you my analysis. Some of this I saw at the time, some aspects I only saw too late. Learn from their mistakes.

Here's what went wrong:

• SGI succumbed to the Innovator's Dilemna. Unwilling to canibalize their high-end graphics, they refused to enter the PC 3D graphics space and left it open for 40+ hungry competitors. Many of their engineers left for said competitors when it was clear SGI was going nowhere. (Jim Clark recognized the dilemna in 1992-1994 by the way and jumped ship himself, a harbinger of things to come.) In fairness, its hard to canabalize a 1-2 billion dollar workstation graphics market in hopes of winning a 50-200 million dollar 3D PC graphics one.
• SGI refused to go to NT early on when they would have had leverage in making the move that could have forced/encouraged MS to adopt OpenGL exclusively for 3D. Instead, they said no to NT for too long, and when they said yes, it was a me-too decision that was later partially reversed in favor of Linux. Rather than recognizing and admitting they'd lost the war and pursuing the best possible terms, they chose, either conciously or through inaction, the "go down fighting, maybe we can still win" route.
• SGI's bread and butter midrange workstation 3D graphics was prone to "good enough" copying by competitors, Sun and HP. SGI's engineers spent a lot of time focusing on developing unique high-performance texture mapping serving 2% of their market (the entertainment sector) rather than on improving geometry engine performance further for the benefit of the biggest 40+% market (CAD).
• SGI's choice of a strategic response to PCs was poor: "We'll have highly differentiated systems" (the O2). Unfortunately, the differentiation (UMA, texture mapping, imaging, system bus architecture) was largely in areas that didn't add much value to their largest segments of customers. They built not what most of their customers wanted, but what the "cool" customers wanted. What most of their customers wanted was lower prices- that's what most of them ended up going to when dumping SGI.
• SGI engineers were late. Whether through lack of focus/discipline, resistance to "impossible" marketing schedules (that turned out to be necessary), choice of agressive cutting-edge/bleeding-edge component technologies that proved hard to debug, whatever. Your pick. Key products in the timeframe you mentioned were late, late, late. The midrange IMPACT graphics were announced June 95 as shipping but in reality didn't really ship for another 6 months, more or less. (In the meantime, Wall Street lost faith in the company as a momentum stock and SGI stock price dropped from its alltime high of 45 down to mid-20s.) Subsequent products also had a tendency of being late (O2, a year later than needed, Visual PC was late, etc.) SGI engineering exhibited a lack of discipline when instead they needed increased focus to adjust from product design cycles of 4 years (traditional workstation graphics) to 6 months (90s PC graphics). In their defense, this wouldn't have been easy. But at least some there knew about this. Which brings us to the last problem I'll go into.
• Arrogance. SGI was arrogant. No PC could beat us. We'll always stay ahead. Sun? HP? Ha! Yeah, we can do low-price graphics, look at Nintendo-64; see a PC beat that! etc, etc. Not everyone at SGI there was, but a heck of a lot were.

--LP

P.S. I didn't even get into their server strategy, Cray, and later events. Another time perhaps.

SGI has been faithful at least

[HerrGlock]

It seems the ones who have been faithful to their commitment to Linux are SGI and IBM. The others have tried it and then decided it was not worth the effort to reach such a small segment of the population.

I'm glad there are still big players in the Linux field, though, it helps forward the cause and the OS and lets people know there IS an alternative. By all means, SUN and other, keep your propriatary stuff available and have that as the default, but allow people the option to choose another OS if they so desire.

DanH

Re:15 years for MicroSoft to 64 bits?

[cr0sh]

Correct me if I am wrong...

But hasn't there been a 64 bit version of Linux for a VERY long time - ie, for the Alpha?

What I find interesting (and it is just a personal conspiracy theory of mine, and probably holds zero water) is that it took "this much" time for Intel to release their 64 bit chip - how long have we been hearing about it now? At least 2 years...

Is it just a coincidence that Microsoft finally has a running platform for it just at the time this chip comes to market?

???

Slashdot

[Anonymous Coward]

This story:

http://dailynews.yahoo.com/h/nm/20010809/tc/tech _s upercomputer_dc_1.html

Has been out for all most *five* hours. The story talks about how Linux is going to be used as the OS for the *biggest* *cluster* *of* *super* *computers* *in* *history*

It is the greatest news I have heard in months and it "matters" if you ask me. The Super computer(s) will be funded by the National Science Foundation(NSF) and it is reported that the super computer(s) will be able to calculate in one second, what it would take a hand calculator ten millions years to calculate. In addition the total disk space will be enough to hold all most one and a half million full-length novels.

In other words, the Linux OS is going to be used for the largest computing grid in the history of the world.

This story has been availavle on Yahoo!(TM) LinuxToday, Newsforge etc for hours. I submitted the story 3 hours ago and nothing...

I used to read slashdot for the news and told myself I could ignore the mindless trolling and moronic comments, now I realize the news service is garbage and I have no reason to read /. anymore.

Looks like it is newsforge or LinuxToday for me
:-)

Re:Slashdot

[kietscia]

A whole 5 hours before it showed up on Slashdot. Oh the humanity . If I'd only known about this a few hours ago when that extra $10M laying around on my boss' floor I could of had one. Impatient twirp went and spent the whole wad on 1 month of 24x7 support for our lone Win2K server.

</sarcasm>

Reading /. is not done for timeliness (although I usually see things here first) its for the breadth and variety of articles. If you want a place where things show up picoseconds after they hit the ether then you've shown up for the wrong party. And yes, there are trolls and moronic comments (scroll up a bit and you might see one) but the good stuff far outweighs the negatives. Since I gave up coffee (I'm still in mourning over that one), /. is the first thing I do in the morning.

IA64s are kickass...

[Telek]

I had an opportunity to work on them about a year ago (the first one we received was a doorstop, literally... The sucker weighed 73 pounds in it's shipping package (I'm NOT KIDDING... They reeled the box in on a trolly, and I laughed at the guy cuz it looked small enough to carry, but then I tried to pick it up...) and didn't even boot, but intel shiped them with 2GB of ram and a kickass SCSI system, so let's just say that my desktop became a SWEEEEEET machine.), but once we got ones that did work, they were sweet machines. I was porting bigint libraries for encryption that I had hand-coded in assembly for the x86 platform, and going from 8 general purpose registers with 1 predicate register (i.e. only 1 carry flag) to having 128 general purpose registers, and 63 predicate registers was a GODSENT.. AMAZING... For anyone who's coded math routines in assembly, you know how much of a PITA it is having only one carry register. This was simply amazing. I could do 1024-bit RSA purely in registers, no memory access outside of the initial read of the data and the final write. Needless to say it flew. It was interesting because literally you wouldn't need a hardware crypto card if you have an Itanium system. So basically Intel really put in a lot of good effort into designing this new platform to avoid the pitfalls of the problems that they experienced with their x86 architechture.

The machines also had 4GB of ram, so it was fun to do:

char * myStr = (char *)malloc(-1);

and have it succeed! (that's a 4GB memory allocation)

Re:IA64s are kickass...

[blair1q]

Which obviates your point.

A 32-bit CPU can address 2^32 bytes of virtual memory just fine.

--Blair
0xffffffffffffffff

Re:IA64s are kickass...

[schwap]

With that many registers, wouldn't it be better to let the compiler do the work since it should be able to optimize it quite well? Being an assembler programmer at heart, I know the necessity of getting down to the bare metal, but my experience was that I only needed to do it because the x86 arch had so few registers that I could, indeed, do it better than any compiler.

Re:IA64s are kickass...

[blair1q]

The machines also had 4GB of ram, so it was fun to do: char * myStr = (char *)malloc(-1);

You're going back to school, m'laddo.

malloc(3) has overhead. You should have expected that to fail if you were thinking only of how much RAM you had. Fortunately, modren cowmpooterz has virtual memory. So you could have got away with that when you got your first 5GB hard drive.

--Blair

Re:IA64s are kickass...

[blair1q]

"The memory you get from malloc isn't actually allocated until you access it."

malloc(3) has a job to do, and that's to keep you from using memory you can't use. If it doesn't "allocate" it until you access it, then whatever system you're using is incontrovertibly broken.

malloc(3) is at least two layers above the actual memory. Plenty of room for the MMS and MMU to translate it.

Don't go assuming what I was running.

And if Linux limits memory to 3GB (which I did not know, as I don't often dig around the internals of toy computers) then what good is it?

--Blair

Re:obvious

[ThatComputerGuy]

he said no imagining... from now on, all beowulf clusters must be fathomed.

in related news

[Matthew Luckie]

there is a much larger cluster of linux machines going to be created care of the NSF. press release here. good day for linux!

Re:in related news

[Perdo]

"The DTF will be the largest, most comprehensive infrastructure ever deployed for scientific research"

"creating the 13.6-teraflops system--the most powerful distributed computing system ever"

OK, what exactly does that make seti@home? 16.98 TeraFLOPs/sec as I write this, distributed computing system, scientific research... What am I missing?

geez.. sounds like your describing Deep Thought

[Pengo]

We can ask it for the answer to the great question of the universe?

a dialogue

[the_tsi]

Slashdotter: I wish to complain about
this company what I read about not half an
hour ago on this very website.
Me: Oh yes, the, uh, the Workstation manufacturer...What's,uh...What's
wrong with it?
S: I'll tell you what's wrong with it, my lad. it's dead,
that's what's wrong with it!
M: No, no, it's uh,...it's resting.
S: Look, matey, I know a dead company when I see one, and
I'm looking at one right now.
M: No no it's not dead, it's, it's restin'! Remarkable company,
the SGI, idn'it, ay? Powerfull CPUs!
S: The CPUs don't enter into it. It's stone dead.
M: Nononono, no, no! It's resting!
S: All right then, if he's restin', I'll wake him up!
(shouting)
'Ello, Mister Bob Bishop! I've got a lovely fresh government
contract for you if you show...
M: There, it moved!
S: No, it didn't, that was you faking a press release!
M: I never!!
S: Yes, you did!
M: I never, never did anything...
S: (yelling and hitting the cage repeatedly) 'ELLO SGI!!!!!
Testing! Testing! Testing! Testing! This is your nine
o'clock alarm call!

See, guys, I told you they still had life left! :)

-Chris

It is worth noting...

[uncleFester]

The person who submitted the story (Troy Baer) is also the admin of the beast. Troy had an interesting article on the current (previous?) cluster setup at OSC in one of the recent Linux mags (Linux Journal, 2001 July). To call Troy a proud father of this setup might not be too far off. ;)

I knew Troy from school, admin-ed with him in the Ohio State engineering labs. Ask him what he's doing with that Aero Eng. diploma nowadays.. ;) Overall, a pretty damn sharp guy. He gets to play with Linux/SGI clusters now, I'm stuck with Alphas & an O2000 in a back room somewhere.

-'fester

Not a bad deal

[wirefarm]

"... The old system will be divided into smaller clusters and cascaded to faculty ...

"Damn. I asked for an iMac, but got this stupid Linux cluster instead!"

;^)

Cheers,
Jim in Tokyo

Re:Why should I _care_ about having the same memor

[RelliK]

Oh man, fro your last post I got the impression that you saw the light. Apparently I was mistaken.

Actually, I was providing a counter-example to a specific comment of yours: If software was optimized for P4 and Athlon, P4 would still be pathetic.

OK, let me repeat what I said in my last post. Your benchmarks show that P4 with RDRAM is faster than Athlon with DDR DRAM. This does *not* imply that P4 is faster than Athlon. To give you a mathematical analogy, if you know that A + B > C + D, you cannot claim that A > C : you do not have enough information for that. (This is especially true when you know that B > D). In the case of your benchmarks, you cannot claim that P4 is faster than Athlon, especially since you know that two channels of RDRAM have a greater bandwidth than once channel of DDR DRAM.

They do all favor high memory bandwidth systems, but that's the nature of the beast in HPC -- almost all scientific applications are memory bandwidth hungry.

Who cares if the benchmarks use different algorithms? They all test exactly the same thing: memory bandwidth. This statement of yours actually makes my point above stronger. You admit that all of these benchmarks are highly dependent on memory bandwidth. It appears that you agree with my supposition that these benchmarks depend more on the memory bandwidth than the CPU speed (integer sort for example would be 100% memory I/O). Yet you are still trying to claim that P4 is faster than Athlon. These benchmarks do not prove that. They prove that the bandwidth of two channels of RDRAM is greater than that of one channel of DDR DRAM, but we kind of knew that already, didn't we?

So, your counterexample is invalid and my claim stands. I really think that we are starting to go around in circles.

Re:Shut the fuck up

[BigBlockMopar]

IT'S INDIGO FOR FSCK'S SAKE

Well, of course it is. I was building up to it, and revealing the name in the color of the box was rather self-defeating to the melodrama inherent in later naming it an Indy.

Wither XIO?

[green pizza]

Hmm, PCI and no XIO? Would be nice to at least have a couple of each. Though 2 rack units isn't much room to work with. Wonder which version of IRIX it'll run? 6.5.13 + patches? 6.5.14?

Bullshit alert

[RelliK]

Oh man, technical errors abound. The P4 is actually faster than the Athlon for scientific computing, except for the compiler fact

Pentium 4 has an absolutely pathetic floating point performance. Even Pentium 3 at 1000MHz outperforms Pentium 4 at 1500MHz on floating point. See here for example. Your claim that Pentium 4 can do 3 floating point operations per clock cycle is nothing more than pulling numbers out of your ass. (unless you can somehow substantiate your ridiculous claim).

The P4 looses to the Athlon simply by the reason that the compilers can not use the vector instructions properly.

AMD has never had code optimized for their CPUs. They have always fought an uphill battle. Yet they managed to beat the crap out of intel in absolute performance (price/performance they had for a long time). The whole compiler crap is a strawman's argument. AMD has 3Dnow instructions which nobody uses. If current software was optimized for AMD, P4 would look even more pathetic.

Why anyone would be an Itanium instead of a dual P4/Athlon beats me.

Uhhh, perhaps because there is no such thing as dual P4?

It has less on-chip cache than a Celeron (128kb total)!! Sure it's packaged with a lot of sram, but still.

I don't know how to break it to you, but 1) Celeron has exactly 128KB L2 cache, and 2) SRAM stands for Static RAM, which is used for cache (as opposed to Dynamic RAM, which is used for the main memory).

Re:Bullshit alert

[RelliK]

x87 FPU, sure. But that was only on for backwards compatability. Once SSE2 turned on (which, unlike the Athlon, is contained in a seperate unit on the processor and does not run in the FPU), the Pentium 4's FPU becomes a force to be reckoned with (surely you saw the SPEC scores?)

That's like saying Windows supports win32 binaries only for backwards compatibility. P4's FPU really stinks and now all of a sudden intel wants everybody to switch to SSE. Well, sure it will help (and, as you noted, it will also help Athlon which currently supports SSE, and future versions will support SSE2 as well), but high-precision floating point arithmetic is impossible with SSE, so x87 FPU is not going to go away. As for having x87 FPU and SSE in separate ALUs, well, that won't really matter if, as you claim, everybody will up and switch to SSE, will it? So my claim stands: P4's FPU is pathetic. If software was optimized for P4 and Athlon, P4 would still be pathetic. If future software uses SSE2, Athlon's performance would improve just as much as that of P4. And, furthermore: using the "compiler is not able to take advantage of some wiz-bang feature" is a strawman's argument. I repeat: AMD has always fought an uphill battle and has never had software optimized specifically for their CPUs, yet they still managed to beat the crap out of intel.

They're called Intel Xeons. While it's true that it's not technically a "Pentium 4" by name, it is a P4

A Xeon is based on P3 core. Xeon is just an overpriced P3 with a larger L2 cache. My claim stands: there is no such thing as dual P4. If and when Xeons with P4 core are relased, then we'll talk.

15 years is pretty good

[Proud Geek]

Just be glad that Apple isn't the company doing it. It took them 16 years to get a 32-bit OS after their computers started using 32-bit chips, and they already had 23 years of OS writing experience. Contrast that to Microsoft, which took 6 years with only 14 years of OS experience.

Want innovation? Who do you think has it? Also, one company uses a UNIX based OS; the other at least tried to fix the mistakes of the past, although they did make more in the process of doing so.

MIPS is at a deadend

[jc2436]

The MIPS architecture has run out of gas. All it has left in it is a few more speed bumps. SGI has no money to invest in something that would succeed the MIPS chip. Another note: Why did Compaq dump the Alpha chip ? Because it can't run lock-step, so it cannot be used for Compaq NonStop (formerly Tandem) processors. McKinley chips apparently can be run lock-step, so they will be used for the new NonStop systems.

Not exactly... (Re:It is worth noting...)

[Troy Baer]

The person who submitted the story (Troy Baer) is also the admin of the beast.

To give credit where credit is due, the admin of that system is Doug Johnson, who has done an enormous amount of work to get this thing working. I'm just a user support guy who writes lots of documentation and happens to dabble in systems stuff like Maui and PVFS in my Copious Spare Time[tm].

--Troy

Re:Why should I _care_ about having the same memor

[RelliK]

And I suppose the fact that width of the P4's memory bus interface (256 bits wide) is twice as wide as that of the Athlon (128 bits wide) has no influence on the memory bandwidth available?

That's just plain not true. The P4 memory bus is 32 bit wide (16 bits for each RDRAM channel). Athlon memory bus is 64 bit wide (one DDR DRAM channel). You are probably thinking of the width of the L2 cache datapath, which is 256 bit for P3 and P4 (I don't recall what it is for Athlon).

Or that I can't get an optimizing compiler for the Athlon that's comparable to the Intel Fortran compiler?

I think I said it 3 times already, but I'll repeat it again for posterity. AMD has never had software optimized for their CPUs. They always fought an uphill battle. Yet they still managed to beat the shit out of intel.

Look, I don't have the luxury of caring which processor is better in a "fair" test with the same memory, etc. -- my job is to figure out which system (processor/memory/IO/etc.) is fastest for our users' applications

That's fine and that's a reasonable thing to do. But you cannot use your benchmarks to claim that P4 is faster than Athlon. You benchmarks show that P4 with RDRAM is faster than Athlon with DDR DRAM. This does not imply that P4 is faster than Athlon, which is what you were trying to claim. Also, you cannot make such a claim based on only one benchmark.
Once again, I have no problem with people trying to find the best system for their needs. I have a problem with unsubstantiated claims.

BTW, I suspect that for your particular application, memory bandwidth matters more than the CPU speed. So you may actually be benchmarking memory instead of CPU.

Re:Why should I _care_ about having the same memor

[RelliK]

oops I screed up html. pressed post too soon.

Re:SGI Sucks ( read on )

[uweber]

Well sadly enough i can only agree with you.

I actually do not have any hope left for SGI. Here is why:

-Their workstations do not cover enough markets to sustain themseves let alone generate enough revenue to set them back ahead of the pack.

-Their single image clusters are not cost effective enough for the scientific comunity and their use for Visualisation, where people would be willing to pay the insane price, isn't large enough.

-Anybody can build UNIX servers.

-Anybody can build MPI clusters.

So I think they are reduced to niche markets which will not cover their cost in R&D which means they will have to license technology or buy components from others (CPUs, bussystems etc.). However they might run out of breath in the time they redesign their gear to take advantage of the new tech, and reduce the production cost.

So my guess is they will be consumed slowly by their competitors - lets only hope some of their tech remains.

Re:Interesting

[Guppy06]

"One only wonders if this would be posted on Slashdot if the OS was Microsoft Windows 2000 Datacenter."

It wouldn't be worth mentioning, since you can't cluster more than four nodes with W2k Datacenter. When you compare that to this cluster of 70+...

Mod Timothy redundant! It's been done folks..

[Myself]

IBM has a story about this, the second cluster "to be installed this summer" already exists.
Also read The story at NCSA if that's not enough for you.

SGI Sucks ( read on )

[green pizza]

Prepare to lose all karma...

SGI sucks.

Most of their hardware is great, as is most of their software. But their head is completely up their ass these days.

Stagnant desktop machines. Impressive but overpriced big iron. OEM PCs. And a terrible logo. What went wrong? Where to begin??

Once upon a time there was a company called Silicon Graphics. They got their start by making wickedly powerful terminals to provide 2D and 3D graphical front end to massive minicomputers and supercomputers. Mind you this was two years before Apple introduced the Macintosh and Xerox was still playing with the underpowered Star. Shortly there after they began selling a line of large rackmount, standalone graphical computers that used multiple large boards covered with cpus, fast ram, and other goodies to churn out decent primitive 3D in real time using the GL framework (later called IRISgl, which eventually became OpenGL). This was about the time your dad upgraded from a C64 to a IBM XT.

Fast forward to 1995. You and I were probably playing with a Pentium 100 and looking forward to the rumored 3Dfx Voodoo card. In that same year, SGI upgraded their Onxy graphical supercomputers to InfinteReality graphics... providing performance on par with a Geforce 256. Except the IR could handle 64 MB of dedicated texture ram and 320 MB of frame buffer. Three IR "pipes" could be installed in a single system, and each pipe could even be broken down to multiple channels. IR allowed the world of graphical simulation to finally approach photorealistic quality with multiple projectors / monitors providing a wrap-around display (keep in mind that much of this was available on a limited scale 1991 with SGI's RealityEngine pipes). Both the Onyx and SGI's non graphical server, the Challenge, received a CPU upgrade. Up to 24 MIPS R10000 CPUs running at 195 MHz (each providing 390 MFLOPS + 390 MIPS) could be installed in the Onyx. The Challenge could take up to 36. SGI's flagship desktop machine, the Indigo2, received upgrades as well. The top of the line model had an R10K/195 CPU, up to 640 MB of interleaved ram, two channels of SCSI, and Maximum Impact graphics (4 MB of dedicated texture ram, 27 MB of framebuffer, and performance somewhere around that of the TNT2).

SGI's machines continued to get better. Indigo2 was replaced with the Octane. Onyx and Challenge were replaced with the Onyx2 and Origin, and later with the Onyx 3000 and Origin 3000.

Here we are in the middle of 2001. SiliconGraphics has become "sgi" with a NYSE stock price below $1. Their O2 desktop machine hasn't changed much since 1996, and aside from the new gfx card and faster CPUs, the Octane2 isn't a whole lot different than the original Octane in 1997. Onyx 3000 uses updated graphics based on the original IR from 1995. Perhaps the only noteworthy change has been the architecture of the new Onyx and Origin. Both can scale as a single machine to 512 CPUs with 1 Terabyte of RAM. Many of these massive machines can be clustered together for even more power... at an insane cost.

The company that brought us 3D on the desktop has pretty much come to a halt. Their desktop machines haven't change much in almost 5 years. Their big iron is impressive, but expensive as all hell. And their PCs... where to begin on the PCs... They tried making what could have been the coolest pair of PCs of all time. But due to delays and driver issues, the machines ended up being overpriced, nonupgradable ho-hum boxes. Pretty soon they hit the other end of the spectrum with generic OEM PCs. And now this, the "SGI 750" Itanium. A box that is identical to that which is being sold by HP and Dell. The only thing SGI about it is the logo. We're not even dealing with the same SGI. This new "sgi" couldn't have possibly come from the same roots as the old, grand, SiliconGraphics.

I can't help but wonder what the old SiliconGraphics would be doing today. Like another poster pointed out, the Octane would probably have an ever faster architecture, better graphics, and probably 4x the CPU power. This new linux cluster would probably be based on much better machines and using something better than Myrinet (which is limited by the 66MHz/64bit PCI bus the card sits in). The old SGI would have made a complete fire breather, not some OEM stack that anyone could build themselves. The old SGI would have the cube logo *and* rightfully wear it.

When I look inside my old, used Indigo2 from 1995 what do I see? I see its 750 watt power supply. I see not a graphics card, but *three* massive cards working together and connected to the power supply via a thick jumper cable. I see engineering at its best. I see a product that pushed the limits of silicon and interconnects. I see something that was worth its $50,000 pricetag. I see something that was indeed an order of magnitude more powerful than anything else on the desktop.

When I look at the current SGI desktop machines, I see something I can buy for less at Best Buy.

I recently saw a demonstration of the Onyx 3000. One of the demos was a visualization app used by an automobile maker. The app showed a few different cars in full detail across three screens (each 1280x1024) in a panoramic configuration at a sustained, locked 75 Hz + 75 FPS. The cars had complete reflection features that interacted right down to the metallic flecks in the paint. The detail was right down to the 3D textures that made up the subtle surface of the dash plastics and the seat leather. It was truly photorealistic. I've seen the Geforce 3 demos, they were nowhere near as impressive as the car demo.

Another demonstration showed the Onyx's power at loading textures. The machine they had was connected to several RAIDs containing over 500 GB of satellite and aerial photos. On the same three screens and in the same 75 Hz + 75 FPS were able to zoom down to a national park, pan across to another state, and zoom back out to planet Earth floating in space. All in real time. The RAIDs were clattering so loud I could hardly hear the man giving the demonstration. The Onyx never missed a beat.

If the old SGI was here today, we'd have that kind of power on the desktop. And it would cost $50,000 and consume 750 watts. Not $500,000 and 9,000 watts.

And we wouldn't have a Myrinet connected stack of Itanium PCs. We'd have something a whole hellofa lot better.

[end rant]

it's worse than that...

[green pizza]

You're thinking of "Rocket" Rick Belluzzo, former CEO of SGI. He was responsible for putting MIPS/IRIX on hold, courting the Wintel crowd, and the "sgi" logo. He successfully put SGI in a steep nosedive they'll probably never recover from.

Where is Mr. Belluzzo today?

Hold on to your hat...

http://www.microsoft.com/presspass/exec/belluzzo/d efault.asp

15 years for MicroSoft to 64 bits?

[peter303]

It took MS 15 years to have a full 32-bit OS
after those chips came out. Hope they are faster
this time. 32 bit NT on an Itanium would be a waste.

SGI and SUN have had full 64 bit OS for 7 & 5
years. Yes, there are bugs to shake out in the
beginning. OF course Bill & Steve will announce
they are "just about to ship" for years until
they do.

Competitors Know the Itanium Sucks

[Orbitalb]

Apple knows that at present, the Itanium (which is based on a RISC-like design scheme called VLIW, or EPIC) is stuck at 800MHz even in its prelease form -- slower even than the shipping PPC 7450 (G4) which is running at 867MHz and expected to cross the GHz barrier by year's end. The Itanium, while offering many important features lacking from previous Intel chips, has also been criticized for its poor performance - particularly when running existing x86 applications and operating systems. Apple's next-generation processor is the PPC 7500 or G5, which is also 64-bit and employs the first complete architectural revamp since the first PowerMacs shipped in the mid-90's...but the G5, unlike Itanium, can process existing 32-bit PowerPC instructions at full performance and is expected to continue to scale upward in clock rates; the initial G5s slated to ship in early or mid-2002 are currently projected to run at 1.2GHz with per-clock performance targeted for roughly 65% beyond Itanium.

Also of note, IBM has stated that current plans for the G5's SIMD/AltiVec engine specify a 256-bit system, rather than a 128-bit one in the G4. This will be one kickass CPU.

Itanium?

[Bobo the Space Chimp]

Dr. Adams: "Welcome to the Silicon Graphics Tantalus V Itanium research campus."

Stan: "Why did you call it 'Itanium'?"

Dr. Adams: "I have a rare marketing disease that prevents me from pronouncing the first 'T' in 'itanium.'"

fascinating

[Anonymous Coward]

Here's a snippet from his little bio thing:

Belluzzo was formerly chief executive officer of Silicon Graphics Inc. (SGI), where he was responsible for defining and executing a return to growth and profitability for the company

Re:SGI Sucks ( read on )

[Anonymous Coward]

Well said - I agree with most of what you've written, and think that you've put it all very well.

However, I think the reason that SGI is not producing a desktop version of Onyx 3000 is obvious - SGI tried to do battle with Sun, etc. and failed. SGI tried to do battle with Dell, etc. and failed. They're not about to do the same thing to NVidia...

The original SGI targetted what was at the time a niche market - 3D graphics. It looks like the new SGI will also retreat into niche markets - very high-end graphics and compute servers.

You're also right about the Itanium server - there's nothing very interesting about it. I believe this machine is only intended as an interim solution to allow developers onto the platform until SN-IA is available (whenever that might be).

Then we'll see something a little more impressive than OEM Itanium boxes with low-bandwidth Myrinet interconnects!

IA64 & Myrinet

[ShavenGoat]

So you guys know, Myrinet is a 2Gb peak interconnect with a 7us minimum latency. FULL CROSSBAR SWITCH :-) Fiber or serial.

A interesting fact is that up until a few days ago, Myrinet only supported 1 GIG systems. I ran into this while setting up the University of Nevada beowulf named cortex.

I must admit, IA64 with Myrinet 2000 is gonna kick some serious computational ass.

The article says that Myrinet will run MPI, but it will also run PVM and TCP/IP stuff too.

Check it out at their site