Slashdot Mirror
Cray for Sale - Cheap - Some Assembly Required
"For quick sale: Supercomputer of distinguished pedigree, a CRAY Y-MP C90, one of the world's most powerful only seven short years ago, an extremely reliable workhorse for R&D computing. Get the jump on your competitors! Bring this black & gold beauty home to your research center or lab today!" Read the story or place your bid. ...
If you've ever wanted to go to the top of the distributed.net stats, here's your chance.
I went over to Penguin Computing and priced their eight-Pentium-III rackmount server with all the trimmings. It came to $70,000 for a system that comes surprisingly close to the Cray in power.
The Penguin Computing system has 8 Pentium III 550 MHz processors (with 1MB L2 cache each). If you were buying a real system, you'd probably try to use the much nicer Athlon series, but you'd certainly get something closer to a gigahertz from either AMD or Intel. Either processor has multiple pipelines, so you're likely to get more than 1 FLOPS per MHz of processor speed in optimized applications. You could expect perhaps 0.6 - 1.2 GFLOP per processor with this system, or 1-2 GFLOP with a system that used higher clock speed (Athlons might deliver higher pipelining multipliers?). Multiply by eight processors, and you find you should be able to expect 10-16 GFLOPS out of the rackmount server. The Penguin Computing machine has a fully cross-linked bus, so RAM bus contention is probably about equally problematic between the two machines. The PC machine has 2GB of memory (compared to the Cray Y-MP's 4GB), but this may be extensible (at ~2K/GB). The Magnus has dual 75GB hard drives, easily matching the Cray's disk space, but it also has a gigabit fiber link and a kickass graphics card, which the Cray lacks.
So the rackmount Magnus system offers comparable performance to the Cray, at twice the initial bid for the Cray (a reasonable estimate for the final price).
Now let's consider additional costs. The Cray requires that unspecified pieces of paper be signed to satisfy the U.S. government, adding unknown bureaucratic cost. The Magnus does not. Shipping and installing the Cray will require thousands of dollars. The Magnus requires about a hundred dollars. The Cray requires special power and a cooling system. The Magnus dissipates about a kilowatt and hence can get by with an extra HVAC vent. It does notrequire special power. The Magnus, dissipating about 1kW of power, would require perhaps $60/month for electricity; the Cray, with at least a factor of 10 more power, requires at least $600/month (plus the added cost of cooling the dinosaur pen).
So the Cray sounds fun to get, but (surprise!) it just doesn't stack up once the inconvenience factor is added in.
I suppose I shouldn't be surprised. Even an I-Opener is in the same ballpark these days as a VAX 11/785 -- at least for memory and raw FLOPS.
Hi, I actually work at PSC (the group selling the system) and I have to say its a damn fine computer. You should all buy one now. Seriously though, this is not a linux box and the possibility of porting linux to it would be a daunting (if not impossible) task. I'm also not sure that the current set of licenses for the OS and other tools found on the machine are transferable. You'd also have a devil of a time installing this. The power requirements are outstanding. Not only do you have to deal with the power draw, cooling issues, wireing issues and so forth... You would also need to rewrite most any program you wanted to run on the sucker if you wanted it to use the full capabilities of the machine. How handy are y'all with FORTRAN? And yes, there is a good chunk of precious metals in this box in the work of wiring and connectors. It would cost much more to extract them then the value of the metals. And no, we can't ship this outside of the United States and I don't know about any restrictions on sales to foreign nationals. Lastly... Quake? What the hell are you smoking?
We used to have a Cray Y-MP where I work... One little detail that has not yet been mentioned is that a Y-MP is water cooled. So, anyone without their own heat exchange unit and/or chilled water suppy need not apply ;-) Also, the thing used about $20,000 of electricity every month. Yipes!
Imagine a Beowu--ok, I'll stop.
--
Actually, you are right, you do get a wholesale rate. Part of the requirements of running one or more Crays, as I have done, is to have sufficient backup power. We had 12 battery backup units, each with 32 car batteries. That would keep the Crays running for 4 minutes. After that the 3 12-cylinder deisel engines would be running at full speed and supply power endlessly.
When we had hot days in Minnesota, the local power company would call us and ask us to switch to deisel power so they could use their energy to power everyone else's air conditioners. In return, we got a major discount on power.
Note: the 4 story building takes up a whole block, but has no furnace. Even in the dead of a Minnesota winter, the Crays heated the whole building including the indoor parking ramp...and we still needed to vent heat out via fans.
ASCI White is the only machine doing 12 TeraOps. The next fastest is a trio of 3 TeraOp machines at each of the 3 ASCI centers (LLNL,LANL,Sandia).
As fast as 12 TeraOps is, its still far, far too slow to do real 3d, first principles physics simulations that involve a wide range of scales (which most real problems do). When they get to 1000 TeraOps, we'll be getting somewhere.
Xunker is right, if you have a RISC-based AS/400, i.e. a modern one, it is using PowerPC under the hood.
:)
IBM has recently committed to porting Linux to the AS/400 platform (not to OS/400, the operating system) - this will be done very much the same way they already ported AIX, i.e. you'll have Linux running directly on the PowerPC hardware, with some extra software to arbitrate between OS/400 and Linux where needed, and handle Linux-OS/400 communications.
Should be very interesting when it's all done - every single IBM platform, including the most proprietary AS/400 and S/390 ranges, will be able to run Linux apps, and porting should be relatively easy providing apps are not byte order dependent etc.
Ironically, IBM tried in the 80s and 90s to provide a way of easily porting apps across all its different OSs and architectures, through a set of development tools and APIs known as System Application Architecture (SAA). They failed. It would be rather cool if Linux solved this problem for them
Linux on S/390 is there because they want to port e-business apps to all their existing platforms, and because they understand Linux is going to commoditise operating systems. Unlike many other vendors, IBM really seems to 'get' Linux.
One useful application for S/390 Linux IMO is hosting very large numbers of dedicated-server websites on a single box - every customer gets their own virtual Linux machine, hosted on VM/390, which has 30+ years of tuning behind it.
The customer gets more power and flexibility, and the hosting provider gets more maintainable and reliable hardware, rather than having racks upon racks of Intel-based servers that will inevitably have frequent individual box failures, even though most of them are available at any one time. The only flaw in this argument is the cost of mainframe hardware and system programmers, but if you host enough websites it could be worth it.
Last week egghead was auctioning off a group of IBM AS400's. Okay, it's not in the same league with a Cray, but still... when I looked in on it, they were going for $9. For a moment, I toyed with the idea of getting one, but then I saw 2 gotchas...
1. Shipping was $185.
2. Operating system NOT included.
Since they're weird RISC boxen, God knows if they'd run anything but OS400, which would probably cost several thousand dollars.
Garg
Garg
Alumnus, Xavier's School for Gifted Youngsters
Well this is kind of a good idea, but you will be happy to know that it has already been done.
The US government spends millions of dollars on supercomputers every year. Some of the computers are for Classified projects, but many of them are for research purposes. These research computers are like a national computing resource. You paid the tax money for them and, if you're so inclined, you can probably use them. If you think you have a project that would benefit from a supercomputer, you can apply for time on one.
If you buy your own Cray you'll be guaranteed time on it, but you'll also be burdened with maintenance and upgrades.
Try these links:
NPACI Allocations
www.sdsc.edu
www.ncsa.edu
That means only 8 G4s to equal the computer power of the Cray
Apples and Oranges (sorry about the pun). The bandwidth, bus-speed, memory size/type, etc. of the Crey will destroy any workstation you could throw at it. If all you're doing is SETI, go for the G4's. If you are doing scientific high powered computing, there is no comparison.
Sometimes I've believed as many as six impossible things before breakfast.
I work for a group that could actually use this box. But, alas, while we could afford to buy and power it, I don't think we could afford to pay for the setup costs.
And I know for a fact that I can't administer it!
Ok not thies... but the much older Cray 2s... back when they were new and such a setup did not yet exist.
War Games.... The computer in the book was 7 Cray 2s.. They scaled it down to one larg black box in the movie...
(I guess the tech of 7 Cray 2s wouldn't impress the non-tech movie going public... my mother thought it was some sort of joke to use one larg black box sence she recognised it as a mini-mainframe.. she not being any sort of tech felt anyone would recognise this if she could.)
Anyway... Yes someone accually thought of doing a cluster of Crays... a bit before anyone accually made such a system...
Now imagin Wopr going HAL and trying to pull a Forben project...
I don't actually exist.
According to this article the original cost of a Cray Y-MP C90 was $30.5 million.
Some specs from utk.edu :
Apparently, today's fastest supercomputers are at about 12.3 teraflops! Still, I bet the power bill on the C90 still packs a punch! (But at least you won't need a heater in the winter!)
---
In a hundred-mile march,
I ran the original distributed.net RC5 client on one CPU of an 8-cpu C-90. It got about 85Kkeys/sec. Yes, that's _85_. Not 600+ like intel processors of the time get.
C-90's do vectors. They don't do integer work. vi? slow as a dog. emacs? slow as a dog. CFD simulations? it'll knock your socks off.
Most people were very surprised when they would log in and see how _slow_ it was running a shell. Think 2-3 seconds to get a response to the 'date' command.
Would you use a sprint car to go to the grocery store? Wouldn't get you there any faster than my Geo Metro, would it?
C-90's are incredible machines; the details would fill you with awe. But given that there's no hardware documentation available, and hence no OS's other than COS and Unicos for them, and limitations like _no MMU_, it's really only useful for batch processing of vector work, i.e. floating point.
It'd make a kick-ass rendering back-end, if you could get someone to write the software. Otherwise, leave it alone.
--plambert
PSC is one of the biggest supercomputing centers in the world. They were just awarded a $45 million dollar NSF grant (As Seen On Slashdot) for a new array of computers. PSC is no gamble! The ebay thing is just for publicity...
The non-obvious thing that you're probably missing is the shared, multi-port memory. For some classes of problem you absolutely have to have a single large bolus of memory for the crunching to work well. Progressive problems like 3-D fluid simulation require that each processor work in more-or-less parallel on each timestep, but be able to interact with data that's handled by the other ones. That's the weak part of distributed computing systems (which have higher latency and lower transfer rates than dedicated systems with shared memory), and it's why I didn't consider the bare-bones Beowulf solution. But, yah, for many problems the bigass collection of cheap workstations really is the best way to go.
Could somebody explain to me why you would *ever* run emacs, vi, or the like on one of these babies? Wouldn't it have been far more efficient to have cross-compilers and the like, and only ever run code that the Cray was built to run - cracking Russian cyphers, forecasting weather etc. etc.?
Any sufficiently advanced technology is indistinguishable from a rigged demo
--Andy Finkel (J. Klass?)
Also, if you want a serious cluster, you're going to want faster interconnects than 100bt. For what it's worth, you can get 1u rackmount nodes with Pentium III 800s and gigabit ethernet from Penguin Computing for about $2000.
--
It looks like this would be about comparable to your average O2000 SGI. These systems go for a lot more than 35k. The problem is that research grants take much longer than 5 days to go through, and even if 100k was allocated for an SGI, the department can't just change the wording of the grant. Also, research departments like to get thier hardware from official sources. Getting something on Ebay would be considered a gamble, and one doesn't gamble with 35k of thier research group's money. I don't think ebay is the right medium to sell this system. It will probably be bought privately for much less than it's worth.
Sometimes I've believed as many as six impossible things before breakfast.
There real question here is, does the seller except payment by paypal? ;)
Vote Technocratic! Government by killer robots!
In addition, the Cray is going to push a lot more stuff between disk and memory than that G4. Not important? Well, try dealing with several hundred gigabyte files constantly and disk performance becomes an issue.
Basically, the Cray is designed to run at or near peak performance constantly. The G4 (and all other personal computers) are not.
Also, it takes 8 G4's to equal a Cray from 1991. That's pretty good for an almost 10 year old machine...
Go Badgers! -- #include "std/disclaimer.h"
Does it run Seti-At-Home?
Gonzo
-Dave
Please try to get 8-way SMP Intel boxes to do any serious inter-process communication. They suck! memory contention issues and bus-speed (not to mention sucky Linux SMP) make 8-way (and actually most 4-way SMP) machines *much* slower than 8 single-cpu Pentium boxes. The best bang for the buck is probably a cluster of dual PIII-Xeons with the new 133MHz buses.
Yes, you can kick a Cray's ass with that thing, but a) you are gonna need some fast (gigabit speeds) iterconnects that aint cheap, and b) you need software that will take advantage of the setup. And still, if you re-write your software to be vector-friendly (not trivial, but not the hardest thing in the world either) a Cray can still kick some serious Beowulf butt.
The Cray could be your dorm room.. With the fans that thing must have, you'd always have cold beer around. And when chicks hear of your 16 gigaflops, they'll be lining up.
If the OS license is locked to the box (as I understand it is), how could an AS/400 exist without a license?
I'm really curious, since I'd love to own an AS/400 just for the sheer heck of it -- but I'm sure the price of getting a license would be prohibitive on, say, one of the older $200-500 machines.
D
----
I always just wanted the case.
It would make a cool couch/refrigerator combination! No living room should be without one.
The real thing would eat up my entire paycheck just in power usage alone. Might help heat the house though...
"Trademarks are the heraldry of the new feudalism."
Many leases don't let you have waterbeds because of floor loading issues. Now, think for a minute about the floor loading of a CRAY YMP C90. Hint: I've never seen one installed on a 2nd floor. They're mostly in basements.
Maybe the G4 can theoretically achieve 2 gigaflops, but I challenge you to prove it to me on a useful application. I'd be proud of you if you could get 25% of that number.
The main reason you would want this instead of a G4 is that it's memory is much faster than a G4's. The C90's main memory (4GB) is like your G4's cache (not 4GB) in terms of speed. Even when your faithful CPU meter reads "100%", your CPU is often idle. It has to sit and wait for memory requests which take quite a while. The cache can only do so much.
It is true that this C90 is getting old and most of them have probably been decommissioned. However, I am sure that there are still C90's out there in use and they can't be replaced by Macs.
1 * Cray = $35,000:
- 4Gb RAM
- 16 vector CPUs
175 * PSX2 = $35,000:
- 5.6Gb RAM + 700Mb embedded VRAM
- 175 CPUs
- 350 vector units
350 PSX2 vector units could process 1400 floats per clock, and they run at 150MHz. That's some serious gigafloppage, and it's not even using the rather capable CPUs.
Just a comment on the distributed.net comment written by the Slashdot monkey. I've compiled the stock testing core on the C90 at my work. Single processor, it did something pathetic... ~12kkeys/sec. The Sparc-1s we have do better than that.
Yes, if someone wants to try vectorising the core and implement MPI (rather than running 16 seperate clients at once), it might do better. But I doubt it. These are built for math, not bit-shifting.
On that note, the Seti client, I could see doing a LOT better. Again, after vector optimisations. But you can't get any code for that client...
> Its name is Mario, after Mario Lemieux of the Pittsburgh Penguins,
Irony I'm sure...
I suspect Tux didn't exist yet when Mario was named...
Still one must suspect if Tux is a sports fan this would be his fav team...
I don't actually exist.
Because, based on the other messages on this thread, it would be worth absolutely nothing to them. It's a specialized machine designed to solve specialized problems.
If it costs $ 35,000 to de-install the system, a church or charity couldn't afford to take it in any event.
D
----
According to a (rather old) PowerPC FAQ at http://www.motorola.com/SPS/PowerPC/library/ppc_fa q/ppc_faq.html, the skinny is:
"IBM uses a custom 64-bit PowerPC processor -- the PowerPC AS -- in their AS/400 business computing systems."
I, too, own a Tracker (hey, I just turned 18, so flah)...as if the thing doesn't roll easily enough already.
Barrel Rolls = BAD
Barrel Rolls w/ a Cray on da Hood = REALLY BAD