Slashdot Mirror
Supercomputer Breaks the $100/GFLOPS Barrier
Hank Dietz writes "At the University of Kentucky, KASY0,
a Linux cluster of 128+4 AMD Athlon XP 2600+ nodes, achieved 471 GFLOPS on 32-bit HPL. At a cost of less than $39,500, that makes it the first supercomputer to break $100/GFLOPS. It also is the new record holder for POV-Ray 3.5 render speed.
The reason this 'Beowulf' is so cost-effective is a new network architecture that achieves high performance using standard hardware: the asymmetric Sparse Flat Neighborhood Network (SFNN)." Because this was a university project, KASY0 was assembled entirely by unversity students, which while being a source of cheap labor, is also a good way to get a lot of students of involved in a great project.
Imagine a Beowu... errr... Oh..
Note to moderators, Beowulf cluster jokes CANNOT be offtopic.
Imagine a Beowulf cluster of Beowulf cluster jokes!
How much electricity will these super computers use up?
All those wires, it looks like it takes up alot of juice.
If you use Linux, please help development of Autopac
gigaflop
As a measure of computer speed, a gigaflop is a billion floating-point operations per second (FLOPS).
you may find the Higgs in this signature.
Supercomputer Breaks the $100/GFLOPS Barrier
Not after you factor in the SCO license fees.
I personally love to see this kind of stuff. As a kentucky native, anything related to my home turf gets extra kudos. On a more technical note, I just want to know why I can't have one of these?
"We don't know what we are doing, but we are doing it very carefully,..." Wherry, R.J. Personnel Psychology (1995)
Not to mention I'm 100% sure you use Windows, and cant even take advantage of the power of that.
How would you use this super computer?
If you use Linux, please help development of Autopac
the older kernels, they are SCOless......
Remember, everyone, this was a university project. *BSD was also a university project originally, and now *BSD is dying. So obviously university projects are not of very high quality.
Obviously, I don't get it. This doesn't look any different than redundant backbones or what is frequently done with VLANs. Multiple paths between hosts is what I see. How is this "new"?
Ponders while there are not University students pictures in the National Geographic Article on Slavery....
but super computers as in giant iron are becoming more specialized and as such would woop the pants off a Beowulf cluster when competing in the specialty.
of course, if you just need a lot of general purpose super computing, it is obvious that you cannot compete with this.
I am the Alpha and the Omega-3
And it was introduced to consumers just a couple years
ago. Sorry, the AMD beowulf cluster at $100/GFLOP just
isn't that impressive.
though is how many mp3's are these students sharing on this monster ?
Do not try to read the dupe, thats impossible. Instead, only try to realize the truth
What truth?
There is no dupe
Sounds good to me, there's too many people around here already. Let's start with the yuppies that think the most important things in life are the status symbols they drive or that every blade of grass on their lawn is exactly the same height.
each node has two side case fans! that's gotta be the most dedicated case modding job i've ever seen! 132 pc's with 2 fans! too bad they didn't put fan guards ... or interior lights.. or blue led's... but i guess all that junk about a supercomputer makes up for it...
The government does give financial aid to some of them, along with scholarships and other forms of payment, so they get to live for free on campus with government money, its kinda like the military deal, you work hard, you get an education, and the government supports you.
In this situation I'm sure these kids, all of them, will win all kinds of scholarships.
(payment)
If you use Linux, please help development of Autopac
Looks like Frink was a bit off:
Well, sure, the Frinkiac-7 looks impressive, don't touch it! But I predict that within 100 years computers will be twice as powerful, 10,000 times larger, and so expensive that only the five richest kings of Europe will own them.
I think working at mc donalds is slave labor, you cannot pay your rent, you cannot buy your food, you cannot survive, thats slave labor. I dont think this is slave labor, students live better than a person who works at mc donalds. Students do get support from the government via financial aid, they do get scholarships, they get stipends, fellowships, and many many payment systems setup to help them.
Basically their full time job is "student" just like your full time job is whatever it is that you do. They do stuff like this and it pays their bills, they live off scholarships, you live off your paycheck, I'd rather be a student however than work at McDonalds, which is my option if I decide not to go along with the "slave labor camp"
So you can be a student slave, or a corperate slave, which one do you prefer?
If you use Linux, please help development of Autopac
Don't let that happen again. You will only harm yourself.
Thank you.
... a beowulf cluster of these!
Where do you get the opinion that a 5.5 giga flop machine costs 199 $?
The Apple machine is a super computer but at a cost of 1940 $/Gigaflop. Single machines cost a lot.
How you get the value of 199 $ is beyond me.
You are very uniformed and shooting from the hip.
and it still can't run Doom III at a decent rate.
--krahd
mod me up, scottie!
mod me up scottie!
It's saturday evening and time to start drinking beer!
There's always use for Gflops. How about distributed DVD ripping, packing and then serving them to the people on the net?
What a mess of cables! I understand they were hitting a price point, but would it have killed them to spring $500 or so for a cable management system?
There's something professional looking about having the cables look neat. On the other hand, maybe i'm just anal about things.
HBI's Law: Frequency of calling others Nazis is directly correlated with the likelihood of the accuser being Communist.
I toured the previous cluster these guys did (KLAT2) and was very impressed. However, using AMD Athlon Thunderbirds last time, it did get quite hot. I remember standing by the cluster looking at all the wiring and being bombarded by an overhead cooling vent. I'm also assuming that these cooling issues is the reason that each case has two blow-holes. I'd also like to see these guys post in-depth specs of each machine. Being a hardware nut, I'd like to see how they got so many machines so cheap, and maybe even what vender they used. As I remember, they worked REALLY hard on their last cluster to keep costs to an absolute minimum.
I'm guessing the latter. You see all sorts of BSified numbers from marketing departments on processors, but they have little to do with reality. The number for this AMD cluster is a real, actual, measured-using-a-real-world-app number. To give you some idea of BS console numbers, the Xbox has a PIII 733 processor in it (ok, technically it's a little different, but it's a P3 core). Now the Gflop claim is 2.93. Out of a P3 733? Ya right, on paper perhaps but never in the real world, much less on a real app.
Then, of course, there is the issue of specialised chips vs normal chips. A GeForce 4 4400 can claim, roughly, 80 Gflops peak. That sure beats the hell out of any sinlge CPU I've ever heard of, including the Power4. Thing is the GeForce 4 is a graphics DSP, it isn't a general purpose CPU. It can do that kind of math when all its units are working at what they do best, but try to reprogram it to do something else and it will slow to a crawl (for that matter I'm not even sure that it is turing complete).
So don't take any hype on a console to equate to real performance in a general task. Oh, and the BS marketing number I see for the PS2's Emotion Engine is 6.2Gflops.
5.5 Gflops, I dunno if it can really do that, but ...uh..the point is that it's the first *supercomputer* to break the $100/GFLOP barrier. The Playstation2, last I checked, isn't a supercomputer, it's a videogame platform.
My journal has hot
Dear customer,
.. Price .. Qty .. Total .. 8,377,500$ .. 128 .. 1,118,400,000$
At the cheap introductory price of 699$ for 80 lines of code in the Linux kernel, it will cost you 8,377,500$ by kernel since we have discovered that in fact 1000000 lines of SCO IP were copied into Linux.
Designation
Linux kernel
So you must pay us only 1,118,400,000$, and in my kind almighty I will offer you a discount of 118,400,000$ so you only have to pay ONE BILLION DOLLAR if you pay before tomorrow!
Please send you creditcard number at darl@sco.com
Sincerely yours,
-- Darl Mac Bride
- Casio sues them for trademark infringement?
- SCO asks them 92268$ worth of licenses and/or sues them for copyright infringement?
- KFK sues em for patent infringement? (you know, "the method to fry one billion chicken flaps (GFLAPS) for under 100$ with AMD processors")
Ah the USofA, Land Of the Lawyers.
I'm a chainsmokin' alcoholic sociopath, so-ci-o-path
I didn't see hard drives on their parts list. Why is that? How do they boot them up?!
Looks like most of the wiring jobs I've seen done by students: kasy0core.jpg.
;-)
God forbid they use cable gutters
Other than that, kick ass job guys!
-nate
These numbers for microprocessors etc mean nothing because they are usually referring to operations on data in cache.. you'ill find that real life performance is 10-20x slower because thats how much slower accessing main memory is.
Note that every PC has at least one single-switch-latency path to every other PC; some PC pairs have more than one such path.
Every host does have at least one pathe to every other host but, most hosts have multiple paths to other hosts. It is true however that all hosts do not necessarily have multiple paths to all other hosts.
In my country there has been a lot of discussion about how "evil" it is that many university level theses that have been done in collaboration with corporations are declared classified (=won't be made public until a few years later).
I just would like to know what is the point in doing research if it does not benefit the society? If classified theses are no longer possible, the business will lose all interest in the academic research and the stereotype of a acadmic researcher will become reality. There should be a healthy balance between applied and basic research.
Hey! I used to work there.
Way to go Dr. Dietz!
So, mod me anyway you want, karma to burn.
Just be sure to wear the gold uniform when you beam down -- you know what happens when you wear the red one.
Nice how you take the numbers from a marketing press release and treat them as if they are the absolute, indisputable truth. Can you show me the actual, reproducible benchmark that produced those numbers?
Also, the PS2 is not a supercomputer. It has a slow processor and very little RAM, so it wouldn't be able to do much number-crunching. You can't hook PS2s together, anyway, so comparing a single specialized machine to a cluster is absolutely meaningless.
128+4...
That's like 132 isn't it?
In reality, beowolf clusters are good for only a subset of supercomputing tasks and the "real" supercomputers are still best at general purpose supercomputing.
If you can paralize your application well enough, beowoulf rules, but if you need a lot of node2node communication, the network cost quickly surpasses the cpu cost of the system
HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
I am a grad student at UK Computer Science dept...
... see http://www.metaverselab.org
They are working on projects to use this type of power
My favorite is the hooking together 16+ cheap ($2000 or so) projectors together adhoc to build a display that covers walls/floor, and combining that with head tracking and video cameras that look for shadows so that other projectors can fill in! This needs a lot of GFLOPS!
I wonder which universities/institutes have larger and maybe cheaper clusters, but just don't bother with running benchmarks. I for one are sitting next next to a tiny cluster with 40 dual-cpu nodes, which is connected (GRID like) to a 340 dual-node cluster in a nearby town. Non of us high ernergy physicists bothers with running any benchmarks on our clusters, other than our own applications. I wonder how many "linux-cluster-supercomputers" are out there which would easyly make it into the top 500, but noone has ever heard of....
Cheers.
KdenLive/PIAVE - non-linear video editing
At the risk of being flamebait- No. Using university students is almost always purely a way of getting cheap labor to do semi-mindless, or completely mindless, stuff the staff doesn't want to do- it's a common myth that students 'learn' by doing grunt work. I should know- I have several grad student friends, and they've thusfar spent a large part of their academic careers working in labs doing mind-numbingly boring stuff(according to them.)
Imagine if a Bio lab did this. The following would sound pretty absurd: "Help us move our lab, you'll learn about cellular recombination!". No. You'll learn what a bunch of lab equipment looks like, how eccentric the professors are, and how expensive/fragile/heavy the equipment is, and the next morning what sore muscles are like. Let's get a reality check here.
(from the site):Our group develops the systems technology for cluster supercomputing; the more people we can show how to apply these technologies, the better.
Huh? What cluster supercomputing "technology" does assembling a PC and plugging it into ethernet teach you? Did they give a presentation about how clustering technology works, for example? Did they explain to each person, as they put a machine in a particular place and wired it to a particular switch, WHY it was going there etc? Obviously I wasn't there, so perhaps someone from the group can contribute on this point.
Please help metamoderate.
I mean these things are Athlons! Heck, they're saving money just from the fact that they'll never have to turn on the furnace again!
Did you guys notice from the pics that there doesn't seem to be any fans in the holes on the sides? Are they crazy? These are Athlons. I hope they put enough fans in those things.
My journal has hot
Now that the university students have graduated and moved on, there isn't any documentation, nor do they know how to use the darn thing...
-1
This price/performance ratio seems to make them very attractive compared to general purpose CPUs. According to the NASA G5 Study, the P4 2.66 GHz is only able to achieve 255 MFLOP/s. And the P4 costs about 4x the price of the 6711 DSP.
It seems that DSPs should be the clear winner in supercomputer applications, what are their disadvantages and why are they not used? Granted there is a lack of mass produced hardware such as motherboards for DSPs, but that alone should not exclude them from the supercomputer realm.
Yeah, good work.
What will they do with all this processing power? Farm it out? Boast? Serve a pr0n company?
http://www.us.playstation.com/hardware/networkada
every day http://en.wikipedia.org/wiki/Special:Random
"Insightful"?!?
Yeah, right.
Guess everybody from universitys to big corporations spend tons of money for supercomputers/clusters because they have no use for them.
Sounds very likely.
Given the fact that your other posts in this topic all concern the blackout or McDonalds->
MOD PARENT DOWN (OFFTOPIC)
HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
Nice machine, but this January, CITA and the astro department at the University of Toronto brought a 256 node dual Xenon system on line: "1.2 trillion floating point mathematical operations per second (Tflops) on the standard LINPACK linear algebra benchmark." Total cost: CDN$900K (including tax) (in January prices, that's $600K U.S. or $0.50USD/GFlop.) It's being used for some very cool Astro simulations...
See http://www.cita.utoronto.ca/webpages/mckenzie
KASY0 nodes are completely diskless; there isn't even a floppy. (from the FAQ)
So how are the nodes booted? Are there bioses out there that can netboot?
-c
"If you are an idealist it doesn't matter what you do or what goes on around you, because it isn't real anyway."-R.P.W.
The key word here is supercomputer. A PS2 is not a supercomputer.
The previous price/performance champ was in fact a PS/2 cluster, mentioned here, but this AMD cluster is roughly three times the performance for the dollar. You can check the stats with different assumptions on their FAQ page, particularly the section labeled 'Is KASY0 really the first supercomputer under $100/GFLOPS?'
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
Not to downplay the fantastic accomplishment, but there is nothing new about this network architecture. Not topologically, as Dietz has been claiming for years now. When did a mix of full and partial mesh suddenly become new?
:D
Sweet cluster, though
Huh? Isn't the obvious solution to use rear displays?
Gah feel free to mod the previous version of this comment into oblivion, I hit submit accidentally.
The numbers you're looking at are marketing numbers first off, and overly generous. Second you don't scale for free - you never get anything like 100 times the performance of a single box when you wire 100 together, for the same reason that you don't get twice the horsepower out of an engine twice the size.
The previous price/performance champ was in fact a PS/2 cluster, mentioned here, but this AMD cluster is roughly three times the performance for the dollar. You can check the stats with different assumptions on their FAQ page, particularly the section labeled 'Is KASY0 really the first supercomputer under $100/GFLOPS?'
=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Friends don't let friends enable ecmascript.
Did you read the article? Whats off topic?
Just because we have supercomputers does not mean we have the software or AI to take advantage of its power. Alot of people build junk just to say they built it, especially college students.
Mods, please mod the poster above as troll.
If you use Linux, please help development of Autopac
It's not the first time that these folks in KY work around the definition of the acronym "Flop". A Flop is a floating point operation on 64 bits, not 32 bits. All entries in the Top500 used results with 64 bits HPL, nobody else in the world is running HPL on 32 bits. So claiming the moon on 32 bits is easy, useless for the sake of comparaison and almost unethical. I cannot believe that Dr Dietz do not know the difference by now.
The same machine would yield average results on 64 bits. Difficult to draw attention without headline numbers...
... they're going to have the largest Quake LAN party ever!
Come on, 1 billion dollars on a credit card? Have you ever heard of a chargeback?
On a serious note I'm sure Paypal would be more (sic) secure and convienient.
(Those familiar with the University of Manchester's Department of Computation, in the UK, will understand what I mean. The architecture is designed around the computer room. Even after the truly massive lumps of iron were removed, it still wasn't until the mid 1990s that the building had a ground-floor entrance, as such an entrance would have limited the heating effect.)
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Further, it would also accelerate the product enormously - Linux on a Chip would be blazingly fast, as it wouldn't take any processing power away from what it was running - thereby also reducing the cost per GFLOP.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
But if they cleaned it up they couldn't use it as a hammock.
"Sic Semper Tyrannosaurus Rex."
For the record, the Metaverse Lab and Hank Dietz have nothing to do with each other.
I can't tell by this poster's text if he means that they do or not, or is just talking about the University of Kentucky's CS department in general.
that's not always feasible, it's often more expensive, and there are problems with back-projection that are not present with front-projection
if you're back-projecting, you better have a nice surface that allows light through it (walls won't allow this, while front-projection is fine for walls)
there's also the "hot spot" problem with back projection where if your eye/camera/viewing device/etc is looking directly into one of the projectors, you'll see a hot spot...much much greater intensity from that angle
the biggest problem with front projection is occlusion by the observer...and the Metaverse Lab has solved this problem with shadow removal using multiple projectors...the unoccluded projector kicks in and shows the piece of the immersive display that was occluded by the observer
Granted there might be some heat problems, but judging by their setup, I'm guessing the room is well-cooled.
The sending of this message pretty much inconveniences everyone involved.
But the "FA" says $1000 per gflop not $100
Did you RTFA?
210A at 120Vac via the power law comes to 25.2kW, not kW/h (whatever the hell kW/h is supposed to stand for).
Users with IDs over 100 thousand shouldn't be allowed to post.
Besides, nobody in their right mind would run a parallel program of any importance on a "rigged" setup like that.
DEAR SIR...
My name is Mumfasa Thumbutu, and I am writing to you from Nigeria. I work for SCO, and here in my country, we have many SCO license fees to collect. But due to our bad government we cannot put our money in the bank like normal people. See, everyone in Nigeria is corrupt, except for me and my friends (and SCO, too).
So we need your bank account number, social security number, mother's maiden name, date of birth, keys to your car, wallet, and your computer so that we can put the sum of ONE BILLION DOLLARS into your account.
This is not a scam, you see, because we are working for a reputable company (SCO) who has a lot of money.
SINCERELY,
MUMFASA THUMBUTU
(I tried posting this in all caps, but was thwarted by the lameness filter)
I work @ UK and the students used were on a pure voluntary basis
projects @ http://spectechnologies.net
India breaks $9.99/GFLOPS barrier, using a team of veteran network engineers.
A kW*h is the unit you talk about. It makes no sense to talk about kW per hour (= kW/h), since kW is a unit of energy. It's like saying "kiloJoules per second per hour". Just senseless.
The rest of your post is just senseless. A light bulb uses an amount of kWh per day (or Joules, if you want to use SI units), not kW. And so on, and so forth.
I meant kW is a unit of *power*, not energy.
What a shame. Freeloaders. They would never be able to achieve such performance if not for the fruits of labour of SCO .. eeeh.. lawers?
<^>_<(ô ô)>_<^>
Er...you can do that with parts from ebay or craigslist without too much trouble.
Doesn't it make you feel good to know that our freedoms are protected by politicans, lawyers and journalists.
Nice end example. Indeed there are many factors to be considered in what architecture to use, These systems lower the cost and provide higher processing power for smaller, low budget, projects. I see great potential for these clusters to bridge the gap between grass root and high end projects.
Yes, beowulf clusters have amazing potential, bringing high end computing to the masses.
The site said that one year of power for the cluster would cost about as much as all the hardware cost. So I guess if you include power, it certainly doesn't beat the $100/GFLOPS barrier.
Sig: I stole this sig.
It's interesting to note that the this cluster of 128, 2 GHz processors is only about 9 times faster than a single 1 GHz Itanium 2 processor at performing the PvRay benchmark.
You have to include a people time, building overhead etc. A reserach grant may be billing $500 - $1000 a day for this. If this takes 50 man days to set up, then the cost is is another $50,000.
that shelving they used looks almost identical to the kinda stuff i have in my bedroom, unfortunatly my shelves are not filled with a supercomputer (yet)
They even have a tool on their site to help you wire it up! You can buy the hardware at any computer shop. Of course buying several hundred PC's at once and multiple switches takes a pretty heft credit limit! --Oh and you need to figure out how to power several hundred dual Athalon toaster ovens from your meger 200 amp service! But no one's stopping you. I think I'd be cool to do this to an office of PCs. After all an extra $50 per PC for a hobby project might get by the PHB.
we worship thee!
Slavery-ha, If I was available [unfortunately I'm in michigan] I'd be there for free too! Besides college students have fewer rights than slaves--every one knows that...
(for $39,500 you can buy only 10 g5!)
The students are actually PAYING to be their University's slaves. I guess that makes it legal, if somewhat masochistic.
Side note: I wonder if my postgrad supervisor would make good dominatrix...
Forget thrust, drag, lift and weight. Airplanes fly because of money.
Okay now it's time to add in the licensing fee to Neal Stephenson for using 'metaverse'.
Maybe he'll go easy on them if they claim to be running finux...
What're the extra 4 for? They don't mention it. Hot spares? Command + control? Scheduling?
(S+C) x (B+F)/T = V
I guess you have to pick a more interesting article on the same topic or know someone. :(
OK, so they're not off-topic - they are likely to be "obvious to the skilled practitioner" :-) And yes, I'd mod myself down, but I'm posting it as AC...
Now, there's the subtlety that they're doing SIMD stuff using the 3D-Now vector processing, which may run a bit hotter than regular random behaviour, but I don't know if it really does.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
They did comment that the machine is located in the basement and doesn't have any windows, so it's dependent on the building's cooling system. Otherwise they could have taken care of much of the heat load by simply blowing the hot air outside, at least if they can tolerate Kentucky summer heat/humidity ambient conditions, which is a lot more efficient than actual airconditioning.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
http://slashdot.org/comments.pl?sid=72589&cid=6558 445
Sorry for the sloppy reporting; posted in a followup the citation.
...in the W. T. Young library on the U of Ky campus here in Lexington. Anyone in the area who wants to grill him should try it the old-fashioned way.