Slashdot Mirror
Virginia Tech Announces Supercomputer Plans
CousinVinnie writes "Previously noted in this Slashdot story, the administration of Virginia Tech has announced they're puchasing 1100 G5's (another story) in hopes to build a top-10 supercomputer by October 1. Tech will be spending $5.2 million over five years on the project, which should help it pull in more research money." Maybe VT can use the new computers to beef up their web site.
Does anyone know who else was considered for this contract? I'd love to see the arguments for the different platforms!
-WS
An operating system should be like a light switch... simple, effective, easy to use, and designed for everyone.
Burns: [throws his glass at Homer]
You call this Postum?
[bashes a 5-feet high pile of paper]
Burns: You call this a tax return?
[bangs a CRAY with his cane]
Burns: You call this a supercomputer?
The comparison is like Apples to Oranges. Most people end up asking "Orange you going to build a beowulf cluster of those Apples?"
"Learning is not compulsory... neither is survival."
--Dr.W.Edwards Deming
Maybe Apple will use this G5 cluster against a single-processor itanium to show that, yes, they ARE the fastest personal computer!
The only problem will be finding a desk big enough to fit the guys...
So far we've seen that it's a cluster and what the building blocks are. What's the interconnect? What's the OS? What are the nodes using for a network filesystem? Are they at all? Is this intended for parallel jobs or for embarassingly parallel work?
Do you know why the road less traveled by is littered with the bones of the unwary?
Wait a minute - you're complaining about the cost of a G5, but go on to suggest they buy a Myrinet, a rather expensive interconnect. Something doesn't compute here.
Slashdot summary:
1) Itaniums are for pussies.
2) Go Apple!
3) Opterons still kick the G5's butt.
4) I can't wait to run doom3 on my backordered G5.
5) People griping about apples proprietary hardware and software, and how this cluster could have been built cheaper from oem parts, and ebay ethernet hubs.
6)Dumb lists summarizing other trolls.
Here's the article from which the Collegiate Times article has paraphrased: http://www.technews.vt.edu/Archives/2003/Sept/035
Oh nos!1!! The supercomputer will be all loud and stuffs! Whatever shall we do? It will be a pain to play quake on it and ecverything. A monumental failure this.
Post suggestions here!
The comment has already been made. Let's move it along people. Nothing to see here.
Why aren't they waiting for the Xserve update? Rhetorical question, but still...
I haven't seen one, but it looks like the PowerMac G5s are about 4U wide. 1100 x 4U = 4400U / 42 per rack ~= 105 racks.
Not only is this going to take up an enormous amount of room, but the power and cooling requirements are going to be crazy as well. And they don't have rails so getting them in the racks, and working on them once in the rack, is going to be a PITA.
1100 G5 Xserves would need only about 25 racks. Many fewer UPSes and A/C units to power in each rack. Much easier to install and work on.
I know Apple is gung-ho about this validating their "Fastest PC Ever" claims. But it seems a little poorly thought out on the University's part even if they got a sweet up-front price on the machines. Remember: the system price is a small part of TCO.
obviously no deficiencies vs. no obvious deficiencies
Anyone get the feeling that Apple might be pulling a Be, Inc and is trying to pull off a focus-shift?
Remember Be, the "multimedia" OS turned "Internet Appliance". Remember the death of Be. (damn, that stings. I miss the BeOS.)
Now witness Apple:
For decades, seemingly the darling of the press-production (DTP) world, catering to artists of all magnitudes, it was the computer you used to create real, bona-fide art. It attracted the freaks, the hippies, the art chicks. For many people, this was unnerving. Different people get "different" looks.
Now who's Apple targetting?
With OS X, I'm thinking geeks. We're different people, too, but in a, well, different manner. Instead of the artists, Apple's going for traditional suits, the realm of IT. It may be a matter of sheer survival that Apple penetrates here, because they don't stand a chance in these days of "homogenous" work environments.. Out with Apple (even if it works) and in with Dell WinXP machines! Linux faces the same dilemna, although Linux has some other benefits/detriments for it's widespread adoption. If Apple can show it's worth in the server room (just like Linux is doing), then maybe, just maybe, they'll start looking at Apple on the corporate desktop (just like Linux is doing).
Now, the idea of catering to suits is somewhat.. frightening. The whole damn market is different. They don't care about "look and feel", they care about numbers (see economic downturn, outsourcing to India, massive layoffs, H1B abuse, etc). This means Apple will have to change from being "cool" to utilitarian. But wait, I think I just painted myself into a corner here... Wasn't that the point of Apple? To be a tool and not an obstacle? Instead of creating computer art, we're now creating databases? Maybe Apple is on to something here...
If you were me, you'd be good lookin'. - six string samurai
I'd love to see the arguments for the different platforms!
I think the argument for G5 came from here.
Belief is the currency of delusion.
to get any reasonable super-computing power out of it, you would need a low latency (expensive) interconnect.
Well, that very much depends on what type of computing you're doing. Some scientific computing is more tolerant of high-latency environments and would rather have the bandwidth.
I can't seem to find the quote from any of the articles right now, but VT is planning on using an Infiniband interconnect from Mellanox. While I don't know the relative price points, they are touting the fact that this is a high-speed interconnect that's faster than Myrinet or Quadrics at a fraction of the cost. I can't say for sure, since the Infiniband cluster we're helping to build at Stanford is not yet assembled.
This should be interesting to watch. I'll be very interested to see the $/gigaflop ratio for VT's cluster (though that doesn't have a bearing on the interconnect).
Can someone explain the " Maybe VT can use the new computers to beef up their web site" comment? It loads perfectly fast for me. It looks pretty good. It even runs PHP, so it couldn't be a "They shouldn't use ColdFusion" type remark.
Am I missing something, or was that just a completely random comment?
________________________________________________
suwain_2
Jokes like this can get you put away in the punatentury for a very, very long time.
-----
jonathan barket
There are a bunch of people posting gripes that this was a bad idea. But I don't think it's that bad. We should at least withold judgment until we see some data. One thing's for certain, it will outperform YOUR cluster.
Among the top complaints were:
You could buy several AMD's for that.
You might be able to, but the G5's they are buying already have 2 very good processors. As long as they're dividing up tasks among processors, it's nice to have all the memory management and overhead taken care of at a level of two processors per node instead of one. To be honest, I've never seen it done before, and it could have very interesting results.
The Mac's aren't designed for this sort of thing.
We don't know all the details of this cluster because they weren't all mentioned in the story, but my hunch is that Apple might cater to them a little if they are offering to dump $5 mill on a cluster. They might package the cases differently (sans curvy plastic or with shared power supplies).
Anyway, when it comes to speed of high precision calculations, the G* chips have proven their worth. And most High Science applications fall into that range of operation. We all know that clustering and distrubuting is touchy. The cost and speed don't scale linearly. And the cost vs speed ratio definately doesn't scale literally.
There is a possibility these computer science professors know something. So we might want to see how this thing performs before we rush to judgement.
Slashdot Syndrome: the sudden, extreme urge to correct someone in order to validate one's self.
Lots of speculation and rumor, too, if you're into that sort of thing.
This is slashdot! We're all about speculation and rumor. Innuendo, too, especially on the weekends.
Oh, and sentence fragments.
I want to drag this out as long as possible. Bring me my protractor.
Lots of "WHY?" questions, with lots of pointless trolling on the G5; but none of them actually look for answers. Mostly just more idiots who can't understand that a good vendor is important; that their own time is important; that ease of use is even more important now than it ever has been before. Luckily, these same idiots spend all their time setting up sendmail over their 14.4 modem. As for the G5, here are some strongpoints for it: - A fast memory pipe (1GHz) - Good heat management (9 fans but it's quieter than its predecessor) - Damn good FP performance (To get comparable FP performance on intel, you have to use the -fviolate-ieee flag on gcc, think about that) - Vendor-installed, vendor-supported Unix, with the vendor employing the entire OS's development team. - Fast system interconnects with network & I/O - Easy system setup (this matters a lot when you've got 1100 of them) - Proven apple reliability (and if you're going to fight this one, have something better than "is not!") (again, very important when you've got 1100 of them) Oh yeah, and OS X. Mach microkernel, Rondezvous, and distributed builds in the default toolset. Again, the idiots I mentioned above wouldn't have a clue about this stuff. As for _why_ VT getting this, VT's one of the largest engineering schools in the country. We've gotta simulate airflow over wings, heat propogation over materials, and other stufff this CS major doesn't understand. And we've got big development in bioinformatics. All kinds of CPU to crunch. AFAIK, the cluster's being paid for by federal grants or something like that. And now fools, flame me. Prove me right.
Care about electronic freedom? Consider donating to the EFF!
the fan configuration will make it extrordinarily loud
Apple specifically engineered these systems to be quiet - the compartments are set up the way they are so they can get maximum airflow with minimal blowing. Just because you think "loud" when you hear nine fans doesn't mean they're actually any louder than anything else. You're spreading FUD.
it's built to cater to the end user, not to the embedded machine market.
Perhaps your definition of "embedded" is different from mine, but I'm pretty sure it doesn't fit this application. This is a supercomputer cluster. However you are correct that these machines were designed to be desktop computers. Apparently that's not all they're good for.
Many companies build physically smaller machines that still pack a lot of power,
Yeah, so does Apple, but these are faster.
or sell parts to allow someone to design their own layout in a chassis.
If Virginia Tech wants to order 1,100 of them, don't you think Apple would be flexible if this was a concern?
Remember, individual cases, power supplies, and the like become way overkill in such a large computer, and it would probably be cheaper to convert electricity once for a large section of the computer, supplying 12v, 5v, and 3.3v without each computer converting itself.
This is an interesting point I hadn't considered. Feeding 110v into each of 1,100 individual power supplies can't be as energy or heat efficient as what you describe. However, it's possible that they will actually be doing this - I don't think I've seen it mentioned anywhere.
Another consideration - apparently VT was pressed for time and they needed something that would be available quickly; Apple was able to deliver quickly. This may explain why they'd be more inclined to use stock off-the-shelf boxes instead of something more customized.
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;
Myrinet eats CPU when sending data?? You must be using the suckiest driver and firmware possible. And Infiniband has lower latency than Myrinet???? Infiniband is a combination of protocol and hardware and actually Infiniband has slightly higher latencies than the best MPI implementations on Myrinet. Myrinet is a just a piece of hardware. You can write firmware in Myrinet to do almost everything in the Lanai processor present in the card itself, without consuming any CPU cycles. The performance you get out of Myrinet entirely depends on the libraries you are using.
Apple has won other contracts by being able to act quickly. When the Postal Service needed to set up an intranet, everyone said one to two years. Apple using WebObjects was able to do it in 6 months. At the time the Postal Intranet was the largest in the world.
photosMy Photostream
As it turns out, this is the minimum recommended system configuration to run OS X 10.3 Panther.
Mainframes have one job: to move data from point A to point B as quickly as possible, while doing a relatively minor amount of processing on the way. Mainframes are what you use when you want to process every ATM transaction that happens around the world, all at the same time. In fact, your average mainframe is not really any more powerful than a dual- or quad-CPU Intel server, raw processing wise.
Supercomputers are the exact opposite. They're stacks and stacks of CPU's that process largely independent chunks of data. They do huge amounts of processing on each chunk of data. They do *not* move data particularly well. In many cases, supercomputers are held together with Gigabit Ethernet. That's not exactly *fast*...
Different computers, different tasks.
Linux IT Consulting and Domino Development in Michigan
Read why here.
That that is is that that that that is not is not.
It is a troll because the money for the supercomputer came from a NSF grant for that specific purpose. Furthermore the university expects to make a five-fold return, as have most universities in the top-x supercomputers.
Have you ever been to VT? We've got construction going on all over the place. The football stadium is about to get another "upgrade" after having received on just a year or two ago. We've got major construction going on in at least 3 different places, not to mention many smaller construction projects.
Meanwhile teachers are getting let go, classes that were taught in 30-person rooms 3 years ago when I started, are now taught in 400+ person lecture halls.
Does it suck? Certainly. However the money for the construction projects, football stadium, and supercomputer are all from grants, donations, and other means intended for a specific purpose. They can not legally take the money from a supercomputer grant or football stadium donation and use it to pay a teacher's salary.
We have uneducated rants in the school paper at least once a week saying "why are we upgrading the football stadium if we cant pay teachers!@#$"
Yeah, it does suck, but the university has no choice in the matter.
Or so they claim here. It seems they have all their bases covered and don't give a damn about ECC for a reason.
[Srinidhi Varadarajan, an assistant professor of computer science at Virginia Tech, and Jason Lockhart, director of the College of Engineering's High Performance Computing and Technology Innovation, initiated the venture at Virginia Tech. Varadarajan is an expert in reliability, a key issue in successfully exploiting terascale computing.]
They keep on going:
[Component failures are endemic to any large-scale computational resource. While previous generations of supercomputers engineered reliability into systems hardware, today's high performance computing environments are based on inexpensive clusters of commodity components, with no systemic solution for the reliability of total machine.]
And now for the solution for your reliability problem.
[Virginia Tech has the first comprehensive solution to the problem of transparent fault tolerance, which enables large-scale supercomputers to mask hardware, operating system and software failures - a decades old problem. It's a software program called Deja vu, designed by Varadarajan. He also integrated the software with Apple's G5s. This work will enable the terascale computing facility to operate as the first reliable supercomputing facility, according to Varadarajan, a National Science Foundation Faculty Early Career Development Program (CAREER) Award recipient.]
So maybe, just maybe, you and other people could:
1. READ before posting.
2. Then READ a little more.
3. Did I say READ already.
-sigh- Whatever.My boss here at VT is a volunteer for this project... they've been designing and building rackmount shelf-type units to store all these new G5s, as well as helping with the cooling system. Here's some info he gave me.
;-), so there's a chance that he might use some PPC distro at some point.
The cluster will eventually run Mac OS 10.27... he said eventually, and Jason Lockhart, the project leader, is a friend and fellow Linux geek of mine (please don't hammer his inbox
Interconnectivity will be done with Cisco equipment, among the onboard gigabit LANs. Infiniband cards will also eventually be installed for 10 Gbit throughput.
You guys can offer alternative solutions and troll this as much as you want, but this is what VT is going with. In my opinion, it's not a bad choice... the New IBM PPC chipset is balls-to-the-wall computing, and Apple's 'stock' offerings in the G5 (Gbit ethernet, serial ATA, etc.) are all strong selling points. The fact that this cluster is intended for intense vector and matrix-based algorithms is another bonus, b/c of the PPC vector processing unit.
Apparently Apple shifted us up to the top of their production ladder, in order to make the contract, thereby extending the wait times for consumers itching for a G5... I find that a little humorous. Can't wait to see gigaflop statistics!!
May the threads progress competently.
In talking to the person who is recruiting me to help lug the computers around when they arrive, the OS is to be OS X 10.2.7 on arrival, with plans to upgrade to Panther upon it's release. Straight out-of the box releases, with NetBoot planned to be used to distribute the images to each computer. This contradicts the rumors I've heard before, but is closer to a source who is on the planning team, who is too damn busy to talk to a luser like me.
Those who are possible volunteer recruits, there is an info session in Andrews ISB in the Corp. Research Center at 7:30 tonight and tomonrrow night (same presentation both nights). You *cannot* be on wage for VT to be elegible. I'm not sure if GAs count as this, since I'm not one, I didn't check.
I am, and always will be, an idiot. Karma: Coma (mostly effected by
Well I don't know about other science work but my dad does AIDS research and they use Macs exclusively in their labs for all of their model simulations and experiments. There's apparently a lot of biology research software that is available only on the Apple platform as well. I guess maybe because of their ties to educational institutions over the years?
Wonderful: three pointers to Apple's web site, pointing to pages with slick corporate "interviews". Do you actually work for Apple or are you just insanely zealous?
There are an awful lot of scientists using Macs for their research and work. I use them almost exclusively now after retiring my SGI's in favor of the OS X boxes and judging from the meetings I attend, I would say Macs have anywhere from 10-40% penetrance in science depending upon the subfield. For instance the last vision meeting I attended (ARVO, the big one for the vision research community), there were Powerbooks and iBooks everywhere. Probably a good 33% of the laptops I saw.
Visit Jonesblog and say hello.