Slashdot Mirror
Drooling Over VA Tech's 1100-Node G5 Cluster
Mr. Slurpee writes "Virginia Tech's 1100-node dual 2 GHz Apple G5 Terascale Cluster is getting racked up and ready to roar. If you're a penniless geek like me, at least there's some tech pr0n for us to drool over. There's 1100 of them ... think they could part with one?" Update: 09/22 02:55 GMT by T : Matt submits a link to this full mirror of the photos, writing "The page owner's comment on the original mirror being taken down due to bandwidth? 'Bring it on!'"
Imagine a beowulf clu...oh, wait.
Sometimes i wish smaller schools had some pull so we could get projects like these...or even quarter, nay, a n eigth the size! problem is we don't have the immense research backgrounds and it holds us back.
JediLuke
-Do or Do Not, There is no Try
here.
Oh God.
Imagining each one of those came with just a little bit of Steve Job's Reality Distortion Field, someone from NASA might want to head over there and make sure that some kind of tear in space/time doesn't occur right there. With that many G5s, we don't know what level of destruction could happen.
Why are they using G5s? These Macs are for looking good, for style & class, for home or office, not for getting jammed into racks as in the pics.
:)
What's the point? Do they actually offer anything over a similar PC? Do VA's apps only run on PPC? Surely it's a pretty expensive option, and all those good looks wasted...
Using full sized cases seems like a rather inefficient use of space to me. But I guess those cases are all fairly full - the heatsinks in those things are enormous. Wish PCs had heatsinks like that, then maybe mine wouldn't be so noisy.
-kidlinux.
This presentation contains content that your browser may not be able to show properly. This presentation was optimized for more recent versions of Microsoft Internet Explorer.
Why make a website "optimized for IE", when the content of the said website is of interest to people who are probably not running IE or Windows?
"Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it." -- Linus Torvalds
These photos are from the Virginia Tech Supercomputer Cluster composed of 1100 PowerMac G5's. These photos were originally hosted at this link but access was removed, presumably due to excessive bandwidth usage.
:P Let's see how much power they really have!
Hmm. Something tells me that they should hook up a T3 line to each one of those G5's
----
Go canucks, habs, and sens!
do you think it would take this nifty cluster to correct the barrel distorition from their wide angle lesnse?
FooGoo
People who bite the hand that feeds them usually lick the boot that kicks them
They've got 1100+...where's mine? I ordered a Dual 2.0 GHz G5 in July....still no sight. Supposed to ship on Tuesday....but online time will tell....
Sigh...Maybe they'll loan me one if mine gets delayed!
PS--anyone got the rest of these pics? There were a TON of them...Mirror? COMPLETE?
insert all your g5 are belong to us joke here
I can just feel that some admin will get a deja vu very soon.
Probably just a small glitch in some system.
Proud patriot and republican voter.
What I really would like to know is how they install and configure all those machines. Their method of doing that will be very useful for even the (relatively) smaller networks that don't necessarily have to be clusters.
For example, I've yet to figure out a way to effectively get a computer lab with 30 eMacs installed and configured the same way. DHCP/Netboot is slow because we only have 100mbit switches. Split CD images are slow, and Jaguar doesn't yet have free software that does that yet (besides the dd of course). I'm not sure how to keep them all updated either.
I really hope they describe how they maintain the operating system on them.
$2,999 x 1100? 3.3 Meelion dollars? I hope they got Apple's .edu discount.
That's nice. So you have only 0.5 billion more nodes than a rat. I'm sure the rest of use, with our 100 billion nodes will laugh at you
He had to admit they didn't have any in stock, and weren't expecting to get any from Apple for some time.
I guess I know where the dual-G5 systems are all going. Ah, well, it's all for a good cause. I hope.
Welcome to the Panopticon. Used to be a prison, now it's your home.
If I was working on that project I'd be like "Hey guys you only need 1099 right? I mean, it'll be okay if I just take ONE, right?"
Small mirror with a couple pictures if you click here
Kick in the Head
Somehow I think this cluster could beat me at Chess....especially the GNUChess that ships with OS X....
I've heard various rumors that mac's are significantly faster for things such as graphic processing, video editing, and other multimedia type things. Considering the G5 is a 64-bit processor, that would put a dual 2ghz G5 at around 8ghz equivalent. Would a intel or amd based 8ghz machine (quad 2ghz?) perform equivalentely to a dual 2ghz G5? Maybe it would still be superior to a 8ghz i386 machine with multimedia apps? Am i totally out of my realm of computer knowledge? I'm lame.
Hi there
No words to describe it. Poetry! They should've sent a poet. So beautiful. So beautiful... I had no idea.
It's a bit of a cocktease to post this link right now...Most of the mac community sites linked to the pictures at Virginia Tech's site but brought it down. Try clicking on the "pictures" link on their site and you'll se that they chmod 0'd the whole site so that the bandwidth usage won't peak out again
The pics at chaosmint are a small selection of what was originally on the site.
But to be on topic I'm suprised that Apple didn't get them Xserve G5's for the cluster. While the desktop G5's look cool it's really unneeded to use up all that space.
Pink at LANL has the following:
:)
1024 nodes
2048 cpus
1024 power cables
1024 Myrinet network cards
2048 fiber cables (8.8 miles)
3072 Myrinet switch ports
4096 sticks of RAM (2 Terabytes)
7168 fans
1 hard drive
1 CDROM drive
Not only do they have pictures of its assembly, they have movies.
Check the web page for more stats and better quality movies.
Oh, yes, it's unclassified
I haven't actually tried it yet since I don't have access to enough Macs, but I imagine it's something you would start and let happen overnight... I mean, that's more or less how Apple does it in their own stores, wipe and restore overnight, I think. Or at least after the store closes and before the next opening day.
GPL Deconstructed
If you had the choice of building a supercomputer for your school then would you use brand name inexpensive processor like AMD or expensive chips like G5's or Itaniums ?
I really don't get what you can get out of a single fancy expensive processor when a couple of less expensive chips can do the job .
Good thing VA Tech got their 1100 before all of the full-price-paying customers who ordered these as soon as they were announced.
I have been using both Macs and x86 systems extensively for all manner of purposes since 1998. Neither really holds much of an architectural advantage when it comes to a specific type of function, and any slight advantage that is had by either tends to be so small that it isn't noticable. One thing is for certain: x86 hardware is far cheaper for the same amount of power.
Example: Photoshop 7 runs just as fast and efficiently on my dual 867MHz G4 with 512MB of RAM (booted into OS 10.2) as it does on my AthlonXP 1800 (1.533GHz) system with 256Mb of RAM running Windows 98SE, Windows 2000 Pro SP4, or Windows XP SP2. Video playback is identical (both systems have identical 64MB Nvidia graphics cards). Differences in compiling times are negligable.
Don't buy into the marketing hype. A $2,000 Macintosh system will run just as well as a $900 self-built x86 system. Everyone that has to use both platforms daily knows this. I don't have a preference for either architecture or operating system - just take my word as someone that has had to do extensive product testing on both x86 and Macintosh for years.
It's not inverted; the Power Mac's USB, Firewire etc. jacks are on the left side of the front panel, so it's right-side up. See a photo of the Power Mac.
www.firastudios.com
Because they didn't have to pay for it. VATech received the grant from an 'anonymous donor' for the project. 10 to 1 the money came directly from Jobs. I guess when he saw that all the real clusters were being built with Opterons he needed to show a G5 cluster could compete. Unfortunately for him this market is more concerned with price/performance than pretty cases.
Heh. So it does, at first glance. But I'm not sure if it would be practical to store 1100 g5s + racks on a floor made of your everyday celling tiles.
... or just look at that photo and notice that there are fans at the top. Either that or the guy is walking on the ceiling
Karma: Could be worse (could be raining)
I hope they were able to run these without video cards. I can't imagine 1100 brand-new sweet ATI video cards sitting idle for years...
It's rightside-up. The G5's drives are at the top of the case, ports at lower left. I'm curious why they don't face the computers.
HIV Crosses Species Barrier... into Muppets
As a Professional * Information Technology Location Analyst and Physical Security Specialist I need to use my professional abilities to make a professional analysis of the situation for my professional collegues so that we may put forth a professional solution to this problem.**
* - I really, really hate people who make gratutious use of the word "professional" as some sort of elitist mark of supremecy
** - I would like to run in there, see if the machines are locked down, and grab as many as I can hold.
(And yes, I'm just joking, I don't want to steal anything from them and I neither have the plans nor the means to do so, it's just a joke people)
All those silvery slick conforming cases remind me more of something from the book 1984. Ironic considering there advertising campaign of twenty years ago.
Interesting project though, and for those wondering what it will actually do: (from VT press release)
Virginia Tech researchers are already active in a number of areas that will benefit from the new supercomputing facilities, says Kevin Shinpaugh, director of research and cluster computing for the university. These include: nanoscale electronics, quantum chemistry, computational chemistry, aerodynamics through multidisciplinary design optimization, molecular statics, computational acoustics, and the molecular modeling of proteins.
And I thought it was going to run something with a cute name like iFiniteElementAnalysis...
think they could part with one?
They could, but they won't.
One of the reasons VTech went for a G5 based cluster WAS price-performance...The mac option is cheaper then a PC aption and easier to install and maintain then Linux says the slide show. I might not fully agree, but thats their reasoning.
Sig (appended to the end of comments you post, 120 chars)
Looks to me like those racks will eventually be enclosed, so those fans will be probably expelling hot air and drawing cool air in from the bottom... note the holes in the shelves to allow air to pass through.
NO! It won't beat the ES. It won't even come close. The ES is a very tightly coupled system designed with heave I/O throughput from the ground up. This cluster isn't. And, ES up until recently was faster then top 20 US super computers combined. I doubt this VA-Tech cluster is faster in any dimension.
now supporting:
cmdrTaco for president '04
michael for oval office intern summer '05
I'm really feeling for the poor slobs who have to lift 1100 of those beasts onto shelves. G5s are heavy!!!
Ah, vivid memories of the cover of Softtalk magazine, with a picture of the Apple II assembly line with hundreds of machines. Just imagine... 200 * 64k = 12.5 MEGABYTES! That would take 90 floppies to store all that data!
Now some statistic pr0n:
There were about 5 1/2 million Apple IIs sold, so at an average of 64k each (just a guess), that would be 343 GB of memory total. Adding up the couple of computers in the office (it's a 4 person company), we're about 1/70 of the way there. Assuming 2 140K floppy drives per computer, that would be 1.5TB of disk storage -- that would be 6 hard drives, and they would occupy less space than a single pair of old floppy drives.
HIV Crosses Species Barrier... into Muppets
Nope. That's the ceiling. The fans would be blowing outward to help along heat convection. The pans (bottom) of each cab would also either have a fan blowing in or just simply a vent that would suck cooler outside air into the cab, through the perforated shelves past each machine and out the blow hole at the top. The reason the air is taken from the bottom is because cooler air is down there. Heat rises etc etc.
I run Contiki you insensitive clod!
Anti-social? My code is just platform-specific.
We once had a 'discussion' in an apple chat channel about errors made by the Apple Calculator. I just hope they are not visible when you want to do calculations on that cluster... 3.083 - 3.014 = 0.0690000000000003 is what appears on the paper tape from the calculator if you try this calculation. The display shows 0.069, but that is only cause it rounds off. I wonder where that 3*10^-x (too lazy to count) comes from, but can imagine that stuff fed to a supercomputer could give really wrong results if even small calculations are bogus!
Don't worry about them. All the lifting will be done by volunteers hoping for a chance to touch the g5s. The sore backs will be worth the chance to brag about this to their grandchildren.
...fork!
Speaking at Defcon 12 - Credit Card Networks Revisted: Pen
I, for one, welcome our new 64-bit overlords!
Hi, your troll sucks ass. Perhaps you should elaborate on how the CPU is inaccurate rather than one stupid application.
Actually, the site is still up and more bountiful if you happen to be on VT's Campus. There are a lot more pictures of it available on that same link of the entire construction effort.
Though those of you who aren't at VT can't see this image, it's still quite hilarious.
a le/images/g5ordering/IMG_0099.JPG
Apparently Tech ordered the G5s through iTunes: http://computing.vt.edu/research_computing/terasc
hidden goatse cgi redirect link! beware
I wish I had mod points right now... you sir -- YOU owe me a keyboard, for I have just spit out my Mountain Dew.
Karma whorin' since 1999
Does it run Quake III???
SCO Sucks
No, the 'big fan-things' are called XDVs from Liebert (http://www.liebert.com/dynamic/displayproduct.asp ?id=1077&cycles=60hz). The mount on top of the rack.
read the fucking article
Agreed!!! Near as I'm aware, apple doesn't offer a rackmount solution, and franky it's a pain in the butt to mount a g5 class motherboard in a standard rackmount case. I have much respect for Apple but strongly disagree with their policy on computer sales, basicly it really isn't an option to buy mac motherboards... so for the average person looking for a reliable resource for mac boards... you gotta buy the turn key solution and chuck the cases / drives / whatnot.
Apple probally should offer a rackmount solution, or at the very least offer some barebones systems for the enjoyment and pleasure of those who want to use this platform in the industrial enviroment or for the benifit of hardware hackers. Apple cases are spiffy but no replacement for a solid rack by any means.
There is no sanctuary. There is no sanctuary. SHUT UP! There is no shut up. There is no shut up.
Exactly.
Learn to read, VT told you
Because the G5 systems were the cheapest AND fastest
There is no rackmount version of the G5 yet. That would be an upcoming G5 Xserve that has not been announced yet.
Plus, I guess when this cluster needs upgrading, they can sell off the Dual G5s which should hold their value for quite a while, as they are just stock G5 workstations.
It is quite the fashion statement :)
(Excuse the blurriness and poor lighting - crappy cam and crappy dorm lighting)
$1537800, for 2200 CPUs. IF they run lunax. OS X would be just as good, and non-infringing.
Macs appear in films because of price, Apple pay for Macs' to appear in films.
Maybe VA got a 'special deal'?
thank God the internet isn't a human right.
All I can say is if you are getting write speeds that slow, when was the last time you defragged your hard drive? I've never experienced that slow of a write for a 17 Mb file on any of my macs. Is your hard drive full?
http://www.apple.com/xserve/
Configure one machine, use that one to produce a dupe, repeat with all currently configured machines as parents until out of unconfigured machines.
Still a huge job.
god I wish I could go there with a big block of cheddar cheese I bet I could shred it in like, 10 seconds!!!!
http://saveie6.com/
MacOS X on a recent Mac is a nicely stable and rather efficient (if slightly quirky) Unix, at least that's been my experience with a 700Mhz 384MB RAM iBook. It could well be that you need at least that much RAM before it starts to shine.
The IBM floating point is fast and almost too accurate (Consistency between runs on different architectures is sometimes more important than those last few bits of accuracy). Going to single precision tends to slow it down (extra step in throwing away information).
The bus in the new machines looks awfully nice on paper.
In a cluster: Reliability, Fast networking, fast bus and fast/accurate floating point are mighty big draws.
I don't think anybody will know for sure if this was a good move until it's been running awhile but it's not an unreasonable thing to try.
nice generic flame. i usually see it with *BSD instead of mac.
d'oh
My mistake... but still tower cases are not well suited to that enviroment
There is no sanctuary. There is no sanctuary. SHUT UP! There is no shut up. There is no shut up.
How many times are you going to post this same stupid message? If seen it for about two years now, it's some sort of ancient Microsoft Form Letter from 1997.
An 8600/300? I doubt that you even know what that means. PPC604's kicked ass in their day.
It's mandatory to wash your hands before returning to the land of Dairy Queen.
I remember years ago that it was difficult to get spare parts from IBM for PC-XT machines. You could go to the Ham Radio Swapmeets and there would be people selling stripped XT cases for a good price, because they were associated with a company that needed spare parts, and it was cheaper for them to buy IBM XT boxes and strip them for the parts they needed. It seemed nuts at the time, but, then, if you were servicing IBM equipment at the time the money was good servicing the machines.
A Good Intro to NetBS
The boxes were purchased at the standard academic discount price from Apple. Not even a volume discount. It's said so in at least two of the articles I read about this story.
But the truth usually doesn't stand very well against the 'obviously true', does it?
-fred
Sign #11 of Slashdot overdose: You see the phrase 'moderate Republican' and you wonder if that would be a +1 or a -1.
...after a good ten minutes of googling, I can't actually find anything out about it.
Basically, he made it up, because he doesn't like the idea that something like this might be possible.
-fred
Sign #11 of Slashdot overdose: You see the phrase 'moderate Republican' and you wonder if that would be a +1 or a -1.
Funniest thing I've seen all day.
And I'm a Mac guy, too. I wouldn't mind wandering through that room for a while myself... though I probably would keep my pants on.
-fred
Sign #11 of Slashdot overdose: You see the phrase 'moderate Republican' and you wonder if that would be a +1 or a -1.
Never mind that Apple is one of the very few consumer computer companies making money right now. The G5, whether or not you're a Mac fan, is a very nice processor, and it will continue to drive Apple sales. Apple has a niche market, and they will probably always have the niche market. It's doubtful that they'll ever gain more than 20% market share, but they'll always be around. Maybe you should use that dessicated brain of yours before spouting off.
But with the opteron solution you don't get 4G of free ram per node.... So by the time you _pay_ for the 4G of ram the Opteron is more expensive.
That's the whole point of the PPC970's Altivec unit, isn't it?
Here, scroll down a little to see the CPU performance on RC5, and note how a dual GHz G4 is 2x as powerful as a dual 1.533GHz Athlon.
Extrapolate then to a dual 2.0GHz G5 vs a, I dunno, dual 2.0GHz Opteron... why wouldn't it be, if not 2x faster, at least as fast, if we want to be generous and assume that AMD somehow managed to figure out how to increase the performance of the Opteron over the Athlon by more than 2 (in order to take into account the fact that the G4->G5 increase is 1.0GHz, but the Athlon->Opteron increase is only 500MHz...)
GPL Deconstructed
So they reimage about 5gb worth of OS + Applications per machine, and assuming it's a decently fast FW drive at 20mb/s means about 5 minutes per machine.
So an iPod with 30 eMacs in a lab should take about the same, since an iPod clocks in at 16mb/s (I own one) but there shouldn't be nearly as many applications/software on an eMac as on a G5... so 150 minutes, or a little over two hours a night.
And at standard lab-rate of $5/hr, that's $10 to reimage the entire lab.
GPL Deconstructed
The only thing I found of interest is some BONUS apple pr0n pics:
Look at the rack on that one!
Even bigger rack!
What the G5 sees
Looking in.. notice no nose prints/drool marks on the window(yet)
Pulling "facts" out of your ass to make your "point" just makes you look like an "idiot".
I am a believer of momentum and curves.
I concur. Now is the time for a +5 Flamebait moderation. You have the power, you can give this humble AC the gift that we all know he deserves. An Apple story about a cluster of G5s cries out desperately for a truly excellent piece of flamebait. I think this rises to the occasion. Please do your part. Invoke the "underrated" mod, and now. Thank you for your cooperation.
Could you possibly use a few more quotes in your next reply or were you just confused about what you were saying. Looks like another Steve Jobs groupie puzzled by their own words.
I'd recommend some basic computer maintenance.
- run disk tools... have it repair everything. Do this from the Boot CD. Insert Boot CD... hold down C key while machine boots until you see Welcome To Macintosh. Once it boots, locate Disk Tools in the Utilities folder and run repair on all available partitions.
- run a current version of Norton Disk Doctor if you have it and have it auto fix all errors.
- Pony up for some RAM. 64 megs has been puny for a long time, especially for graphics. I recommend a min of 128megs for Mac OS (classic) based systems. Preferably more if you do graphics.
- Check your Virtual Memory control panel. Set it to no more than double the installed RAM.
- if you have Norton, use the defragger. I'm sure the drive's never been defragged.
- reboot the machine
Should work great after that. My old 8500 still works fine. I can copy several gigs on it in the 17 minutes or so.
Or do you just want to bitch?
The real answer is that the problems that are going to be solved with this cluster are easily parallelizable. That's the IDEA, right? 1100 machines, each running one chunk. Well, the G5, and more specifically the Altivec vector processing section of it, is SO MUCH better for processing big bites of easily parallelizable data at a time than any of the alternatives that it can run rings around any Intel or AMD machine you care to name with fewer than double the number of processors. (And in the cases of some particular kinds of calculations, it beats those, too. But you can't count on that for all your problems.)
We've seen this before a number of times... I seem to recall a gene sequencing program that was running five or six times faster on a G4 than it was on a Pentium IV of the same speed. And then there's SETI@home, which runs much faster, cycle-for-cycle, on the Mac, and doesn't even USE altivec. (Though I believe it does take advantage of the 'multiply-and-add' instruction of the PPC, which is another nice little feature.)
Altivec is an astonishingly clean and usable interface for an amazingly powerful vector processor that is, in 99% of the Macs out there, underutilized to the point that if it suddenly disappeared, most people wouldn't notice any difference at all. It's kind of a pity, really.
Basically, Intel came out with MMX (and all the later developments) in order to have a talking point on a slide presentation about their processors, about the time when competitors like AMD were starting to come forward: functionally, an awful mess, and impossibly difficult to program. (In fact, for the first few years, Intel would send programmers out to work with companies to implement MMX, because otherwise none of them would bother.
AMD came up with something that was a little less hacked together in a very short period of time, as a response to Intel. But it still wasn't pretty, at least partially because of the limitations of the archetecture, and the performance wasn't *that* much better than just doing without.
Apple (who really designed a lot of the basics themselves when it comes to Altivec, so don't think this was a Motorola invention) said, 'Hey, wow, we need something like that, in order to compete.' First they decided on a coprocessor, but that didn't fly any better with the PPC than it did with the older Macs (840av, 660av) with DSPs in them. So they sat down and came up with a really *good* spec for a set of multimedia extensions. And they've only gotten better since.
I've toyed with altivec code, and I can tell you that in one application that I wrote, one instruction (vector permute) did the work of ten or more non-altiveced instructions on four times the data per cycle. Mind you, I just did it for fun, I don't know enough about parallel computing problems to come up with anything useful... but there's some interesting stuff under the hood.
Of course, nobody is going to believe this, because as fashionable as it is to like MacOS X on slashdot these days, nobody wants to admit that, for *some* subset of problems, Mr. Jobs's reality distortion field might not be quite as much of a distortion as you might think...
-fred
Sign #11 of Slashdot overdose: You see the phrase 'moderate Republican' and you wonder if that would be a +1 or a -1.
np: Aphex Twin - On (Reload Mix) (On Remixes)
"I'm not anti-anything, I'm anti-everything, it fits better." - Sole
VATech received the grant from an 'anonymous donor' for the project.
Funny how you didn't cite a source for this, mister AC.
Oh, that's right, because there isn't one.
Mod me down and I will become more powerful than you can possibly imagine...
towers work just fine, most are front to back cooled, just like a rack mount
also the hardware has more room too for more heat disapation.
if you have the rack space, its not a bad idea. I imagine that the racks they are on are cheaper than "real" racks. And I know a g5 tower will be cheaper than a comperable g5 xserve, whenever such a beast exists.
-Tim
-I just work here... how am I supposed to know?
Apple fanboi. Apple story, apple slur, time to post!
Nice example of the ad hominem argument. Nevermind the fact that the point he was making was correct. Go stick your head in a pig, or back in a pig, as the case may be.
political_news.c: warning: comparison is always true due to limited range of data type
Well, kick down then! Surely you know how to set up a mirror =]
political_news.c: warning: comparison is always true due to limited range of data type
I'm still bleeding from the adds and cookies zinging by on that page that had three poorly-produced photos.
Unless you want to look at a disgusting picture of a girl in a bathtub eating her feces. No, I'm not kidding, that's one of his links, and it's quite possibly groser than goatse.cx.
political_news.c: warning: comparison is always true due to limited range of data type
with the g5 solution you get 4g of free ram???
that certainly isn't a fair comparision then, just a special deal comparision.
world was created 5 seconds before this post as it is.
17600 FANS!!!!!
Maybe they used screws... (Looking at the G5s the picture semms ok)
May not be perfect, but it's still the best government we have. For now.
[notices "HAIL 64-BIT" sign taped up, tears it down]
Oh, yes, by the way, the spacecraft still in extreme danger, may not make it back, attempting risky reentry, bla bla bla bla bla bla.
We'll see you after the movie.
Between the on-campus nuclear reactor and the supercomputer cluster, I'd keep an eye out if I were Tech's cross-state rival, University of Virginia. I'd say the Hokies are just one diabolical dean away from becoming an evil university bent on world domination. And five bucks says they start in Charlottesville.
DecafJedi
my weblog: apropos of something
When I first heard about this I thought these would be custom boards that would contain a minimum of components, SBC style that plug into a back plane and net boot, with tens of nodes per rack, etc.
Never in my wildest imagination did I think these would be the full Apple G5 desktop's, cases and all.
Those fans in the G5 switch on when the system gets hot. Imagine the noise when this cluster is pegged at 100% CPU occupancy!
War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
If you believe 100mbits is too slow why not use IP over Firewire. I use it on my MacOSX server and it is screaming fast. You get 3200 mbits.
There is really nothing to do just daisy chain your mac with Firewire cables and configure the new network. On MacOSX client you will need to install IP over Firewire manually.
This could be multicast... provided the network is setup correctly. That makes it a lot easier.
Also, in this kind of setup you often keep a copy of the master image locally on the dive of each machine, on a separate slice / utility parittion... if done properly this lets you re-image in seconds or minutes,
They're going ot cluster with ethernet cards? Dude, this thing is going to be hampered wiht that ...
for windows users do as follows
1. Set up 1 machine how you want it.
2. Use Ghost / Drive Image or what ever your favourite drive cloning utility is, to image all the drives in no time at all
3. Go in and change HD, Network, etc names.
4. Smile because you just did something in 4 steps that took a mac user 6
I was one of the hordes of CS majors who helped setup the super computer (grunt work is fun!). VT is using inifiniband cards w/extremely low latency copper cable (forget the name) which acheives the same bandwidth as fiber optics.
Loads of cisco catalyst switches are involved also.
X86 fanboi, Apple story, time to post FUD! Like you're any better. A fanboy's a fanboy.
Considering my 256 cost me 100 dollars, thats x 4 x 4 = 1600 dollars per machine.. x 1100 = 1,760,000 dollars...
heh
Dave--
I don't know how fine a point I can put on this, but...
He's right. You're wrong. The reason why really shows up here: "What throws most people is that although they are used to 1/3 being a repeating decimal, they think 0.1 should be an exact number in floating point."
No, they don't. Most people don't have a clue what floating point means. They are quite blissfully unaware that there's any mindset that would find 1+1 to equal 2 but 0.1 + 0.1 to only sort of equal 0.2.
And they're right not to. 0.1 is fundamentally a nonrepeating value in the standard, Base 10 Arabic numbering system -- it is not a range, it is an edge. That computers don't use Base 10 internally is not an excuse for them to be providing inaccurate answers to relatively simple problems, it's only an explanation. Intel's FDIV bug couldn't be explained away with a detailed technical description of the bug("Oh yeah! That's because we forgot to set that bit! What? There's still a problem? We told you its cause..."); neither can the use of floats be forgiven in applications that are expected to operate on discrete values.
I am, of course, not the first to point this out. More than a few systems -- usually for the financial types -- operate using BED(Binary-Encoded Decimals), which are quite slow and horrifyingly inefficient but do not suffer from the artifacts mentioned. Fundamentally, while floats(and particularly doubles) can be munged into being "accurate enough" for most discrete computations, ultimately their use is very reminiscent of the classic quote by John Von Neumann:
"Anyone who attempts to generate random numbers by deterministic means is, of course, living in a state of sin."
--Dan
One quote for the power consumption of a Power Mac G5 Dual Processor stock is 800 watts. Now 1100 of these - 880kW! 880kW at 120 volts is about 8000 amps. Mind you, this is not including all the glue electronics... I am glad I don't have to pay THIS electric bill!!! :-)
i could but i dont want to spend the huge amounts of money from the slashdot effect again.
I thought VA Linux was changing their name again.
--
"Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
YHBT.
Where the fuck are you getting your ram from? I just paid $42 for 256mb ram. Although one wouldn't put 256mb chips in those, so lets say 1gb chips. Quick pricewatch, $179, so lets say $185 x 4 = $740 per machine x 1100 = $814,000
This gives them 1101 good computers - a kickass cluster now, and 1100 workstations later.
[100% ISO 646 Compliant]
SVM, ERGO MONSTRO.
I read the article, and it looks like it was a REAL amature decision to go with these systems.
Sure, the systems probably offered good performance for the (heavily discounted) price. However, they extremely large size of the desktop systems means that they're going to spend a lot more money on physical space to keep these systems. They also have rather high power consumption for super computer use due to all the completely useless (for a supercomputer) extras that the G5s come with (eg the high-end gaming video card). They are also likely to run into a number of problems with failing fans. Each of those G5s has 9 fans, and fans are about the least reliable part of any modern computer.
What's more, they now have 4.4 terabytes of non-ECC memory. Given the Soft Error Rate of today's memory, it's likely that they'll be getting at least one soft error every few days in this cluster. Now, maybe they'll get lucky and the error won't affect things, but they are leaving a LOT to chance here! This looks like a MAJOR oversight if you ask me. Ohh, and before someone starts talking about that "Deja Vu" technology, it does absolutely nothing to detect or correct memory soft errors.
I seem to recall a gene sequencing program that was running five or six times faster on a G4 than it was on a Pentium IV of the same speed.
It's called BLAST, and it's up to twenty times faster, according to the tests that were run at the time. You know the drill: fastest available Power Mac, fastest available whatever from Dell.
I imagine that the racks they are on are cheaper than "real" racks.
They're standard Stantron racks, with shelves.
The mirror has been removed, and now redirects to the original page.
But when it comes to web designers these days, I suspect they've either got major stock in adobe (pdf), in microsoft (IE specific code) or in macromedia (flash).
So I'd be tempted to rewrite it as "Never ascribe to malice that which is adequately explained by stock ownership."
Though, I do know enough web designers that I suspect few own stock and I know just how lazy and incapable many of them are. So, more sensibly it should be written as "Never ascribe to malice that which is adequately explained by a complete inability to learn anything more difficult than 'Dick and Jane do Power Point'."
And the fact that so many of our educational institutions are on all fours for the convenience of Microsoft and other corporations helps not a whit to alleviate the general witlessness.
AMD has been making Intel-compatible processors under license from Intel since the 8088 processor.
2+ million BTUs of cooling capacity using Liebert's extreme density cooling. This system uses rack-mounted heat exchangers with R-134A refrigerant and an overhead chiller unit
(R)Traditional AC systems would have resulted in a wind velocity of over 60 mph under the raised floor
Holy hell, that is quite mad.
I am NaN
At our school, we have developed an ad hoc solution with ASR (Apple Software Restore), it comes in 10.2 (client): try "man asr" on the command line.
ASR is damned fast ! We restore from scratch 2Gb system disks in 5 minutes. The image file is stored on a standard AppleShare server (100 Mbit/s switch). The speed is due to ASR ability to restore using block level disk access instead of file level, and from compressed images so less data has to travel on the network.
Each client has 2 partitions: one of them has a minimal system so we can boot on it when we want to restore the "main" partition. Everything is done automatically with ssh.
All us Geeks afflicted with interminable penury should merge our truly meager resources and nearly infinite technical skills to build a massive supercomputing cluster!
Hell with the unemployment rate at 30% for techies here in Silicon Valley we could use something to do!
I killed da wabbit -Elmer Fudd
We use it in our lab here at work and yes it is very very evil
-ravan_a
I don't know what era you're from, but I doubt that most people would agree that 11% is "heavily discounted".
Also, please explain (I really want to know) since you apparently know so much about it, why Deja Vu will neither detect nor correct memory soft errors.
http://www.chaosmint.com/mac/vt-supercomputer-2/
"Project started back in February; secret with Dell because of the pricing issues; dealt with vendors individually because bidding wars do not drive the prices down in this case"
BLOW ME DELL!!!!!
HA HA HA HA HA. Jobs must really love reading that.
Just blogged it. It's a quality quote.
This is exactly what I did with my G4 Cube, and we all know how that went over.
"You know why you do not see me styling wit my homies? Because I have no homies!!" -Mojo Jojo
According to you, then, Apple, when it sells direct to a customer through the Apple store, is making less than a 15% margin on its machines. Because that's the typical maximum discount they give, except in cases of clearing up back stock to make way for new models. Even then, I suspect that Apple's margins are 'healthy' enough that they're not selling below cost, but I *know* their margins are more than 15%. It's one of the things people like you complain about, that Apple makes big margins on their machines, 'overcharging' their poor, hapless consumers.
Basically, grow up, There are plenty of nasty, awful things that Apple has done in the past, and a number of them they still do. When you make stupid accusations with no basis in fact, just because it's Apple and you hate them, you make it harder to have a reasonable dialog about what's really going on.
-fred
Sign #11 of Slashdot overdose: You see the phrase 'moderate Republican' and you wonder if that would be a +1 or a -1.
Talk about the lowest density (modern) cluster I've ever seen. 12 nodes per rack is complete crap; even if each node is dead-sexxy.
But they didn't become popular, didn't become a common replacement for Intel chips in the real world, until the K5 and K6, and the 6x86 (from whoever made those), etc. Before that, Intel didn't have to worry about them; they weren't much of a threat.
-fred
Sign #11 of Slashdot overdose: You see the phrase 'moderate Republican' and you wonder if that would be a +1 or a -1.
probobly AC ducts. Cold air falls, remember.
That cluster is probably capable of compiling Gentoo from source in only 2 days!
Using full size boxes is nuts. Ask Apple for blades. But perhaps they got money to burn.
But it doesn't have to be that way. It is really a user-interface issue.
It is clear that what the user of a calculator program wants when they enter 3.083-3.014 is the mathematically correct answer, not the closest approximation to the difference between closest_floating_pt(3.083) and closest_floating_pt(3.014). So the program should do the extra work to make it so whenever possible. In most cases, it is possible.
And the original poster was talking about the fact that the rounding error (from 3.083-3.014) showed up in the paper-tape printout. That would seem to be a bug, pure and simple.
What operating system will they be running on these things?This is not a troll, I'm geniunely concerned to know if they will be wasting these lovely resources on OSX ? (Which is not truly 64bit)
Electronic Music Made Using Linux http://soundcloud.com/polyp
Just curious, one thing I haven't been fully able to understand is what they mean by 100 nodes? Since the G5s they installed are dual processor, does that mean 1100 Computers/2200 Processors, or 550 Computers/1100 Processors?
"Reality is merely an illusion, albeit a very persistent one " -Albert Einstein
Well, I guess they weren't the same speed then, were they?
And it's Pentium 4, not Pentium IV.
"Hot lesbian witches! It's fucking genius!"
They were not a threat at all because they made the chips for ... wait for it ... Intel. They were a third party manufacturer. They made identical chips to the 486 and 386 and 286. Why? they had the exact same masks as Intel. Belive it or not Intel could not make the chips fast enough. They were that popular. So they licenced it out to several other companies. Intel got a slice and the company making it got a slice. But Intels name went on the chip.
When Intel started to get capacity to make enough chips and didnt need AMD. AMD started making their own chips based on the same thing. Intel was about 1 gen ahead of AMD for a long time. But Intel basicly milked the ppro series for too long. That is why AMD caught up. In the last few chips AMD has blown by Intel. But MHZ is king in the marketplace. Its a place for people to compair. Its not a good indicator but people do notice. If AMD can scale it newest chip up to the same speed as Intel, Intel will be in a bad position. It does not have the next gen chip ready yet and it will be rushed.
AMD was making pin compatible chips for a long time. Intel noticed this and tried a lock out using a differnt socket design and copyrighting it. It didnt work too well. Which is why we are back at a chip grid array instead of the slot.
Where is motorola in all this? They want out of the chip busness. They want to sell IP. But they can not figure out if they really want out or not. Otherwise you would see the ppc chip running at 3ghz and running circles around intel and amd.
No way dude, the G5 pulls are 70Watts at 2Ghz. Where did you get those numbers?
TY for not being a karma whore.
They should've hosted on thier beowulf cluster of macs....
http://tartarus.uwa.edu.au/~trs80/Terascale/
I was one of the volunteers and yes the sore back was worth it.
Actually, we didn't spend more money for physical space to keep these systems. The whole thing fits into about 3,000 square feet. We have over 10,000 square feet of machine room that we built back in 1989 (sized to hold an IBM 3084 and an IBM 3090-300J plus all the disk drives that went with them). Good thing our director at the time spec'ed out "all the floor space we're likely to ever need".....
Nobody cares about the noise. It's in a machine room, not office space. All the offices are on the other side of the atrium.
And I doubt that it can be much noisier than the printer room was when there were 2 IBM3800 printers going full-tilt (400 pages a minute each. No, that's not a typo. Four Hundred. Feed it a box of 3,200 pages of fanfold, and you *might* have time to go pee before it ran out again).
Well.. the processor itself is 70W. There's 2 of them, that's 140W. Then there's memory (4G of it), a 180G disk drive, all the support chips (yeah, things like disk controller chips and PCI driver chipsets and memory/cache interface chips are nice to have if you actually want to do any computing), the PCI card for the Infiniband... and pretty soon that 750W power supply isn't as much overkill as you thought.
Nobody cares about the noise. It's in a machine room, not office space.
Yeah, I know. I've worked in many a computer room, including huge rooms full of DEC big iron.
I wonder if they'll drown out the air con.
War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
I weep for the state of reading comprehension in this country.
I am a believer of momentum and curves.
It's just VT. It's not VA Tech, VTech, it's just plain VT damnit!
http://www.maximum-cars.com - My little hobbie.
Hate to say it but.... you probably won't have that much noise. Just saw my first G5 in the flesh and..uh...well...mmm
I couldn't tell it was on til I put my ear ON the case.
Sorry, but according to pricewatch, dual opteron motherboards start around $400 and the lowend SMP capable Opteron starts at about $250. So you're going to be looking at least $1000 for the cheapest Opteron machine that is somewhat the same sort of configuration as these G5s. Unfortunately, the G5 would be able to kick the snot out of such a machine. Use the 246 Opterons and now you're talking about a machine in the $2000-$2500 range. That's the type of machine these guys would want.
From looking at these pictures, all I kept thinking was: what a bunch of wasted space and under used equipment. As a compute node, what use is wi-fi, bluetooth, USB, audio subsystem, etc. If they have video cards, is the graphics processor & memory going to be used as some sort of coprocessor? If not, that's a waste too.
Apple was stupid to not create a G5 version of their rack mountable XServe machine. It would be a better fit for what these people want to do.
the good ground has been paved over by suicidal maniacs
Look at the heat sinks in the g5's. I doubt they'll be getting that into a 1U anytime soon. I'm betting on a 2U Xserve for the g5s.
Apple just probably didn't want it all rolled out at the same time. They'd have nothing to show off for a couple months.
Mod point free since 2001
The Apple System Restore tool can erase a disk and copy a configuration onto it from a compressed disk image. This is what Apple uses for setting up machines on the production line.
"man ASR' ion Terminal.app to get started.
-Mark
Same image canvas size, but lower image file size:
http://technojunkie.org/berniec/VTpr0n/
Total images filesize before were around 61MB; now down to 17.5MB.
I don't doubt that it would be a while before 1U G5s show up, but a 2U server would be better than just stacking deskside towers on a bunch of shelves. Good point about not having something new for a while. Maybe the university didn't want to wait.
It would be cool if instead of announcing a machine and then waiting a couple months for it to show up, Apple could have gotten the school to wait, secretly install a slew of rackmount G5s, and then have Jobs and the univ big wigs unveil them at a ceremony the next day in the server room.
the good ground has been paved over by suicidal maniacs
For those interested in a lot of FAQs about the project, I'd recommend a visit to Your Mac Life, the weekly net radio program about Macs. Their Sep 10 show has a half-hour interview with Hassan Aref, the Dean in charge of the school and the project. Much useful info.
Having seen this cluster in person last week (I work right across the street), I can say that it is very awe inspiring to walk through. The woman that gave my friends and I the tour mentioned all of the logistical problems that were overcome to house that many computers in one room. I guess I am naive but I had never considered how fast the temperature would increase in that environment. She said that if the AC were to fail that the temperature would increase to around 150F in about 10 minutes. That is incredible. Because of the an extra phase of power was brought into the building to just for the AC units. There is a room (larger than my computer room at home) just to house the UPS batteries. And the UPS is designed to run only long enough for the diesel generator to kick in. The rows of computers were designed with hot and cold isles to help with air circulation. One thing that I did not expect to see was out of the box G5s sitting on the shelves. I thought that they were just using the processors. The woman giving the tour did not know the answers to all of the technical questions that we started asking about file storage, back ups, process management, but it was still an experience that I will remember for the rest of my life. It also puts the ~24 node Beowulf cluster that I have been working on to shame. :)
I couldn't tell it was on til I put my ear ON the case.
Curious. Was it being placed under stress for long enough to fire up some fans?
I imagine in the showroom, people will be very impressed with the noise, but when they get it home and push it will some games, 3D rendering or 2D filters, it might suddenly be a lot louder than the user bargained for.
BTW, I'm not trying to put shit on the G5, I'd love one. I'd also love to know how it sounds when the fans are going hard.
War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
for all those G5s, or signed them up for .Mac?