Virginia Tech Supercomputer Up To 12.25 Teraflops

gonknet writes "According to CNET news and various other news outlets, the 1150-node Hokie supercomputer rebuilt with new 2.3 GHz Xserves now runs at 12.25 Teraflops. The computer, the fastest computer owned by an academic institution, should still be in the top 5 when the new rankings come out in November."

Comments

hrm

[gutterandthestars]

6.40tflops should be enough for anybody

Re:hrm

[tmj0001]

Hans Moravec's book "Robot" suggests that 100 teraflops is about the level required for human intelligence. So we are up to 10% of his target. But human intelligence still seems very far away, so either he has badly underestimated, or our collective programming skills need significant improvement.

Re:hrm

[TimothyTimothyTimoth]

I think Morevec's method of simulating human intelligence involves modelling a scanned copy of the human brain, in real time at a neuronal level. It would be similar to modelling the global weather system, a software capability we already have. Current neuroscience would expect this model to be functionally equivalent to a human mind in terms of matching inputs and outputs. As an aside, I know that Ray Kurzweil has I much higher required estimated of a 20 petaflop (20,000 teraflop) computer, based on more conservative assumptions. 20 petaflops is due around 2009/10 under Moore's law. (And I for one offer an early welcome to our expected new AI overlords ...)

Re:hrm

[Quobobo]

I think the reason lies within the latter.

Think about it; how is throwing more and more hardware at it going to solve the problem? What we're lacking is the software itself needed to do this, and it's obviously not going to be an easy task to write. I see no reason why an AI as intelligent as a human couldn't be implemented on a slower system, unless "thinks as fast as a human" is among the requirements.

(disclaimer, I've never read the book, these are just my opinions)

Re:hrm

[SnowZero]

I actually asked Hans a similar question at a talk he gave a while back, and he didn't really answer it, to my disappointment. My question was that "In nature the algorithm and computer were evolved together, so we'd expect them to be at a similar level of advancement. So, even if we get a computer as fast as a human, it might it not be nearly as smart since our programs do not use it efficiently enough?" In other words, Moore's law isn't helping us write better software (in some ways quite the contrary).

I'm a robotic software researcher, so this notion really affects me. IMO Software will lag well behind hardware, since it doesn't scale out nearly as well. Representation is of course a huge problem I won't even try to touch... But rest assured lots of people are working on all these things. Btw, It also doesn't help that CPU designs aren't even trying to make AI-style algorithms fast, but we can't blame manufacterers for that util there is demonstrable money to be made.

Re:hrm

[TimothyTimothyTimoth]

By the way, IBM BlueGene/L is going to produce 360 teraflops by end 2004, so if the report of Moravec's estimate is correct, and he is correct, that AI Overlord welcome could be pretty soon.

(Although I don't believe brain scanning quite hits the resolution mark required yet.)

Re:hrm

[beders]

In an object orientated system it should be a case of modelling individual neurons and their interactions, the hard part might come getting these tied into the inputs/outputs

Re:hrm

[jimicus]

In theory, the software required is easy. All you need is enough inputs, outputs (doesn't have to be speech) and enough neurones (either real or simulated) to connect it all together.

After that, the complicated bit (training the neural network) is much the same as it is with a baby - talk to it, show it simple things, put liquidised food in one end, keep the other end as clean as possible.

The only minor snag with current technology is the limits to how much it can learn and how long it takes to do so.

Re:hrm

[RKBA]

His estimate was probably based on the common, and incorrect, belief that neurons are purely digital.

Re:hrm

[Short Circuit]

What kind of advancements to CPU design would improve their use in AI? Shorter pipelines? Greater emphasis on bus speed vs cache?

Re:hrm

[dr_d_19]

...or perhaps Hans Moravec was just plain wrong :)

Re:hrm

[Deorus]

I think the difference between human and computer intelligence is that our software (conscious) is able to hard-wire the hardware (unconscious). We may not be able to consciously perform certain tasks such as floating point calculations because our software lacks low level access, but we can hard-wire our hardware for those tasks, this is why our unconscious is so quick and accurate when trained to recognize and respond to specific patterns regardless of their complexity.

Re:hrm

[segmond]

He is wrong. Intelligence is not about speed. I have met people who are very very smart, but they think very slowly. You ask questions, and the I too knows (ITKs) will blurt out an answer so damn fast, but mr smarty pant will think and think, and you would think they are clueless, but when they final answer, you can't tear apart their answer.

We can build a machine that has human intelligence and run it on a 2ghz process. The only issue is that instead of answering a question in a second. Perhaps it will take 1 or 2 hours to deliver an intelligence reply. But it should be able to pass a turing test with time thrown at the window.

Go read what 3D researchers said about graphics in the 70's. I bet they believed a 10ghz was good enough for real life 3D graphics.

What is hindering us is not speed, but our approach to AI research.

Re:hrm

[segmond]

I don't think CPU should be designed for AI-style algorithms, when the said algorithms have not been proven. Assume we finally suceed in implementing the Holy Grail of AI right, then we can seek out ways to optimize and make it fast, thus custom CPUs will come in. Right now, most of the algorithms are a joke.

Re:hrm

[Chatsubo]

What is be really interesting is that when we get these human-brain-equivalent machines, the technology does not stop there.

So the intelligence level of this thing would prob. double in accordance to Moore's law, and in a year outclass it's master two fold. In about another year it will be four times as intelligent as any human being. And, of course, it doesn't stop there....

The implications that this would have on society would be very interesting. Would we believe everything it told us, or claimed that we know better? Would we like all the answers it gave us. Would it start deceiving us for our own good? etc.

Re:hrm

[segmond]

Everything works in theory, but not pratice.

Re:hrm

[Zenmonkeycat]

We'll know when we've hit that mark when every output to the console is accompanied by either "pathetic hacker" or "insect."

Re:hrm

[hackstraw]

Hans Moravec's book "Robot" suggests that 100 teraflops is about the level required for human intelligence.

Yeah. I've been waiting for years for those dumbasses to make a computer that can outperform my ability to perform 100 trillion double precision floating point operations a second flawlessly.

Re:hrm

[Randy Wang]

I, for one, welcome our new Beowulf overlord...

Re:hrm

[TimothyTimothyTimoth]

If you are thinking along these lines you might already be aware of this link, but if not, might I recommend:

http://singinst.org/index.html

Re:hrm

[diersing]

I have a question from a casual observer who comes across this Hokie machine and the top 500 list every now and then. What is it these computers do?

Hearing it referenced in terms of AI helps, but is that the only purpose for a research facility to build one of these mammoths? Are there practical applications for the business world (other then the readily available (read commercial) clustered data warehousing)?

I'm not trolling, just curious.

Re:hrm

[Wudbaer]

It would be similar to modelling the global weather system, a software capability we already have.

Where do we have this amazing capability ?

I mean not only have a rough model ignoring a lot of important influences on weather like water temperatures in the Oceans etc. on a very rough grid , like we have now, but a really accurate weather model.

A recent article I read about NEC's Earth simulator stated that even if this amazing machine was supposed to deliver beneath other things climate calculations with unprecedented accuracy and comprehensiveness it still came short of this ambitious goal. Quite short.

Re:hrm

[benhocking]

Actually, it's not quite that simple. As someone whose research is in modeling the hippocampal region CA3 (about 2.5 million neurons in humans, 250k neurons in rats), I can tell you that the connectivity of the system is a very important variable. And there is still much we don't know about the connectivity of the human brain. Furthermore, there are hundreds of different types of neurons in the human brain. Why so many different types if only 2 or 3 would do? Seems evolution took an inefficient path - unless, as is probably the case, the differences in the neuron types are crucial for the human computer to work the way it does. Granted, some differences might be due to speed or energy efficiencies which are not absolutely critical for early stages, but I suspect that many differences have to do with the software (or wetware in this case) that makes us intelligent.

After we've solved that minor problem, I think teaching the system will be relatively trivial. I.e., if we understand the wetware enough to reconstruct it, we most likely understand how its inputs relate to our inputs, etc., and we could teach it much the same as we teach a human child. Of course, we might also figure out a better way to teach it, and in so doing we might even find a better way to teach human children. (Some of our research has recreated certain known best learning strategies, it is probably only a matter of time before simulators disover a better one!)

Re:hrm

[Gentlewhisper]

Hans Moravec's book "Robot" suggests that 100 teraflops is about the level required for human intelligence. So we are up to 10% of his target. But human intelligence still seems very far away, so either he has badly underestimated, or our collective programming skills need significant improvement.

Judging from the fact that Australians voted for John Howard as Prime Minister.. Nah! Not gonna need that much!

Re:hrm

[Chatsubo]

I was not, and it is quite interesting. It made me think of other possibilities. Like augmenting your natural intelligence with a ArtIntellicorp 5x Brainpower Module (patent pending) completely designed by AI.

Also, this site actually breaks down my previous argument. The rate of improvement would not stick to Moore's law, because that's the rate human intelligence has been improving technology. Once you have intelligence levels above that, Moore's law should look comparatively slow.

Food for thought, thanks.

Re:hrm

[Glock27]

By the way, IBM BlueGene/L is going to produce 360 teraflops by end 2004, so if the report of Moravec's estimate is correct, and he is correct, that AI Overlord welcome could be pretty soon.

If you read the article (I know, I know) you'll find that the peak performance of this Cray system is 144 teraflops with 30,000 processors.

Re:hrm

[TheLink]

Any comments on the recent news on the artificial hippocampus (link #1 link #2)?

Re:hrm

[autophile]

His estimate was probably based on the common, and incorrect, belief that neurons are purely digital.

Worse... even if they're analog, they're probably noncomputable.

--Rob

Re:hrm

[fitten]

Depends on the site and their main focus. The Earth Simulator in Japan (#1 on the list) for example, is used to simulate and predict weather. Various machines at some of the national labs in the USA are used to simulate nuclear events. Some other machines in the biotech industries are used to do protien folding and things like attempting to simulate a human cell. Financial institutions use them to attempt to predict the economy, the stock market, and the like. Automobile manufacturors use them to simulate crash tests. Aeronautic firms use them to simulate new vehicles.

In the past, there has been talk about companies that exist solely to supply compute power. Such a company would have a warehouse full of computers and control them through schedulers (batch, etc.) and sell time on the machines to anyone who wanted it. So far, I don't think anyone has been successful with the idea yet.

Re:hrm

[menem]

ASICS are the key.. At it's core, artificial intellegence is a very very complex signal processing problem. High speed signal processing algorithms are typically simulated on computers and implemented in ASICs. I would guess we could probably get 10-100x the power by implementing the AI algorithms in ASICS. The key is to get the algorithms to work first.

Re:hrm

[autophile]

According to Wired...

Now that the upgrade is complete, System X is being used for scientific research. Varadarajan said Virginia Tech researchers and several outside groups are using it for research into weather and molecular modeling. Typically, System X runs several projects simultaneously, each tying up 400 to 500 processors.

"At the end of the day, the goal is good science," he said. "We're just building the tools. The top 500 is nice, but the goal is science."

--Rob

Re:hrm

[MacGod]

Hans Moravec's book "Robot" suggests that 100 teraflops is about the level required for human intelligence. So we are up to 10% of his target. But human intelligence still seems very far away, so either he has badly underestimated, or our collective programming skills need significant improvement.

Well, that's probably because it requires more than just computing hardware. (Disclaimer: I haven't read the aforementioned book). Right now, programming is nowhere near enough to simulate a human intelligence, even if we had the hardware to pull it off. It's not like 100 teraflops is some magical goal where AI just "appears" magically as soon as we hit 100 teraflops. We need to know how to simulate "intelligence" in some form, or else the 100-teraflop-computer will just be a really really fast calculator (or weather simulator, or whatnot).

That's the biggest hurdle in AI: coming up with something that can really think for itself; adapt and learn, and maybe even evolve. The hardware is just a matter of time; but we need to know what to do with that hardware.

Re:hrm

[TimothyTimothyTimoth]

Sorry about that. You are absolutely right. I was typing fast and it was an easy metaphor that came to mind.

I meant that it is a software methodology we fully grasp (as compared to designing AI software ab initio.) Practically, as with weather simulation, scanning a brain and creating an accurate simulation would be an enormous practical accomplishment, and is bound take several years of worldwide research once the hardware is available. Not to mention the moral, political and legal issues and frameworks around artificial minds and their rights.

Thanks for the reality check.

Re:hrm

[benhocking]

I doubt that it will be entirely successful, but will be happy if I'm proven wrong. Nevertheless, I'm certain that we (as a community) will learn much by studying its shortcomings. I'm really excited about the project!

I hasten to add that when they claim

The brain region they are trying to replace is the hippocampus, which is vital for forming memories. The hippocampus has a well-understood three-part circuit. It also has a regular repeating structure, so elements of all three parts of the hippocampal circuit can be kept in a fully functional state, even in small slices in a culture dish.

my reaction is "I don't think that word [well-understood] means what you think it means."

Re:hrm

[Anonymous Coward]

I, for one, welcome our new Beowulf overlord...

Hey, wait a minute; that is actually funny! Is that allowed?

Re:hrm

[lawpoop]

Of course you can't do it consciously, but that's about what you do when you catch a ball or walk down the sidewalk.

Re:hrm

[ca1v1n]

I'm not really sure what you mean that they haven't been proven. In the sense that they don't give the best answer all the time, this much is obvious. That's why we call it artificial intelligence instead of algorithmics. That said, we know quite well that they work. Most adaptive spam filters are based on Bayesian networks. The best of these are better than humans at identifying spam. We don't typically run the best because the computational load is far too high. Bayesian networks have a delightfully simple evaluation procedure that is basically glorified matrix multiplication. Neural networks are a little more complicated, but not by a whole lot. Recall a recent development that used a neural network inside an 802.11 driver to predictively avoid collisions to improve total network throughput in dense environments. It doesn't reduce collisions to 0, as that would require clairvoyance, but it does a good job. You didn't hear about this 5 years ago because putting a neural net inside an 802.11 driver without killing performance to both network and computer is difficult, particularly without processor instructions dedicated to the task.

It's true that designing a CPU to *be* a neural or bayesian network is infeasible, but that doesn't mean we can't add instructions to accelerate their evaluation. The evaluation and update of a neural net, traditional or biologically modeled, is a rather simple algorithmic process, though people who have worked with such simulations (see Ben Hocking's post above, he was my quite capable AI TA) will tell you that they make rather obscene optimizations to make it run reasonably fast. I'm talking about things that might sound familiar to graphics people, like removing all multiplications from a program that's supposed to be doing them more than all other operations combined. It's a particularly good candidate for SIMD instructions. Most large neural nets are sparsely connected, so even if your net is substantially larger than your cache, you can beat that with prefetching. Threshold conditional addition is an example of something that can be done very quickly in hardware, and is much more of a pain to code and optimize in software.

If you prefer RISC to CISC, recall that even the original SPARC had special DSP instructions. Putting the sigmoid function and arctan on silicon is really not all that outrageous.

Re:hrm

[American AC in Paris]

He is wrong. Intelligence is not about speed. I have met people who are very very smart, but they think very slowly. You ask questions, and the I too knows (ITKs) will blurt out an answer so damn fast, but mr smarty pant will think and think, and you would think they are clueless, but when they final answer, you can't tear apart their answer.

That's because it takes no time at all to answer a question, whether you're a man or a machine. To provide an accurate or informed answer, though, takes time. It takes time, because it takes thought--and faster thought means faster answers. While it may seem that the intelligent man is thinking slowly, his mind is likely racing--testing hypotheses, finding logic holes, computing results, and building a rock-solid answer.

In this respect, intelligence is about speed--if it takes a man twenty seconds of thought to give an accurate answer the question, "Which of these three objects is round?", he's not likely to be considered terribly intelligent--despite the fact that his answer is accurate and informed. True, intelligence is not about speed alone, but speed is unquestionably a factor. The faster we can think, the better. Consider that had his mind been even faster, "Fermat's Last Theorem" could have been "Fermat's Greatest Proof".

We have a lot of work to do in the realm of AI research, but every last tick of speed helps. Even at that, there are limits to what we can achieve with algorithms; at some point, it's going to be a matter of "is the computer fast enough to perform full object recognition in real time?"

Intelligence is about many things. Speed is one of these things.

Re:hrm

[ca1v1n]

But their aggregate behavior is quite easily computable. In the human brain, 70% of neuron firings randomly fail to register in their successor. This not only makes our behavior somewhat random, but also implies that there's quite a bit of redundancy and that our brains operate on aggregate behavior of a large neural net, rather than precise behavior of a small one, otherwise we'd be completely unpredictable, rather than just mostly unpredictable. While it's true that you can't model a human brain reliably with a computer, it's also true that you can't even model it reliably with the same human brain. Generally speaking, any simulation that is as good as reality is good enough, even if reality isn't really right.

Re:hrm

[Gilmoure]

Look at graphic chips. They're specialized and can be used for other type of work.

Re:hrm

[TheLink]

They said they've got it to 95% accuracy. But that's for rat samples.

What if it only works for 95% of the people who aren't really exceptional mentally.

Or maybe it only works for narrow-minded people ;).

Re:hrm

[jimicus]

OK, clearly the bit I did on AI was slightly over simplified.

Re:hrm

[bhtooefr]

Oh, people have been successful with the idea. However, the proliferation of cheap personal computers fscked it up.

Re:hrm

[xenoandroid]

I believe in psychology you learn that there are two types of intelligences. The trait of one is that they're fast and therefore good at interacting with other people. The other is slower, but is better at coming up with solutions to complex problems.

Intelligence can be about speed. If you want to make a machine capable of holding human conversations, one that passes the turing test, then you must have speed. If you want to make a machine capable of intelligently finding solutions to problems in the world then you can throw speed out the window.

How much of your daily conversation is actually a memorized response or involves something you had already spent time thinking about before.

"What do you think of killing?"
Well if you never had a reaction to killing before, never heard anybody's opinions about it, and didn't understand what the entire result of killing was, you probably wouldn't be able to give me an instant answer. The moment you associate killing with taking something very important from someone and learn that the great majority of other people say killing is wrong you have an opinion which you return in speech as, "Killing is wrong."

What you need is a machine capable of discovering and thinking about things on it's own without being prompted by direct human input.

It doesn't have to be fast to reach human intelligence, it just has to be capable of every type of thinking humans have.

How about driving recklessly? Well by itself driving recklessly isn't immediately seen as bad. But when you can think about possibilities and effects. For example, you lose control of your car and run over a little kid then you connect it to killing which you already decided was wrong. Driving recklessly is now wrong. Computers can already make connections between data sectors on a hard drive very fast. Making the connections become more difficult when you factor in, "Driving recklessly is fun though..." Then it becomes a balance of values. Which is more important, not killing someone or having fun?

That kind of thinking is a little too complex to do in casual conversation. A lot of programmers experimenting with A.I. for fun are still designing with a think on the fly model. The program takes human input and then suddenly it decides to think. Of course you'll need raw power if you build that way.

I also believe our hinderance is our approach to A.I. Our own brains are never really idle, even when we space out, why should an A.I. be?

Re:hrm

[GlassHeart]

it should be able to pass a turing test with time thrown at the window.

No, it cannot, because the Turing Test is interactive. What you've done instead is to weaken the Test so it is more easily passed. If something cannot instantly answer "what's your name?" you would not think it is human (without any other input). On the other hand, if it instantly answers "what is the square root of 19378189237123?" you may also realize that it's not human. Time is very much a factor.

Re:hrm

[hunterx11]

Would we like all the answers it gave us[?]

Of course not. How are the philosophers going to get booked on talk shows?

Re:hrm

[BorgCopyeditor]

How are the philosophers going to get booked on talk shows?

As a professional philosopher, let me ask what talk shows these are that you are watching. Also, what are the phone numbers of their producers?

Re:hrm

[Scott7477]

Actually, we don't have the capability to fully model the global weather system. The state of the art models that exist now can only predict with any accuracy at all out to about ten days. Check out the NOAA Climate Prediction Center's website...they have difficulty predicting whether an El Nino will occur in any given year even six months in advance. Now for my soapbox: IMHO in general computer folks grossly underestimate the complexity of the human brain and brain researchers routinely exaggerate the level of their understanding of the brain to lay people.

Re:hrm

[AI_Overlord]

Thank you, it's nice to be here!

Re:hrm

[GrahamCox]

If it turns out that the human brain does indeed have 20 petaflops (or so), what will be remarkable is its efficiency, both in terms of size, but most importantly in terms of energy consumption. A couple of thousand kCal a day for it and its support system - pretty good!

Re:hrm

[ca1v1n]

There are actually services that have people read your mail for you and filter it out themselves. These automated filters can actually do better than that, which is good, since I can't imagine any business tolerating the notion of outsiders reading every last one of their email messages.

Speed at top

[luvirini]

Reflecting on the comment: "hould still be in the top 5 when the new rankings come out in November." There seems to be a serious push for multiprosessor systems, currently the ranking seem to consist of a couple of stars, few big ones(this computer among them) and a huge group of third category, and then the "used to be great" computers. But from my reading of the trends seems that there will be more and more crowding at near the top, so I expect the second category to be much larger, with much smaller differences.

Re:Speed at top

[TAGmclaren]

currently the ranking seem to consist of a couple of stars, few big ones(this computer among them) and a huge group of third category, and then the "used to be great" computers

That's an interesting way of looking at it, but I think so far most of the commentators have failed to pick up what makes this system so incredible. Srinidhi Varadarajan, the designer of the system:

Varadarajan said competing systems cost $20 million and up, compared to System X's approximately $5.8 million price tag ($5.2 million for the initial machines, and $600,000 for the Xserve upgrade).

"We will keep the price-performance crown," he said. "We don't know anyone who's within a factor of two even of our system. We'll probably keep the price-performance lead until someone else shows up with another Mac-based system."

Think about that for a second. The system isn't just in the top 5 (or at least top 10), but it's the cheapest by a factor of at least 2. What's even funnier from a tech standpoint is that the creator doesn't expect it to be beaten until another Apple system is built - which puts a very interesting spin on the old "Apple's more expensive".

Anyway as to in/out of the top 5, Varadarajan reckons there's another 10-20% in optimisations left in the tank...

Data taken from the recent Wired Article on the subject.

Re:Speed at top

[carnivore302]

Actually in a similar article, it states that "Released Tuesday, the 12.25-teraflops benchmark would put System X in fourth place in the world ratings, but it will probably be surpassed by new supercomputers from NASA, IBM and others. ".

Srinidhi Varadarajan, System X's lead architect said ""We expect to be in the top 10. Where, we don't know. Top five is not possible, probably".

So, at least that's different from what was stated on the slashdot story.

Re:Speed at top

[Anonymous Coward]

The system isn't just in the top 5 (or at least top 10), but it's the cheapest by a factor of at least 2.

The $5.8M number is how much the computers (and maybe racks) cost, not the whole system. AFAICT, that number appears leaves out US$2-3M worth of InfiniBand hardware that somebody (probably Apple) must've "donated" so it wouldn't show up as part of the purchase price. IB gear costs ~US$2k/node in bulk, on top of the cost of the node itself. It's highly unlikely someone else could build this exact configuration for US$5.8M without serious underwriting or hardware donations. Heck, I can't even get the Apple online store to give me a price on a G5 Xserve that includes an education discount, and I work for a fairly large public university.

Re:Speed at top

[aalobode]

Varadarajan said competing systems cost $20 million and up, compared to System X's approximately $5.8 million price tag ($5.2 million for the initial machines, and $600,000 for the Xserve upgrade). Cray and other commercial companies have fixed costs (buildings and grounds, benefits, salaries, depreciation) that work their way into the cost of a machine. At a public university, these dont show up in the cost of a machine. I would think that $5.8 Million would be only the h/w cost, and not the total cost. Additionally, we see that Apple built special chips for this purpose and that would imply another price break. Price crown aside, the machine has plenty of merit. Maybe more should be built so that average users can run the GUI for Longhorn.

Re:Speed at top

[jurv!s]

To navigate to the Apple education store, DON'T start in the store. From the main page, select Education from the sub-nav. In the upper right corner, select "Click." From here you will be asked to select your affiliated university and if it's a personal or uni purchase.

I agree this is more confusing than it need be, but then again- the prices are generally hundreds of dollars lower and they never seem to check your affiliation. So yeah- I would make it difficult to find too...

ps Apple's education discount on the Xserve dual processor compute node is $500 or roughly 16% off!

Re:Speed at top

[Jeremy Erwin]

I don't know how much Dell's Tungsten cluster cost but those guys went online last year and got ranked #4 (just behind this Mac cluster) and they're #5 or something now. These bozos have spent a year fscking around with upgrades and from the theoretical #3 (as they were taken out since the cluster couldn't enter production) will have dropped to #7 or more in the next ranking....
My my. you're hostile.

Tungsten is a 9.8 TFlop machine. It's interesting, but it's not as powerful as the 12.5 TFlop VT machine. I don't see the point of having a standard Intel+GigE+Linux cluster "standard." Standards promote complacency.

Besides, 10 TFlop computers are old hat. NASA has a 52 TFlop (Rpeak) monster called "Project Columbia." It looks like BlueGene's designers might just have to settle for a #2 rating after all. Perhaps they have yet to release their final benchmark?

Density

[GerbilSocks]

VT could theoretically pack in 4x the number of nodes in the same space that occupied the original System X. Could we be looking at at least a 50 TFlop (minus 10% overhead) supercomputer with 8,800 cluster nodes?

If that were feasible, you could be looking at toppling Earth Simulator at a fraction of the cost.

Re:Density

[Anonymous Coward]

At linpack. Of course, the Earth Sumulator wasn't built (just) to run linpack.

Also, the Earth Simulator has been around for how many years? 2? 3? Quite frankly, it would be downright embarrassing if it couldn't be toppled at a fraction of its cost by now.

Re:Density

[Ingolfke]

And if we could harness the heat from this machine we could probably power most of the North Eastern United States.

Re:Density

[UnknowingFool]

Not necessarily. Processing power doesn't really scale linearly like that. Add 4 times as many processors doesn't mean the speed will increase 4x.

First, as they try to increase the speed of the system, the bottlenecks start becoming more of a factor. Interconnects is one big obstacle. While the new System X may use the latest and greatest interconnects between the nodes, they still run at a fraction of the speed that the processors can run.

Also the computing problems that they are trying to solve may not scale either with more processors. For example, clusters like this can be used to predict and simulate weather. To do so, the target area (Europe for example) is divided into small parts called cells. Each node takes a cell and handles the computations of that cell.

In this case adding more processors does not necessarily mean that each cell is processed faster. Getting 4 processors to do one task may hurt performance as they may interfere with each other. More likely the cell is further subdivided into 4 smaller cells and the detail of the information is increased not the speed. So add 4x processors only increases data 4x but it doesn't mean that the data is solved any faster.

Re:Density

[alfalfro]

Other factors:
1) Cost, narf.
2) Power, we already have the biggest uninterruptable power supply in the state. If we added another I think our small college town would experience "rolling blackouts."

Re:Density

[koi88]

Of course, the Earth Sumulator wasn't built (just) to run linpack.

I think most super computers weren't built just to run benchmark tests.
Well, at least I hope.

Re:Density

[luvirini]

Indeed, Breaking up computational tasks to smaller pieces that can be processed by these architectures is on of the biggest challenges in the high end computing.

Many processes are indeed easy to divide to parts. Take for example ray-tracing, you can have one processor run each ray if you want, getting huge benefits compared to singleprocessor designs. But many tasks are such that the normal way of calculting them requires you to know the previous result. Trying to break up these tasks is one of the focuses in the reserearch around supercomputing.

"Dick factor" aside

[ceeam]

Would be interesting to know exactly what stuff do these machines do? Maybe they would even be able to share some code so that people can fiddle around with it optimizing (should be fun).

Re:"Dick factor" aside

[millahtime]

Currently they aren't doing anything with them except getting them up and running. Status is listed at...
Assembly - Completed!
System Stablization - In Progress
Benchmarking - In Progress

When up and going the system will probubly do some high end scientific calculations.

Re:"Dick factor" aside

[joib]

Would be interesting to know exactly what stuff do these machines do? Maybe they would even be able to share some code so that people can fiddle around with it optimizing


I don't know about the VT cluster specifically, but here's a couple of typical supercomputer applications that happen to be open source:

ABINIT, a DFT code.

CP2K, another DFT code, focused more on Car-Parinello MD.

Gromacs, a molecular dynamics program.


(should be fun)


Well, if optimizing 200 000 line Fortran programs parallelized using MPI sounds like fun to you, jump right in! ;-)

Note: Above applies to abinit and cp2k only, I don't know anything about gromacs except that it's written in C, not Fortran (though inner loops are in Fortran for speed).

Oh, and then there's MM5, a weather prediction code which I think is also open source. I don't know anything about it, though.

Re:"Dick factor" aside

[TAGmclaren]

Currently they aren't doing anything with them except getting them up and running

Their site is out of date then: http://www.wired.com/news/mac/0,2125,65476,00.html ?tw=newsletter_topstories_html

Now that the upgrade is complete, System X is being used for scientific research. Varadarajan said Virginia Tech researchers and several outside groups are using it for research into weather and molecular modeling. Typically, System X runs several projects simultaneously, each tying up 400 to 500 processors.

If there's a Wired article, and a Cnet article, go with the Wired article every time. It's written by people who love tech.

Re:"Dick factor" aside

[ghutchis]

Yeah, talk about Car-Parinello. Great stuff, but I know past versions have sucked up >1GB per node for even small jobs. But I'd love to get my hands on some CP simulations with 400-500 CPUs at once.

Other open-source comp. chemistry packages include MPQC (Massively Parallel Quantum Chemistry): http://www.mpqc.org/

-Geoff

So compare it to......

[ericdano]

The school said it spent about $600,000 to rebuild the system and add the additional nodes. The original cost of System X was $5.2 million.

Compare it to this new Cray system. Bang for the buck would make the Apple system better.

Re:So compare it to......

[Coryoth]

Compare it to this new Cray system. Bang for the buck would make the Apple system better.

Yup, except the Cray comes with far superior interconnect technology, a better range of hardware and software reliability features built in, software designed (by people who do nothing but supercomputers) specifically for monitoring maintaining and administrating massively parallel systems, and most importantly it all works "out of the box". You buy a cabinet, you plug it in, it goes.

Why do these Apple fans, who justifiably claim that comparing a homebuilt PC to a "take it out of the box and plug it in" Apple system is silly, want to compare a build it yourself supercomputer to one that's just plug and go?

And yes, comparing MacOS X to UNICOS for supercomputers is like comparing Linux to OS X for desktops (in fact that's very flattering to OS X as a cluster OS).

Jedidiah.

Re:So compare it to......

[capmilk]

Bang for the buck would make the Apple system better.

Sure, but what would you rather say: "I just bought an Apple computer" or "I just bought a Cray computer"?

Re:So compare it to......

[Quila]

Why do these Apple fans, who justifiably claim that comparing a homebuilt PC to a "take it out of the box and plug it in" Apple system is silly, want to compare a build it yourself supercomputer to one that's just plug and go?

Apple does build a plug-and-go cluster for bioinformatics. And they're relatively cheap, $75K for a 16-node (32-proc) cluster. Of course three XD1 cabinets (36-proc) would probably run circles around the Mac cluster with their better memory architecture and fast interconnects, but at what price? I've heard "Starting at under $100,000" and I doubt very much under that number.

The Mac is just the great modern way of doing easy clusters cheaply.

Re:So compare it to......

[Overzeetop]

You do know that $5.2m was for a trailer-full of unconnected parts, right? Just like the $100,000 satellites that universities "build," this one would cost far more in the real world. A Habitat for Humanity house may cost only $20,000, but I can guarantee you won't find the same new house for sale for that kind of money on the open market (try 80-100k for a 20k equivalent HH house).

Re:So compare it to......

[sch7572]

But are they taking into account the maintenance and support costs? As it stands now, I believe its mostly done by grad students on "free pizza". Administration costs in production-scale systems typically can run up to 5 times the original equipment acquisition costs. Wonder why no one talks about that aspect?

Re:2.3GHz?

[Ford Prefect]

But the XServers come at 2.0GHz, with the desktop powermacs at 2.5GHz. Is this a mistake?

From the article:

Apple said last week that the 2.3GHz machines were a one-off deal for Virginia Tech and not something the company plans to announce for broader consumption anytime soon.

What I really want to know is what they do with the old machines. The articles speaks of the cluster being 'upgraded' - are the older G5s replaced, or do they just become part of the new cluster?

Still, I suppose there's one or two unwanted G5s - anyone want to send me a couple? :-)

Crays...

[CaptainPinko]

are not designed for the same type of work as clusters. If a probably is not effeciently parallizable and requires shared memory then a Cray is the only feasible option A Cray is not a cluster. It's like comparing mph for a sports car and truck: the car is faster but they are meant for different types of loads.

Re:Crays...

[Coryoth]

are not designed for the same type of work as clusters. If a probably is not effeciently parallizable and requires shared memory then a Cray is the only feasible option A Cray is not a cluster. It's like comparing mph for a sports car and truck: the car is faster but they are meant for different types of loads.

To be fair to the original poster, the Cray system he was referencing is a cluster system. Then again, its a cluster system with very impressive interconnects for which System X just isn't comparable (ie. The Cray system will scale far far better), not to mention the Cray software (UNICOS, CRMS, SFW), and the fact that the Cray system is an "out of the box" solution. So you are right, there is no comparison.

Jedidiah.

Re:Crays...

[ericdano]

So you are saying Apple is the sports car, and the Cray is the truck?

The list of Supercomputers

[ehmdjii]

this is the official homepage of the listing:

http://www.top500.org/

Re:2.3GHz?

[mmkkbb]

they were sold off by MacMall at a slight discount around 6 months ago, along with a certificate of authenticity and a "property of virginia tech" sticker

Re:2.3GHz?

[jdwest]

They were sold through retail channels with the addition of a metal nameplate stating its node number.

Obligatory:

[Dorsai65]

but will it run Longhorn?

Old stuff...

[gustgr]

Before you guys ask I RTFA. I was wondering what do they do with the old processors?

Re:Old stuff...

[Anonymous Coward]

If you're referring to the old G5 Powermacs used in the original System X...they were sold. I bought one!

and yet...

[BobWeiner]

...it still doesn't come with a floppy disk drive.

/sarcasm

Re:and yet...

[Knx]

Yeah, but ya know, you can plug 2300 USB keys! Woohoo! See here.

Re:and yet...

[Short Circuit]

That's a big RAID Array...

Re:and yet...

[xenoandroid]

Yeah we all know that users will keel over and die if they don't have a second mouse button. I kind of wish Windows was designed like OS X to support both one button and two button mice, then I could just say "press control while clicking" to these people who I have to explain what a "left-click" is.

Really, stop bitching about it, if you have that much trouble learning how to use one mouse button then maybe you're stupider than the people who can't use two button mice properly. Why don't you go bitch about something even stupider to complain about like how modern laptops only have one button that turns the machine both on and off or the fact that you have to press shift or caps lock to get uppercase letters. We all know how crippling that is. Oh come on, you know you want to have a contextual peddle in your car, one that lets you select different accelerator options: "Accelerate, accelerate one wheel, red line, about this engine, accelerate the car next to you"

Really it's not that hard, in fact i wouldn't be surprised if Apple's no button mouse is better for avoiding repetitive stress injury, you could operate the mouse functions of their computers fairly easily without moving separate fingers.

School Funding

[BrianHursey]

This kind of funding I wish my school had. We are going to build a cluster system out of about 20 G4's running yellow dog Linux. But this is my chance to actually do cluster programing.

The funny thing is with the class we are actually trying to figure out things to compile, besides bootstrapping our Linux laptops. =P Man we are geeks.

Re:Wow!

[Knx]

Correct me if I'm wrong, but it actually looks an awful like a Beowulf cluster by nature.

Oh, and btw: here are some pictures.

Thank you VT

[Alcimedes]

I have it on good insider knowledge, that this entire cluster is going to be put to the best possible usage.

Not disease solving, not genetic mapping, not calculating weather patterns.

No, what they're going to do is remaster the Original Star Wars series, right from the laser disc versions!!!!

Imagine, a digitallly remastered bar scene where Han shoots first!!@$!@#!one!@

/kidding

Re:Thank you VT

[MacGod]

No, what they're going to do is remaster the Original Star Wars series, right from the laser disc versions!!!!

So, the question of how much it would cost to get an unadulterated version of Star Wars is finally answered: it would cost $5.8million.

Alright Slashdot, everyone chip in ten bucks!

Re:Thank you VT

[sootman]

I'd've been happy for the help. :-)

Re:Article Comparison...

[erick99]

If power was only equated to speed then you would be correct. However, as other posters have pointed out, there are several reasons why a Cray is a more powerful system besides sheer speed.

What do they do with it?

[moberry]

I have a freind who is finishing up his masters, and starting his PhD in computer engineering at VT. I asked him about it and he simply said: "they haven't found anything do actually _do_ with it"

much the same as a baby

[nounderscores]

2:14am EDT August 29, 1997...

Researcher: "Go to your machine room! And no Command and Conquer until you do your homework!"

Joshua:"Oh yeah? Would you LIKE TO PLAY A GAME?"

What is a supercomputer ?

[Animaether]

I'm curious as to the answer to the question (What is a supercomputer ?).

The reason is this.. more and more of these 'supercomputer' entries appear to be many machines hooked up together, possibly doing a distributed calculation.

However, would projects such as SETI, GRID, and UD qualify with their many thousands of computers all hooked up and performing a distributed calculation ?

If not, then what about the WETA/Pixar/ILM/Digital Domain/Blur/You-name-it renderfarms ? Any one machine on those renderfarms could be put to use for only a single purpose: to render a movie sequence. Any one machine could be working on a single frame of that sequence. Does that count ?

I seem to think more and more that the answer is 'no', from my perspective. They mostly appear to me as rather simple computers (very often not even the top-of-the-line in their own class), with the only thing going for them that there are many of them.

The definition of supercomputer (thanks Google, and by linkage dictionary.reference.com ) is :

A mainframe computer that is among the largest, fastest, or most powerful of those available at a given time.

And for mainframe :

A large powerful computer, often serving many connected terminals and usually used by large complex organizations.
The central processing unit of a computer exclusive of peripheral and remote devices.

Doesn't the above imply that a supercomputer should really be just a single computer, and not a network or cluster of many computers ?
( The mention of 'terminals' does not mean they're nodes. Terminals are, after all, chiefly CPU-less devices intended for data entry and display only. They are not part of the mainframe's computing capabilities. )

If the above holds true, then what is *really* the world's top 3 of supercomputers ? I.e. which aren't 'simply' a cluster of nodes.

Any mistakes in the above write-up/though process ? Please do point them out :)

Re:What is a supercomputer ?

[joib]

I don't think there exists any non-ambigous way to define what a supercomputer is.

Anyway,

I think we can disqualify @HOME style projects, since the individual nodes are not under the control of the manager. Similarly, you can't submit some small batch job to a @HOME system and expect to have results within a short time. Uh, that wasn't a very good description but I hope you understand what I mean.. i.e. that to qualify as a supercomputer all the nodes should be dedicated to the supercomputing stuff, and be under the direct control of the administrator.

As for the one node vs. cluster of nodes, it gets trickier. How do you define one node? Shared memory? But then, what about NUMA systems such as the SGI Altix? It is entirely valid to view NUMA systems as consisting of multiple connected nodes, along with some kernel (and usually hardware) support to make it appear as shared memory. Hardware-wise there's no huge difference between such a system and a cluster, essentially the only major difference is that NUMA systems typically have some silicon to take care of cache coherency.

Or should we limit ourselves to shared memory systems where all the memory sits on the same bus? This limitation would seriously limit our ability to build really huge systems, simply because the speed of light would cause ever bigger latencies. Not to mention that this limitation would prohibit even a simple dual cpu AMD Opteron system, which is a NUMA system. So I don't think this limitation is good either.

Of course, we could say that a real supercomputer is distinguished by running a single kernel for the entire system. That would allow NUMA systems, but disallow clusters. Anyway, I think this limitation sounds a bit artificial.

In light of the above reasoning, I think we must accept clusters as legitimate supercomputers. As long as they have enough oomph to make the top500 or thereabouts, that is. Not that linpack is any perfect benchmark, far from it. Oh well, perhaps HPC Challenge or something like that will someday replace linpack as the "official" benchmark for top500.

Re:What is a supercomputer ?

[log0n]

[quote]Doesn't the above imply that a supercomputer should really be just a single computer, and not a network or cluster of many computers ?[/quote]

But if all of the networked/clustered computers are all working on the same task with information flowing between nodes dependant on other nodes processing , doesn't that make them all effectively one large computer?

A renderfarm is similar in many ways to a supercomputer, but I wouldn't think of it as one. Renderfarm nodes generally work on a specific task that is assigned to them. They can be of a larger over all project (like rendering for a film), but really they only process what is given to them then spit the info back. There's a queue manager that sends out tasks, but very little of the information that gets processed by a node is dependant on information that is in use by another node. A renderfarm basically gives out raw processing for a task when requested, it doesn't do much beyond that.

You can still have multiple terminals for data in/out, and in the VT case these are definitely systems that are exclusive of peripheral devices (remote device doesn't make sense - a connected terminal is a remote device).

I do think that definitions have blurred from what they used to be thanks to improving technology, but I do think that the generalities of what they represent are still valid.

$.02, etc

Re:What is a supercomputer ?

[fitten]

The reason is this.. more and more of these 'supercomputer' entries appear to be many machines hooked up together, possibly doing a distributed calculation.

However, would projects such as SETI, GRID, and UD qualify with their many thousands of computers all hooked up and performing a distributed calculation ?

If not, then what about the WETA/Pixar/ILM/Digital Domain/Blur/You-name-it renderfarms ? Any one machine on those renderfarms could be put to use for only a single purpose: to render a movie sequence. Any one machine could be working on a single frame of that sequence. Does that count ?


Yes, all of these mentioned belong to a class of supercomputer applications called "Embarassingly Parallel". These types of algorithms are (by far) the easiest to implement since their calculations don't depend on anything being calculated by other nodes in the system. They are characterized by minimal/no communication among nodes (many times, just the communication to hand the node the data on which it is to compute and then one communication at the end to submit the results back to the central node) and lots of compute resources working on the local data. So, they *are* supercomputing of a type, just one that isn't that interesting from a computer science point of view.

There are many problems that require much more communication between the nodes. Calculations performed by one node is dependent upon the results generated by other nodes in the system. Some known solutions require so much communication/synchronization between nodes that it isn't practical to parallelize the problem and a serial solution is more optimal. There has been lots of work on various problems creating algorithms that are more "parallel friendly" in order to speed the solutions. There are some problems that have prizes for someone who can invent a way to make them more parallel.

Anyway, there are many "grains" of parallel computing. The "granularity" of a problem is the ratio of the amount of communication required per computation of the solution algorithm. Coarse grained problems are problems that have low communication/computation requirements. Embarassingly parallel problems are an example of this. The amount of communication required by a distributed node running SETI@HOME is very tiny compared to the amount of computation required for each WU. The same can be said for a render farm. Each node in a render farm receives the image to be rendered, then goes off for a while and renders that frame of the movie and hands the result back to the coordinator, which gives the node the next frame to render. Fine grained problems are the other side of the spectrum and require more communication for each computation operation. Solutions to systems of equations are an example. A problem that has to communicate its results to each of its "neighbors" and receive the results from each of its neighbors on each iteration so that the next iteration can be calculated is more finely grained.

Beowulf clusters, with their slow interconnects, are good at solving coarse grained problems. Other systems, like the new Cray with the high bandwidth, low latency interconnects, work better for fine grained problems. In a fine grained problem, a machine with "slow" processors but a fast interconnect may outperform a Beowulf type cluster that has the fastest commodity CPUs available but a slow interconnect.

By the way, LINPack (the benchmark used for the Top500) is a rather coarse grained problem. That's why Beowulf style clusters appear in the list. There are plenty of other benchmarks that could be used where these clusters would have a hard time.

Re:What is a supercomputer ?

[YouHaveSnail]

Doesn't the above imply that a supercomputer should really be just a single computer, and not a network or cluster of many computers ?

So what, exactly, is a "single computer"? Most supercomputers that I can think of use multiple processors. The characteristics of each machine come largely from answers to questions like "How big is each individual processor?" and "How many processors does the machine have?" and "How are the processors connected to each other?" One one end of the scale you've got small numbers of very large, powerful processors, like the IBM 3090 vector processors. The one I knew had six processors, I think, and it filled a huge room. At the other end of the spectrum, you've got the Connection Machine with 65,536 processors connected in a hypercube architecture. I never saw one, but from the picture a CM-2 looks to require only about 30 square feet.

I'm sure you'd agree that an individual Xserve is a "single computer," but the Xserve itself has two G5 processors. And if you want to break it down further, each G5 processor contains multiple execution units, so you could almost argue that even a single processor is not a "single computer." It gets even worse with multi-core CPU's.

What, really, is the difference between a CM-2 and the Virginia Tech cluster? Just the number of processors (the CM-2 had about 30x more), the power of each processor (the 64-bit G5 clearly beats the CM-2's 1-bit processors), and the connection architecture. That, and the fact that the VT machine is more modular, and doesn't look nearly as pretty.

Re:What is a supercomputer ?

[diggem]

Look at it this way.

Cray computers were sometimes housed in several cabinets. The fact that there were several cabinets had nothing to do with the fact that it was still a single computer. They all generally have more than one processor/processor board.

The cluster style computer is not any less singular than that Cray. The hardware linking those machines and the software to run it all is no less complicated than the so called supercomputer. The distinction might be that the compute nodes are split into 2 or 4 processors per 'cabinet' rather than all 16 or 32 or whichever processors being in 1 or 2 cabinets. Semantically, there's no difference between the clustered computer and the specialized super computer.

I think what makes something a super computer isn't so much the form factor as how much can it handle? So I think the 500 list holds water, it's ranked by how much each machine can handle, not just it's peak, but the max sustained throughput. I think the ones that have a max closest to peak are closer to the classic definition of 'supercomputer' but there are some clusters that achieve a max close to their peak. Impressive no matter how you slice it.

There were AI CPUs

[scattol]

For a while there were CPUs specifically designed to run LISP, aka AI . Symbolics was one of the better knowns one.

It failed in bankrupcy. My vague understanding was that the designing dedicated LISP processors was hard and slow and with little resources they could not keep up. Essentially the Symbolics computers ran LIPS pretty quickly given the MHZ but SUN and Intel kept moving up the MHZ faster than Symbolics could keep up. In the end there were not speed advantage to a dedicated LISP machine, just an increase in price. Economics might change eventually. Who knows.

Everyone is getting their rigs ready

[Jakhel]

the 1150-node Hokie supercomputer rebuilt with new 2.3 GHz Xserves now runs at 12.25 Teraflops. The computer, the fastest computer owned by an academic institution, should still be in the top 5 when the new rankings come out in November."

Just in time for the release of Half Life 2. Hmmm...coincidence? I THINK NOT!!!

Re:Everyone is getting their rigs ready

[bigman2003]

You mean, they might be able to run an emulator fast enough, that you can really play games on a Mac?

Good point...

don't forget...

[Geek_3.3]

(those that go to despair.com will recognize this) that "You can do anything you set your mind to when you have vision, determination, and an endless supply of expendable labor." Point being, I'm sure having essentially free labor (sans pizza, of course... ;-) might have cut the price down just a little bit too...

Not to poo poo their efforts, but the whole system was essentially a 'loss-leader' for future supercomputers projects using the G5's and Xserve....

Re:don't forget...

[fitten]

Many of these systems are loss-leaders for the companies involved. The manufacturor (in this case Apple) weighs the benefit of the publicity (advertising) generated by such an endeavor against the actual cost to see if it is worth it. It should be quite easy to verify the amount of discount by just getting the prices from the Apple Store and doing the educational discount. Then factor in some labor costs as well.

This is much like the Munich Linux migration story. The costs that are quoted by Munich don't include the time (cost, salary) of many, many (possibly 100s) of techs that IBM is donating to the project to insure its success. If you add in all of the labor that is "free", the cost would go up quite a bit.

Yes, but...

[midifarm]

this only qualifies for the minimum requirements.

Peace

Actually, VT will be #8 this time around

[daveschroeder]

Prof. Jack Dongarra of UTK is the keeper of the official list in the interim between the twice yearly Top 500 lists:

http://www.netlib.org/benchmark/performance.pdf (see page 54)

There have been some new entries, including IBM's BlueGene/L, at 36Tflops, finally displacing Japan's Earth Simulator, and a couple other new entries in the top 5.

Here's just the top 16 as of 10/25/04:

http://das.doit.wisc.edu/misc/top500.jpg

No matter what anyone says, Virginia Tech pulled an absolute coup when they appeared on the list at the end of 2003: no one will likely EVER be able to be #3 on the Top 500 list for a mere US$5.2M...even if the original cluster didn't perform much, or any, "real" work, the publicity and recognition that came of it was absolutely more than worth it.

Also interesting is that there is also a non-Apple PowerPC 970 entry in the top 10, using IBM's JS20 blades...

The actual use is going to be

[hsmith]

Highly concentrated in bio-systems/informatics. Tech just built a HUGE building for bioinformatics. they plan to be doing a lot of processing for that.

But they also plan to sell out processor time "cheap".

and i must say GO HOKIES

What is the point?

[Bill, Shooter of Bul]

They built the origninal and as you say they didn't perform any real work. So whats the point? Its like rich guys that buy ferraris and never drive them.

Re:What is the point?

[daveschroeder]

Rich guys that buy Ferraris and never drive them don't get untold amounts of recognition, publicity, free advertising, news articles, and the capability to catapult themselves to the forefront of the supercomputing community overnight for a paltry sum of money, thus attracting millions of dollars of additional funding and grants to build clusters that WILL be doing real work, such as the one we're talking about now, and the several additional clusters they plan to build in the future, not to mention the benefit of proving that a new architecture, interconnect, and OS will perform well as a supercomputer, allowing more choice, competition, and innovation to enter the scene, which ultimately results in more and better choices for everyone.

Does that answer your question?

Re:Wow!

[Quobobo]

What, compared to the people who post the Beowulf/Soviet Russia/SCO jokes a million times over? Hard to get a worse sense of humour than them, as even this "new" 7xxxxx user is sick of their lame jokes.

What can I say, I got greedy!

[gonknet]

It was late and I should have wrote top 10.... Who cares about the #8 computer, when you can have a top 5 computer!

Hm.. with this much compute power..

[elemur]

If you add in VirtualPC... presumably the clustered version.. you should start to get to the level of compute power that was recommended by Microsoft for Longhorn... though it still wouldn't be the high end. Expect some sluggishness..

Re:Article Comparison...

[upside]

My question would have to be: Teraflops - is it purely an aggregation of processor power, or does it take into account things like interconnects? That is, interconnects inside a single node, and among nodes.

I ask the question having freshly read this article which explains why Opterons are such a huge leap forward compared to Intel's designs. In a nutshell, the bottleneck on a server isn't generally the CPU but interconnects between components. An opteron has dedicated interconnects to other CPUs, its own memory bank, and other components while Xeons, for example, do all this through a slow FSB.

Simulations

[Ian_Bailey]

The vast majority of clusters are for simulating very complex systems that require lots and lots of calculations.

You can get a few hints by looking just at their names.

The number one "Earth Simulator Centre" is fairly self-explanatory, going to their website show they create a variety of models for things such as weather, tectonic plate movement, etc.

The number 3 LANL supercomputer "is a key part of DOE's plan to simulate nuclear weapons tests in the absence of actual explosions. The more powerful computers are designed to model explosions in three dimensions, a far more complex task than the two-dimensional models used in weapons design years ago." I imagine that most US government simulations would be doing something simmilar.

Maybe.

[Bill, Shooter of Bul]

When they do real work, all of my questions and concerns will be taken care of. Until then, its a bit frustrating. A tool is only useful when it is used for its intended purpose. Maybe they built the first super computer with the idea that it would never be used, but thats just a little sad to me. Its as if Michelangelo had created a smaller version of David, but destroyed before anyone could see it because it was just a model for the larger work.

Re:Maybe.

[Ffakr]

It was my impression that the original G5 cluster was up and performing real work for several months before it was tore down for a rebuild.
It's probably safe to assume that the work in setting up the first cluster decreased the deployment of the second cluster so it wasn't just an exercise is vanity even if it didn't perform a lot of production calculations.

Re:Article Comparison...

[joib]

My question would have to be: Teraflops - is it purely an aggregation of processor power, or does it take into account things like interconnects?


Take a look at top500.org. For every supercomputer you'll see two numbers, Rpeak and Rmax. Rpeak is a purely theoretical estimate, basically number of floating point operations per clock cycle times the clock frequency times the number of cpu:s in the system. Rmax is the result from running the linpack benchmark.

The linpack benchmark is run on the entire system, so it takes into account all the interconnects, but unfortunately linpack is almost "embarassingly parallel", e.g. for a cluster the interconnect makes almost no difference, ethernet is as good as some supah-dupah $$$ interconnect.

Re:School Funding, sorta waste.

[BrianHursey]

Well the G4s where not originally intended to be used for cluster computing. They where originally desktops for the past year or so. We got replacement systems for them so it was either we could surplus them or we could use them. We do not have the funding to buy systems just for this project so we just use the left overs.

Great...

[Mysticalfruit]

We'll have the puppet master, but it'll be contained to only a couple computers in the whole world (until you can get 100tflops on your desktop)...

Rankings

[thopkins]

should still be in the top 5 when the new rankings come out in November.

Wow, ranked higher than the Virginia Tech football team this year.

Re:Rankings

[elbles]

Hey, we're improving, give us time . . . we're #22, and moving on up like the Jeffersons. Not to mention 2 60+ point to 0 victories this season, and a win against West Virginia . . . we're pretty good, even if our football team probably has no idea what UNIX is, or even that we have a supercomputer on campus, heh.

VT is in 7th place. IBM G5 system in 3rd though.

[Anonymous Coward]

If you look at the latest list, VT is already out of the top 5. They are in 7th. The new list is here:

http://www.netlib.org/benchmark/performance.pdf (Page 54)

IBM is first with BlueGene (PowerPC 440), but is also 3rd with their 3564 CPU PowerPC 970 2.2 GHz JS20 system.

1) 36010 - BlueGene/L DD2 - 16384 0.7 GHz PowerPC 440
2) 35860 - Earth Simulator - 5120 NEC processors
3) 20530 - IBM eServer BladeCenter JS20 - 3564 2.2 GHz PowerPC 970 G5
4) 19940 - QsNetII Intel Tiger4 - 4096 Itanium 2 1.4 GHz
5) 19564 - NASA Project Columbia SGI Altix 3000 - 4032 Itanium 2 1.5 GHz
6) 13880 - ASCI Q AlphaServe EV-68 - 8160 Alpha 1.25 GHz
7) 12250 - Virginia Tech Apple Xserve - 2200 2.3 GHz PowerPC 970 G5
8) 11680 - BlueGene/L DD1 - 8192 0.5 GHz PowerPC 440
9) 10310 - IBM eServer pSeries 655 - 2880 1.7 GHz POWER4+
10) 9819 - Dell PowerEdge 1750 - 2500 3.06 GHz Xeon

BTW, I wouldn't be surprised to see the 2.3 GHz Xserves announced by Apple in January for general consumption.

Re:Article Comparison...

[Jeremy Erwin]

I don't think Apple deserves any of the credit. They made the case for the xServe? and the OS .. nothing hardware. Once again its an IBM feat. The Apple zealots on here are too much. They make Linux zealots look intelligent.

Thank you. Thank you. All of my friends laughed at me when I said I didn't need a motherboard-- I could save a bundle instead and just use a breadboard.

Re:So he's saying that...

[OmniVector]

pricing a top of the line dual 250 opteron with a mobo that has similar features to a powermac (gigabit, pci-x, 8 ram slots, firewire 400 and 800.. which no opterons offer, etc) gives you a system at rough price around $2,473.00. that doesn't include a case, powersupply, hard disk, cdrom, keyboard, or mouse like the powermac does.

what planet are you pricing yoru "similar" x86 hardware on? look, i know mac doesn't have a low end $200 pc. but their high end offerings are not only competitive, but cheap.

Theoretically Speaking

[MooseByte]

"Everything works in theory, but not pratice."

In theory, anyway.

Don't let them Virginians hear ya...

[IronChefMorimoto]

If I were you guys, I wouldn't be calling their supercomputer a "Hokie supercomputer." Some of them thar Virginians might get a wee rankled thinkin' you said "Hokey supercomputer," and 12+ teraflops ain't too hokey. Who says? The end of my buckshot Blue Ridge rifle, that's who!

IronChefMorimoto

P.S. - Take my word on this as an ex-North Carolinian-- I called an Appalachian State University server farm rather "dairy" and nearly got my ass shot off. ;-)

Was Ty Pennington there with a megaphone?

[IronChefMorimoto]

From the CNET story headline:

"The fastest Mac supercomputer has gotten faster, thanks to an Xserve makeover." (http://news.com.com/Virginia+Tech+beefs+up+Mac+su percomputer/2100-1016_3-5426091.html?tag=nefd.top)

Was that neurotic TLC-to-ABC crossover Ty Pennington (http://abc.go.com/primetime/xtremehome/bios/ty_pe nnington.html) onsite to help with the installation upgrades?

Sorry -- this post was in honor of my wife, who tortures me with that damned show every Sunday night.

IronChefMorimoto

Human Intelligence

[Zobeid]

Humans are not born intelligent, but most are born with the ability to become intelligent through a learning process. So the question becomes: How much of a human brain's "software" is coded genetically (like firmware) and how much is learned? Nobody knows the answer. Not yet, anyhow.

It's not obvious that the genetic portion is so complex that we can never figure it out. It might be relatively small and simple, and there may be evolutionary pressures to keep it that way. If this were the case, it might bode well for AI research. Then all you have to do is synthesize this "firmware" enough to enable the AI to start learning.

It was recently announced that the human genome only contains about 25,000 genes. There's a limit to how much complexity you can encode into that. Intuitively, it seems this should be possible to figure out someday.

Elaborate...

[bill_mcgonigle]

His estimate was probably based on the common, and incorrect, belief that neurons are purely digital.

So.... are they partly digital? Entirely analog? Quantum in nature?

Don't tease.

Re:Elaborate...

[RKBA]

They are both partly digital and partly analog, with the ratio depending upon the type of neuron in question (eg; glial neurons are mostly analog I believe).

Renting Time?

[bill_mcgonigle]

Can somebody not at VT rent time on this or is it purely in-house?

Are there any supercomputer rental outfits out there?

I've heard IBM will truck in a box for you, but that's not really 'net savvy. There was a story about Weta leasing during downtime, but that's a side-line.

Re:Renting Time?

[Quila]

I believe HP leases time, and have done it for the rendering on a couple recent movies.

Re:Wow!

[Ohreally_factor]

What, compared to the people who post the Beowulf/Soviet Russia/SCO jokes a million times over? Hard to get a worse sense of humour than them, as even this "new" 7xxxxx user is sick of their lame jokes.

In Soviet Russia, Beowulf cluster jokes are sick of you.

Not exactly

[daveschroeder]

Yes, it was up for a while, but mostly for testing and tuning.

The one critical problem with the initial cluster was that the Power Mac G5 didn't have ECC memory, meaning that any long calculation would really have to be run twice - or at least until the result was the same - to essentially insure a soft error did not go unnoticed (and no, VT's special "error detecting" software didn't account for this).

The Xserve G5, however, does have ECC memory, making the current cluster just as capable as anything else in the top 10.

I'm not denigrating the original cluster, however: VT played by the rules, and made it to #3 in the world, #2 in the US, and #1 academic for a mere US$5.2M. They also broke the burgeoning Dell/Linux hegemony for cheap clusters, proving that you could use Apple, PowerPC, Mac OS X, and Infiniband to make clusters just as cheap, if not cheaper (note how much better the Apple clusters perform per processor than the closest Dell P4 Xeon 3.06GHz Linux cluster several spots below...additionally, check out this fantastic cost comparison of many of the top machines). Not to mention bringing a new 64-bit player to the HPC table. And one would hope that competition, even in supercomputing, is a good thing.

folding@home uses gromacs Re:"Dick factor" aside

[t35t0r]

Gromacs is used in parallel by folding@home with the stanford group. If you run the folding@home program you can see the gromacs data packets being sent to your computer. The program simulates how peptides might fold into proteins by solvating them in a cellular environment.

So the virginia tech supercomputer could also run folding@home (basically modified gromacs).

Re:12.25 Teraflops ...

[cyfer2000]

I think they are headless, tailess and mouseless servers.

Re:Article Comparison...

[outZider]

So, Apple didn't design the motherboard, or the chipset derivative, or the Apple northbridge? Sure, the bus is based off of HyperTransport.. But you can't take Apple out of the equation.

this is ground-breaking

[SethJohnson]

I'm a robotic software researcher, so this notion really affects me.

This post deserves its own slashdot article all to itself. Not only has an AI-driven robot posted on slashdot, but apparently someone has designed the robot to research software. So it would make sense that the robot would be reading slashdot. I think the editors should set up an interview with this AI drone known as SnowZero.

That's all nice and all...

[xornor]

But it only has 1150 mouse buttons...

Re:VT is in 7th place. IBM G5 system in 3rd though

[rb4havoc]

One problem with this list...
It doesn't include the new Xserve cluster that the COLSA corporation had Apple build, which can exceed 25 teraflops.
http://www.apple.com/science/profiles/colsa/

Re:2.3GHz?

[mmkkbb]

dude, have you never seen those "property of bahdeblah sports team athletic club" shirts?

Re:Article Comparison...

[sydres]

yeah only apple would implement a transit system that does'nt go all the way downtown

Re:VT spanked by NASA

[jtrott]

I don't know were you learnt your maths, but 42.7 doesn't get anywhere near 10 times 12.25, or 122.5 for the slow-at-moving-decimal-places. It doesn't even get close to 5 times the speed, or 61.25. If you do the maths, the factor is actually approx 3.49.

FEA!

[quarkscat]

Or, Finite Element Analysis!

Think about processing thousands of million
cubed matrix calculations needed to simulate
the air flow and thermal dynamic calculations
of a jet engine (or of weather patterns).
The matrix operations are readily split and
transferred piecemeal to a computer cluster.

Actually, you're wrong

[daveschroeder]

I don't know how much Dell's Tungsten cluster cost but those guys went online last year and got ranked #4 (just behind this Mac cluster) and they're #5 or something now. These bozos have spent a year fscking around with upgrades and from the theoretical #3 (as they were taken out since the cluster couldn't enter production) will have dropped to #7 or more in the next ranking....

Tungsten cost $12 million. Just for the hardware.

System X cost a total of $6 million, and it's still faster.

Not to mention that Virginia Tech was able to pull of a publicity coup and become #3 in the world, #2 in the US, and #1 academic for a paltry $5.2M. And they were "taken out" of the list voluntarily, because they dismantled the entire thing to replace it with Xserve G5s. With the renewed US focus on supercomputing, no one will likely EVER be able to hit #3 on this list for something anywhere close to $5.2M again.

Here's the current list:

http://www.netlib.org/benchmark/performance.pdf

Here's just the current top 20, as of 10/26/04:

http://das.doit.wisc.edu/misc/top500.jpg

Confusingly, you seem to have forgotten that since VT dropped on the list, since VT is still much faster than Tungsten, that means Tungsten also dropped. Tungsten is currently #16. For $12 million. VT's 2.5 Tflops faster - a respectable standalone clusters' worth faster - for half the price. Plus VT got all the huge publicity and news articles, and attracted millions of dollars in funding and grants for their new supercomputer center. Not to mention bringing a whole new OS, platform, interconnect, and processor onto the scene, which will benefit everyone (competition and choice is good, right?).

Also, here's a really great cost/performance comparison of all the top clusters.

Nice try at trolling, but next time don't be so obvious and pathetic about it, especially when Tungsten looks like it clearly is the raw end of the deal, when you have to spend over twice as much money to get a cluster that performs significantly worse, and has worse power requirements.

Re:Actually, you're wrong

[Donny Smith]

>System X cost a total of $6 million, and it's still faster.

Yes and in the meantime the Tugnsten has been in production for over a year now.

And after the initial setup VT have spent an additional year to upgrade. That cost them US$600K (more than 10% of the initial h/w cost) for a 20% or so gain in performance. If you add depreciation of the existing equipment, I'm sure it's over 20% of the total cost - in other words, the gain has been negative.

>Nice try at trolling, but next time don't be so obvious and pathetic about it

Of course I have to be obvious to state my point.