Slashdot Mirror
Supercomputers To Move To Specialization?
lucasw writes "The Japan Earth Simulator outperformed a computer at Los Alamos (previously the world's fastest) by a factor of three while using fewer, more specialized processors and advanced interconnect technology. This spawned multiple government reports that many suspected would ask for more funding in the U.S. for custom supercomputer architectures and less emphasis on clustering commodity hardware. One report released yesterday suggests a balanced approach."
Ordinary off the shelf microprocessors don't have the bandwidth to memory or bandwidth to other processors to simulate complex problems. NEC's machine is a Vector architecture (SX-6), similar to the kind you see from the Cray X1. Vector architectures are a SIMD-style processor.
I assume that hard-coding trig functions into the tertiary processors would be advantagious for this. I know it violates the spirit of RISC in general-pupose computing, but for such a large scale system with so many processors it coould be advantagious.
Do HP's Saturn or other such special-purpose processors have hard-coded higher-level functions?
You can't judge a book by the way it wears its hair.
The interconnects are (usually) not commodity parts -- just the servers.
As an example, the first IBM SP "supercomputers" were essentially just common Power workstations bolted into racks, but connected with a custom made SP switch.
Nevertheless, EarthSimulator has shown what can be done by designing the entire server from the ground-up with the application in mind.
We'll have to see how ASCI Purple performs...
Because some problems don't work on clusters--things like large-scale molecular dynamcis simulations with long-range spatial interactions.
Problems that require the nodes to share massive amounts of data between nodes (gigabytes per second and up--these problems often have N^2 behaviors) don't do so well on a cluster since they tend to saturate the network. A shared-memory system, like a supercomputer, on the other hand, can provide much better memory access times (top of the line Cray's have a peak memory transfer rate of 204 GB per sec per node [yes, 204 gigabytes per second]) and since there's only one copy of the memory, there can often be a lower peak bandwidth requirement.
In short, it all depends on the problem you need to solve. Some problems work very well on clusters, others do not.
-JS
Vanity of vanities, all is vanity...
Perhaps you haven't looked at what the Earth Simulator is actually being used for? It is doing plenty of environmental research. The humourous thing is that the US wishes to develop faster systems with more capacity than the earth simulator only to simulate weapons. (see hpc.mil) The Japanese have the right idea, it will take big science and big computers to solve our environmental problems.
computers like the earth simulator go vastly under utilized for the most part
From first-hand experience, such computers are running jobs almost 24x7. Due to job scheduling details there are times when some of the machine is idle, but this is still a small percentage. These machines are used for a vast array of applications, not just the advertized ones.
Now the utilization as a percentage of peak theoretical is another matter. For some algorithms, 20% of peak performance (IIRC) is considered good (ie. a particular code might only get 2 TFlops on a machine rated for 10).
There seems to be an impression in some comments that this machine has some sort of special design that's only applicable to climate modeling problems. In fact, this is a vector-based supercomputer, applicable to any problem where you need to perform vector operations (i.e., operating on large arrays of numbers in parallel).
Certain numerical operations can be performed blindingly fast on these types of machines. Each arithmetic processor on this machine has 72 vector registers, each of which can hold 256 elements. Then you can perform operations on all 256 elements of 1 or more registers simultaneously! If the algorithm can keep the vector units fed, they will scream.
Since keeping data flowing to the processors is critical to speed, the high-speed interconnects (~12GB/s) are a must for any problem that is not completely localized. It's all about matching the problem to the hardware. There may well be problems for which a commodity cluster just can't get the job done like this can. Remember that each node of a cluster consumes power, produces heat, and takes up space. The raw cost of hardware is not the only consideration.
Earth Simulator is impressive in its own reguard, but in no way is the majority of clustering apps going toward these 'specialized' systems. Governments, research labs, etc. want powerful computers that are dirt cheap. Los Alamos's ASCI Q (Installment 1, the Alpha servers) cost over $100,000,000 to build, while their Pink cluster cost about $6,000,000 in hardware. On paper, Pink and ASCI Q are both around 10 teraflops. ASCI Q runs Quadrics on 64-bit 66MHz PCI, Pink is getting an ugprade to Myrinet Lanai 10 on PCI-X (From Lanai 9 on 64/66PCI). Not only that, but Pink runs the open-source, 100% GPL'd Clustermatic software and can be booted in a matter of seconds rather than hours like ASCI Q.
The fact is, systems like ASCI Q and the Earth Simulator just aren't practical. They may look great on paper, but there's not much that they can do that can't be done on x86. Given the choice between paying over a hundred million for a proprietary cluster that might not even be all that reliable (*cough*Q*cough*) and requires expensive software and maintenance contracts, we see companies like Linux Networx offering high-power clusters on common hardware and free software that are a fraction as expensive.
As far as reliability goes, don't get suckered into thinking that proprietary and expensive mean quality. Q's failure rate is almost as high as my old Windows '98 machine hahaha. With the exception of a few missing chillers, Pink seems relatively healthy with only a few minor failures.
If CRAY and NEC want to get into a pissing contest in specs, that's fine. If they offer something that Intel can't, more power to them. Otherwise, the five organizations in the world that own their systems can be proud that they have the most powerful computer on paper for a year or two before someone builds a cheaper x86 cluster that matches or out-performs them.
I recently had the opportunity to speak with the designers of ASCI Purple and Lightpath ... and there is definitely a reason that they cant use stock parts.
... currently all of the supercomputers are designed with two dimensional floorplans ... with the goals of minimizing distances between each various aspects of the computer throughout the room.
... but I can't recall) Even with this low cost design each machine will be a custom made dual processors. The communications protocols will actually be on the processors. To further reduce distances and communications issues, each rack will hold 2 clusters off the midplane. The curious part about Lightpath, is that it is not connected with switches ... each computer is connected in 5 directions, 1 vertically and 4 horizontal. The machines on the end loop around back to the other end. Because of this manner of networking the machine can reboot in minutes, instead of the 12 hours that it takes most super computers, b/c there is no heirarchy and precedence
... the communications needs to be on the chips.
... however we might be seeing an end of the ASCI line of supercomputers, if the LightPath works out ... there will be an order of magnitude in difference in cost for on par performance.
Currently the interconnects are the biggest set back
Lightpath which is designed to be a "low" cost super computer, is based upon a bio-med computer out of NY (probably Cornell
Common workstation modules can no longer be just bolted into specialized switched
Furthermore after ASCI Purple and Lightpath, they are planning to build three dimensionally, although there a quite a few construction and maintence issues to be resolved.
Performancewise, both machines are expected to perform on the order of 100's of terraflops
Sig Nazi- "No Sig for you, come back 1 year."