Slashdot Mirror
The Problem With the Top500 Supercomputer List
angry tapir writes "The Top500 list of supercomputers is dutifully watched by high-performance computing participants and observers, even as they vocally doubt its fidelity to excellence. Many question the use of a single metric — Linpack — to rank the performance of something as mind-bogglingly complex as a supercomputer. During a panel at the SC2010 conference this week in New Orleans, one high-performance-computing vendor executive joked about stringing together 100,000 Android smartphones to get the largest Linpack number, thereby revealing the 'stupidity' of Linpack. While grumbling about Linpack is nothing new, the discontent was pronounced this year as more systems, such as the Tianhe-1A, used GPUs to boost Linpack ratings, in effect gaming the Top500 list."
Fortunately, Sandia National Laboratories is heading an effort to develop a new set of benchmarks. In other supercomputer news, it turns out the Windows-based cluster that lost out to Linux stumbled because of a bug in Microsoft's software package. Several readers have also pointed out that IBM's Blue Gene/Q has taken the top spot in the Green500 for energy efficient supercomputing, while a team of students built the third-place system.
I wonder why they don't use EISPACK?
That is for solving Eigen systems.
I remember in the early 1980's writing a program to check my linear algebra homework using Fortran and EISPACK.
This is why I love the fact that Bender likes to drink "Old Fortran" malt liquor.
I have to admit I don't know much about benchmarking but I remember using LINPACK and EISPACK on the VAX and later the Cray YMP.
"We can't solve problems by using the same kind of thinking we used when we created them." -- Albert Einstein
Also, I have seen cases where compiler optimization is smart enough to remove the entire loop if there are no side effects to incrementing i, and it's not used outside the loop.
Most compilers should be doing this. Hell, even IE9 is supposed to do it for JavaScript now. It gets great scores on SunSpider because of it (the JIT can throw away entire tests).
Leela: How is that good news, Professor?
Professor Farnsworth: I still charge enough per seat to be feared.
The world's burning. Moped Jesus spotted on I50. Details at 11.
The advantage is that, contrary to the arguments of TFA, the test is very representative of scientific and engeneering problems.
No, it really isn't. I work in HPC at a national lab, and our bureaucrats buy these computers based on these benchmark numbers and then expect us to adapt our codes to fit these machines, rather than buying machines that are better suited to the problems we are solving. For example, one of our machines peaked at #2 on the Top500 list, and was essentially useless for real codes. Another machine of ours held the #1 spot for quite a while, and worked well for a small class of problems, but was so limited in functionality that it couldn't even run many of our codes. I've heard similar stories from people using other machines near the top of the Top500.
Real science codes often do not look anything like LINPACK, and the computers that run these benchmarks fast aren't necessarily good for true HPC.