Except they didn't. The TOP500 website isn't functioning right now but I believe the Tianhe-1A is running LINPACK at something between 40%-60% efficiency. That's atrocious. Most supercomputers can run it at 80%-90% efficiency.
It's still a very impressive achievement, especially the power consumption.
And since many supercomputer purchases fall into the political realm (in terms of congressional earmarks to pay for them, or national programs to improve "competitiveness"), you need something that your congressman or premier can handle easy.
No, no, no! This is not how it works at all. Here's how it works. Congress appropriates some amount of money for computing. Big labs across the country submit proposals to the funding agency (DARPA, NSF, etc.) to build a computer. Those proposals have to show how real science will get done. Thus simply putting a LINPACK score in the proposal will get you laughed out of the room and a lifetime ban from competing for computing dollars. The labs have to actually study how their codes will run on the proposed machine and then prove it once it's built. The process of proposing, building and running a supercomputer is complex and highly technical. Politicians aren't reviewing these things, scientists are.
Nobody competent purchases a machine based solely on TOP500. These things go through extensive testing on customer codes before they are procured. Then the machine has to pass a set of tests at the customer facility before it is accepted and money is paid.
The advantage is that, contrary to the arguments of TFA, the test is very representative of scientific and engeneering problems.
No, it isn't. It's perhaps representative one a very narrow subclass of a class of scientific problems. It does dense linear algebra. That kind of code is pretty rare these days.
And that, surprise, is exactly what LINPACK measures: and it's not just raw processing speed that factors into it but also things like interconnect speed and so on.
No, it really doesn't. The communication part of LINPACK, while not insignificant, doesn't dominate the benchmark in the way that happens with real production codes. It's a rare application these days that doesn't have some kind of communication bottleneck.
No, people have known that Linpack is a bad benchmark for a long time. That's why you don't see traditional supercomputer vendors touting their position on TOP500, even if they are #1. Customers know enough not to purchase based on Linpack numbers.
The hardware isn't the issue, it's the software. It becomes very difficult to use so many threads on most workloads. For graphics, sure, but one of the trends with these things is to use them as general accelerators.
The moment you have enough graphic card memory to store all texures/fragment programs/geometry you significantly lower the amount of traffic going over the pcie bus.
I don't think you'll ever have enough memory. We've demonstrated that repeatedly. And again, for general processing it's not feasible to preload everything into device memory.
You cannot put a top of the line gpu in a cpu, you sacrifice too much of both and wind up making both a crappy cpu and gpu.
You know this with absolute certainty? You are absolutely sure we'll never have the technology to do it? I'll take that bet against you in a heartbeat.
You will NOT be seeing a gtx480 equivalent in your cpu die any time soon.
Don't be so sure. I'm just speculating, but cramming more threads into a GPU will only go so far. Right now one of the biggest bottlenecks is the PCIe bus. Moving the GPU on die gets rid of that entirely. I could certainly see an on-die gtx480-like coprocessor happening relatively soon. I'll bet the NVidea people sweat about this every day. It will be interesting to see how they maintain market share as more stuff gets integrated on-die.
You can't have a memory connection that is both very low latency and very high bandwidth.
Why not? Bandwidth is simply data / latency + some pipelining effects. Drop the latency and the bandwidth goes up, generally. To put it another way, bandwidth is simply data / time between packets. Latency determines the depth of the pipeline.
Both on one die may be convenient, for some corner-case uses it may even provide better performance
All things being equal, an on-die GPU/coprocessor has huge advantages in bandwidth between the host and device. That covers a lot more than corner cases. CPU designers are running out of ways to use more transistors. Bigger caches only buy you so much until you hit diminishing returns. Specialized processor cores are the obvious next thing to do.
The massive parallelism comes from SIMD programming, so it is a vector machine. NVidea made up the "SIMT" term so people would think it's something new. It isn't. In fact the programming model would be much less confusing if CUDA didn't try to present the vector processor as a collection of scalar threads. Then all the silliness around "divergence" would go away.
If it looks like a vector machine and is programmed like a vector machine, it is a vector machine.
such as when people moved from custom interconnect to GbE and infiniband.
We haven't. The top two machines use custom interconnect. Big machines that run non-embarrassingly-parallel codes (i.e. not the one measuring for TOP500) still use custom interconnect.
But you're right, we are seeing a paradigm shift. It's called hybrid computing and has been on the radar for several years. We used to do this stuff back when Burroughs and others were still in business. A GPU is "just" a vector processor, after all. What's old is new again.
The problem, as Intel found out with Larrabee, is that a cache that works well for CPU tasks does not work well for GPU tasks, and vise-versa. For a GPU the bandwidth is everything, while for a CPU its the latency that matters most.
That's often application-dependent, but generally you are correct. One obvious solution is to implement specialized caches used by different instructions. Just as we have instruction caches and data caches, it's not hard to imagine scalar caches and vector/streaming caches. We certainly have enough transistors.
This is not a new idea. People have been talking about paired temporal/non-temporal caches for years.
Nowhere did I say raising taxes was the sole solution. We do need to raise revenue and we need to raise it from those who've been living off the hard work of the poor and (rapidly shrinking) middle class. We also need to rip apart a lot of existing systems and rebuild them from the ground up. Not necessarily because they're inefficient but because they're aimed at the wrong goals. Pumping money into a system that's oriented the wrong way isn't going to help anything.
This is not a Democratic or Republican thing. It's common sense. I don't share your cynicism about the supposed lack of intelligence of the electorate. It's simply not very easy to know about the political intricacies when you're working three jobs and have a family to take care of.
That said, it is absolutely false that both parties are equally to blame. The Republicans are much more in bed with big business and generally advocate policies that humanize corporations (see Citizens United) and dehumanize human beings (see opposition to health care access). There are bad players on both sides but one side has many more than the other and the official planks of one side actively encourage that behavior.
Social Security would be solvent if we raised the limit on taxable income. Why is there even a limit?
Our whole health care system would work much better if we went to a single-payer system or to any of the other systems where health is treated as a public good. This has been demonstrated across the world. Simply look at the health outcome numbers and the funding input numbers.
It's really, really not rocket science to fix most of our major problems. The problem, as always, is political will.
here is absolutely no, zero, nada way for the federal government (and the bulk of the states) to meet all these obligations it has promised to people
Citation please. The only reason we can't meet these obligations right now is that the Republicans and their rich cronies don't want to pay their fair share into the system.
Primarily it's a function of the number of cores. I've told people in my own workplace - you want a machine on the Top 500 then write me a cheque - making a supercomputer isn't the feat of skill or engineering that it was in the days of Cray.
Really. Just stop. You're looking foolish. Supercomputing is nowhere near a solved problem. The industry changes so quickly that solutions arrived at for one generation often don't apply to the next. You're under the false impression that supercomputing is about flops. It's not. It's about bandwidth and software.
cientific advancement in the realm of large projects seems to be a positive byproduct
No. Please just stop speculating. Supercomputing is not driven by the TOP500. It's driven by agencies that do real scientific research. We're not pumping out flops and GB/s to make the TOP500. We're doing it because our scientists need it.
Whether serious research needs faster computers, is up for debate. Personally I think it mostly needs researchers that know how to program. But even if faster computers are needed, then the needed ones are not of the supercomputer type.
You have no idea what you're talking about. There will always be a need for lots of computational power connected by networks with lots of bandwidth. A supercomputer isn't "super" because of the flops. It's "super" because of the network and the software.
Sure, encapsulation is usually the better choice, but many things map naturally to inheritance and multiple inheritance. In my case it was a compiler IR, which is naturally tree-like.
Well, "add" might be addition or concatenation. Any function might do something unexpected. "foo" might have all sorts of strange side-effects. The name hardly matters.
Completely agree about default arguments. I've been bitten many times as well. Boost.Parameter is closer to what it should be but without language support, we're left to use these kinds of libraries that involve a lot of boilerplate.
This can be used effectively when you have mixins with a need to "cross cast." Never say never. It's certainly a niche thing but it's there for a reason.
You said it yourself: lambda-like libraries. Phoenix and especially Proto are great examples. && and || are problematic because of the short-circuiting expectation but still, never say never.
Propagating exceptions out of a destructors (can result in a crash)
Yep. It's one of those "shoot yourself in the foot" moments. Of course, there's no way for a compiler to verify nothing in a destructor throws, so we'd have to get rid of exceptions, not what I'd want to do.
Exception specifications
Agreed. Fixed in C++0x. Learning from our mistakes is good.:)
Another one that comes to mind is overloading a function to take either an int or a pointer (because if you pass 0 as a null pointer
Also fixed in C++0x. Certainly C++ is not perfect. I'm glad the committee is still open to learning and making major language changes. The world is very different today than when C++ was first standardized.
Slashdot Mirror
Except they didn't. The TOP500 website isn't functioning right now but I believe the Tianhe-1A is running LINPACK at something between 40%-60% efficiency. That's atrocious. Most supercomputers can run it at 80%-90% efficiency.
It's still a very impressive achievement, especially the power consumption.
And since many supercomputer purchases fall into the political realm (in terms of congressional earmarks to pay for them, or national programs to improve "competitiveness"), you need something that your congressman or premier can handle easy.
No, no, no! This is not how it works at all. Here's how it works. Congress appropriates some amount of money for computing. Big labs across the country submit proposals to the funding agency (DARPA, NSF, etc.) to build a computer. Those proposals have to show how real science will get done. Thus simply putting a LINPACK score in the proposal will get you laughed out of the room and a lifetime ban from competing for computing dollars. The labs have to actually study how their codes will run on the proposed machine and then prove it once it's built. The process of proposing, building and running a supercomputer is complex and highly technical. Politicians aren't reviewing these things, scientists are.
Nobody competent purchases a machine based solely on TOP500. These things go through extensive testing on customer codes before they are procured. Then the machine has to pass a set of tests at the customer facility before it is accepted and money is paid.
The advantage is that, contrary to the arguments of TFA, the test is very representative of scientific and engeneering problems.
No, it isn't. It's perhaps representative one a very narrow subclass of a class of scientific problems. It does dense linear algebra. That kind of code is pretty rare these days.
And that, surprise, is exactly what LINPACK measures: and it's not just raw processing speed that factors into it but also things like interconnect speed and so on.
No, it really doesn't. The communication part of LINPACK, while not insignificant, doesn't dominate the benchmark in the way that happens with real production codes. It's a rare application these days that doesn't have some kind of communication bottleneck.
No, people have known that Linpack is a bad benchmark for a long time. That's why you don't see traditional supercomputer vendors touting their position on TOP500, even if they are #1. Customers know enough not to purchase based on Linpack numbers.
Now if you'll excuse me, there's someone on a lawn that needs yelling at!
And it's you!
Any technology advances will be used to make better cpus/gpus at the high end.
A gtx480 is not going to be high end for longer than about a year.
gpu's are far more parallel than cpus.
The hardware isn't the issue, it's the software. It becomes very difficult to use so many threads on most workloads. For graphics, sure, but one of the trends with these things is to use them as general accelerators.
The moment you have enough graphic card memory to store all texures/fragment programs/geometry you significantly lower the amount of traffic going over the pcie bus.
I don't think you'll ever have enough memory. We've demonstrated that repeatedly. And again, for general processing it's not feasible to preload everything into device memory.
You cannot put a top of the line gpu in a cpu, you sacrifice too much of both and wind up making both a crappy cpu and gpu.
You know this with absolute certainty? You are absolutely sure we'll never have the technology to do it? I'll take that bet against you in a heartbeat.
Don't be so sure. I'm just speculating, but cramming more threads into a GPU will only go so far. Right now one of the biggest bottlenecks is the PCIe bus. Moving the GPU on die gets rid of that entirely. I could certainly see an on-die gtx480-like coprocessor happening relatively soon. I'll bet the NVidea people sweat about this every day. It will be interesting to see how they maintain market share as more stuff gets integrated on-die.
Why not? Bandwidth is simply data / latency + some pipelining effects. Drop the latency and the bandwidth goes up, generally. To put it another way, bandwidth is simply data / time between packets. Latency determines the depth of the pipeline.
All things being equal, an on-die GPU/coprocessor has huge advantages in bandwidth between the host and device. That covers a lot more than corner cases. CPU designers are running out of ways to use more transistors. Bigger caches only buy you so much until you hit diminishing returns. Specialized processor cores are the obvious next thing to do.
The massive parallelism comes from SIMD programming, so it is a vector machine. NVidea made up the "SIMT" term so people would think it's something new. It isn't. In fact the programming model would be much less confusing if CUDA didn't try to present the vector processor as a collection of scalar threads. Then all the silliness around "divergence" would go away.
If it looks like a vector machine and is programmed like a vector machine, it is a vector machine.
We haven't. The top two machines use custom interconnect. Big machines that run non-embarrassingly-parallel codes (i.e. not the one measuring for TOP500) still use custom interconnect.
But you're right, we are seeing a paradigm shift. It's called hybrid computing and has been on the radar for several years. We used to do this stuff back when Burroughs and others were still in business. A GPU is "just" a vector processor, after all. What's old is new again.
So Chevy broke the clean EV design to allow you to drive at speeds that are usually illegal and always inefficient.
No, they broke the "clean" EV design to allow you to drive at highways speeds more efficiently when the charge is depleted.
I will not buy one for this reason.
Either you're being disingenuous or you're cutting off your nose to spite your face.
The problem, as Intel found out with Larrabee, is that a cache that works well for CPU tasks does not work well for GPU tasks, and vise-versa. For a GPU the bandwidth is everything, while for a CPU its the latency that matters most.
That's often application-dependent, but generally you are correct. One obvious solution is to implement specialized caches used by different instructions. Just as we have instruction caches and data caches, it's not hard to imagine scalar caches and vector/streaming caches. We certainly have enough transistors.
This is not a new idea. People have been talking about paired temporal/non-temporal caches for years.
Replying to a massive troll, but what the heck.
Nowhere did I say raising taxes was the sole solution. We do need to raise revenue and we need to raise it from those who've been living off the hard work of the poor and (rapidly shrinking) middle class. We also need to rip apart a lot of existing systems and rebuild them from the ground up. Not necessarily because they're inefficient but because they're aimed at the wrong goals. Pumping money into a system that's oriented the wrong way isn't going to help anything.
This is not a Democratic or Republican thing. It's common sense. I don't share your cynicism about the supposed lack of intelligence of the electorate. It's simply not very easy to know about the political intricacies when you're working three jobs and have a family to take care of.
That said, it is absolutely false that both parties are equally to blame. The Republicans are much more in bed with big business and generally advocate policies that humanize corporations (see Citizens United) and dehumanize human beings (see opposition to health care access). There are bad players on both sides but one side has many more than the other and the official planks of one side actively encourage that behavior.
Social Security would be solvent if we raised the limit on taxable income. Why is there even a limit?
Our whole health care system would work much better if we went to a single-payer system or to any of the other systems where health is treated as a public good. This has been demonstrated across the world. Simply look at the health outcome numbers and the funding input numbers.
It's really, really not rocket science to fix most of our major problems. The problem, as always, is political will.
Citation please. The only reason we can't meet these obligations right now is that the Republicans and their rich cronies don't want to pay their fair share into the system.
Really. Just stop. You're looking foolish. Supercomputing is nowhere near a solved problem. The industry changes so quickly that solutions arrived at for one generation often don't apply to the next. You're under the false impression that supercomputing is about flops. It's not. It's about bandwidth and software.
No. Please just stop speculating. Supercomputing is not driven by the TOP500. It's driven by agencies that do real scientific research. We're not pumping out flops and GB/s to make the TOP500. We're doing it because our scientists need it.
You have no idea what you're talking about. There will always be a need for lots of computational power connected by networks with lots of bandwidth. A supercomputer isn't "super" because of the flops. It's "super" because of the network and the software.
Sure, encapsulation is usually the better choice, but many things map naturally to inheritance and multiple inheritance. In my case it was a compiler IR, which is naturally tree-like.
Well, "add" might be addition or concatenation. Any function might do something unexpected. "foo" might have all sorts of strange side-effects. The name hardly matters.
Completely agree about default arguments. I've been bitten many times as well. Boost.Parameter is closer to what it should be but without language support, we're left to use these kinds of libraries that involve a lot of boilerplate.
This can be used effectively when you have mixins with a need to "cross cast." Never say never. It's certainly a niche thing but it's there for a reason.
Thanks, this could be an interesting discussion.
You said it yourself: lambda-like libraries. Phoenix and especially Proto are great examples. && and || are problematic because of the short-circuiting expectation but still, never say never.
Yep. It's one of those "shoot yourself in the foot" moments. Of course, there's no way for a compiler to verify nothing in a destructor throws, so we'd have to get rid of exceptions, not what I'd want to do.
Agreed. Fixed in C++0x. Learning from our mistakes is good. :)
Also fixed in C++0x. Certainly C++ is not perfect. I'm glad the committee is still open to learning and making major language changes. The world is very different today than when C++ was first standardized.