Oh, get off your high horse! There are plenty of very, very good elected officials. On their area of expertise (generally determined by what committees they are on), they understand a lot more than you and me. They get regular briefings. They read reports. What do you think they do all day, play golf?
We have a representative demoncracy for a reason. There's no way we call all be experts on everything. So we elect people whose job it is to study the issues and make decisions. We may not always agree with them. Sometimes we do know more than they do. But these people aren't idots. Thinking so leads one to a dangerous disregard for good governance.
Bull, Tax cuts do nothing to stimulate the economy. The rich don't need 'em and the poor need a lot more than $20 extra per month. At best that extra money goes into paying down massive personal debt.
The Republican mantra of tax cuts all the time was completely repudiated by the people. The fact that they don't recognize this is why thery're going to be in the wilderness for a very long time. Trickle down failed and Reagan was the worst U.S. president ever.
The Republican party needs a reboot. We need a healthy Republican party to be the best country we can be. I welcome a strong Republican party. This one ain't it.
I agree with most of what you've said except for the bit about those who don't pay income tax. They don't pay income tax. They do in fact pay p[ayroll taxes. So if anyone tells you people are getting free money without paying any taxes, they are straight-up lying to you.
Oh, and I completely disagree about tax cuts. Tax cuts do zilch to stimulate the economy.
Sorry, but this just doesn't make any sense. There are many codes that cannot be optimized "to the metal." Branch prediction and out-of-order execution matter. There are some codes you can schedule perfectly. Most you can't.
Now, for special-purpose stuff like graphics, it's probably easier to statically optimize, but dynamic scheduling and other hardware bits are still relevant.
And bandwith is exactly why parallel programming is hard. It's not threading or the Cell ASM that's hard. It's using the local memory effectively to maintain high bandwidth that's hard.
So they are taking a general-purpose CPU and using is as GPU?
No. As others have pointed out, there are GPU functions on the chip. More interestingly, there is a boatload more vector stuff that's not just a bolt-on. This is beyond AVX and includes gather/scatter, mask registers and 16-element vectors.
See the Larrabee paper for the details. It's an interesting architecture, for sure.
You're not being fair to GNU autoconf here. The problem isn't the tool
No, the problem is the tool. Here are several reasons autoconf is to be avoided:
autoconf doesn't allow any sort of dependence specification among checks. This leads to...
autoconf generates files. This is generally bad. You run into problems like, "should we check generated files into the repository?" all the time. People edit the generated files by mistake and lose their work. It's annoying.
autoconf runs serially. It takes forever. I've seen projects where configure takes longer than the actual build. Most of the checks could be run in parallel. Except you can't with autotools.
Various versions of autotools are incompatible. I've seen projects that insist on a particular version of autotools for ever and ever to avoid these problems.
autoconf relies on an archaic subset of Bourne shell so as to be compatible with broken systems like Solaris. If you're going to require a special tool to compile packages anyway, why not provide a portable environment to program it in?
CMake might be interesting but it shares the generated file problem with autoconf. The thing is, we have a very nice way to execute commands and specify dependencies among them. It's called GNU make. Granted, the syntax could be cleaned up a bit but it's a very powerful tool that too many people ignore. I've written configure systems in GNU make and it's kind of nice to have the configure and build process specified in the same system with all of the proper dependencies between them.
If you really want to change things and make technology more accessible, go do some work up at your state capitol. We need technologically-savvy people in the political realm to advocate for systemic change so we don't have pockets of people left behind the technology wave.
Find a local organization already working on social justice issues, whether racial or economic. Organize a campaign to address some need in your area. Perhaps it's affordable access to broadband. Or maybe it's sufficient funding for urban schools to upgrade technology and create computing curricula.
Remember that the real issue is not technology. It's justice. All of the technology in the world will do no good unless people have access to transportation, education and health. Technology is important. Just remind yourself that getting people engaged in and comfortable with technology is going to take a lot more than some tutoring sessions.
Create a public life for yourself. Get involved in politics. You'll find that the relationships you create will allow you to move beyond your initial campaign and into making real change in all sorts of areas.
And recognize that in the work you will be changed as well.
I'll also add that the "paperwork and hindrances" are exactly the mechanisms we put in place to keep the public sector accountable. You see, when there is public ownership there can be public accountability. There can be no public accountability in the private sector. Privatization of public goods is a bad idea all around.
No. That is the kind of elitist BS you hear from people who like to make themselves feel better. Struggling is fine. Unnecessary struggling is wasteful. The book just doesn't fit well with the way most people learn. Most people tend to learn high-level concepts first and only later dive into the nitty gritty details.
One of the biggest beefs I have with EE programs is that they get intoi all the heavy math first without actually explaining the concepts behind it. Why we care about linear systems and all that. Who the hell wants to know about CMOS before understanding how to actually make a computer do something useful?
Simply awful. The book was written with the hairbrained notion that students should learn about transistors and CMOS first and only at the end of the semester should they actually program in a reasonable language.
I had the misfortune of working with students as a teaching assistant in that course and, predictably, students struggled. There's a reason it's a good idea to learn concepts first and then approach the implementation details later. The concepts are much more important.
The theory was that students should learn what's going on under the hood when they program. A better approach would be to require all CSE students to take computer architecture, compilers and operating systems courses.
Honestly, this stuff has been known in the HPC world for decades. What's interesting is that these troublesome bits are going to hit system-level and lower-level language programmers on everyday tasks. It's not clear to me how this stuff will affect higher-level programming, interpreted code, etc. It will almost certainly be a factor but I'm not sure there's much the programmer can do about it.
Some of the fun things we have to look forward to at the commodity level:
Unsynchronized core interrupts
Cache bank conflicts among multiple threads
Various OS interactions / time to service system calls being different on different cores
Memory controller fairness issues
These (and others) all fall into the general category of "induced load imbalance." They are things the programmer doesn't directly think about; things that happen as a result of system services, CPU architecture and stuff generally out of the control of the application programmer. This is all in addition to the stuff the programmer does have control over such as data layout and the amount of work given to each thread.
Induced load imbalance is the primary reason that scaling to manycore is difficult. It requires a lot of OS work to reduce "OS jitter" to a level that is acceptable when running thousands of threads.
Here's an article on some of the scaling work HPC vendors have done with Linux.
I agree that advances in commodity memory architecture are needed to support large vectors. However, a vector register file is quite a bit different than a cache, just as a scalar register file is very different than a cache. They're architected to serve different needs.
Vector registers can help the memory system over current cache-based designs. We wouldn't have SSE registers if they didn't. Software can control the stream presented to the memory system using registers, for example. If there's enough data reuse, vector registers help tremendously.
Vector registers won't help streaming apps, but that's not what they're designed to do. A stream buffer or similar mechanism is better for that.
We are going to see longer vectors in commodity processors. Sandy Bridge will have AVX. Haswell will have something more. What's not clear is how Intel plans to architect the memory system. That will be interesting to see.
Clusters are not supercomputers. Supercomputing is not only or even primarily about processor technology. The network and memory architecture have a larger impact. Supercomputer vendors are doing plenty of architecture innovation. They're doing it in the memory, network and I/O layers.
Supercomputers do not exist mainly to run single large apps. They are batch machines running multiple jobs simultaneously.
Supercomputer customers are not willing to pay a premium for custom processors because CPU efficiency is not the most important thing. Memory and network performance is. That's why the article focuses on the memory bandwidth problems of many-core CPUs.
What we're seeing with multi- and man-core processing is the continued commoditization of supercomputer technology. It started with the micros' adaption of strong scalar performance. The Cray-1 was not outstanding because it had vector processing. It was outstanding because it was the fatest scalar processor in the world at the time. Then the micros adopted vector processing (albeit with very short vectors, but that will change with Sandy Bridge and Haswell). Then they adopted multithreading. Now they're embracing multiprocessing.
It's the nature of the business that technology trickles down from the high end to the consumer. If you look at the timeline of processor development, commodity CPUs have gained supercomputer features about 20-30 years after their first appearance in high-end computing. What hasn't happened is commodity adoption of high-end memory, network and I/O systems because the home user does not need them.
Cray did not stream vectors from memory. One of the advances of the Cray-1 was the use of vector registers as opposed to, for example, the Burroughs machines which streamed vectors directly to/from memory.
We know how to build memory systems that can handle large vectors. Both the Cray X1 and Earth Simulator demonstrate that. The problem is that those memory systems are currently too expensive. We are going to see more and more vector processing in commodity processors.
I think hyper-individualism is one of the primary traits of the early US citizens that made this into the powerhouse country it is today.
Really? How did we do fantastic things like build the transcontinental railroad, then?
Individualism doesn't preclude helping out your neighbor either, but it is your choice whether you do and to what extent. I'd much rather give and help on my own than be forced to do so by the government.
You're confusing ideas here. Individualism is fine. It's a good thing. Hyper-individualism is bad. It does in fact preclude helping your neighbor and contributing to the common good. Not everyone has bootstraps to pull themselves up with. We used to recognize that as a country. We don't now. I am hopeful that will change with the new administration.
As for being "forced by the government," I suppose you advocate abolishing all taxes. Well, I've got news for you: the churches can't provide all the social services, infrastructure, education, labor protections, etc. that can only be provided through government, which is actually us. Government is the tool we use to collectively make decisions about what kind of community we want to live in.
There is a lot of good culture in America. We just tend to underappreciate the work of ordinary people and instead look to grand leaders to create culture, which by its nature is a grassroots phenomenon.
No. Everything in American culture that is not completely worthless was created by social rejects, people who were thrown out of American society and were left with nothing but poverty, scorn or both, and then-current American culture painted them as targets and enemies of "good American people". They had no choice but to create their own culture, untainted by hostility and shallowness of the mainstream.
That's pretty much the definition of "grassroots."
Society accepted pieces of those rejects' culture when it became commercially profitable and politically convenient -- usually at the expense of most of the original meaning. Majority of American population passively consumed that filtered, diluted and packaged culture -- be it music, art, literature, cinema, or anything else.
Oh please. Don't get all elitist on me. There's plenty of great stuff that was derided as "packaged culture" in years past. It's because we refuse to appreciate our American culture that too many people seem to think we don't have any. Culture is what a society is, for good or bad. By definition each society has one.
You need to get out way more if you think that's the epitome of anything except easy cleanup.
WHOOSH!
Also fuck you and your 'community responsibility' 'common good' BS. Code for 'do what your told' and 'pay your taxes'. I do as little of ether as illegally possible.
Uh huh. Enjoyed those Interstates lately? How about your house? Benefit from the railroads much?
Every culture has its high and low points. Parts of our American culture are quite sick. For example, the hyper-individualism that is destroying any sense of community responsibility or the common good must be reversed if we're going to survive as a nation.
But there are also truly wonderful things that have come out of the unique mixtures of peoples here: jazz, country, rock&roll, for example, each of which influenced the others and each of which is far too pigeonholed today by people who don't care to become educated about our own cultural history.
There are all sorts of American culinary delights: Cajun feasts, Southern barbeque, Western concoctions and the epitome of gastric delight, the Minnesota Hot Dish.
We've got incredible diversity in language and idioms, from Texas hyperbole to Midwestern understatement.
There is a lot of good culture in America. We just tend to underappreciate the work of ordinary people and instead look to grand leaders to create culture, which by its nature is a grassroots phenomenon.
Wrong.
There is a world of difference between a supercomputer and a cluster. The network is not simply "faster" or "higher bandwidth." These things are custom-designed to fit the type of processing a supercomputer needs to do.
These are not embarrassingly parallel codes. In fact, that's why the Top 500 is bogus. It ranks computers based on their ability to run an embarrassingly parallel code. We really ought to be measuring with something like HPCC.
Actually, the analogy doesn't quite fit. Vectorization is quite a bit more comprehensible than the contortions compilers have to go through to get performance out of IA-64.
The user directives are easier, too. It's much easier for a user to say, "this loop has no cross-iteration dependencies" than for the user to mark which branches should be if-converted, which loads should be speculatively hoisted, etc. Thus for IA-64 the compiler ends up having to try to figure all this stuff out. In other words, it's much easier for the user to help the compiler understand the general parallelism available in the code than it is for the user to try to convey minute detail.
IA-64 is in many ways a house of cards. The compiler has to do a bunch of things (predication, static speculation, VLIW packing, scheduling, etc., etc., etc.) and do them all well to get performance. If it fails at one of them the code tanks. For something like Larrabee where there are multipel paradigms available (serial, vector, and multicore), you don't have to use all of them perfectly. You just have to use them well enough.
While it's easier to express parallelism, what's missing in Tim's responses is the acknowledgment that the user will have to do this.
You can read the Larrabee paper to get a good idea of the kinds of things Tim is talking about (software rendering and why it may just work).
I think the point of the article is that computing paradigms are merging. You won't have a CPU and a GPU. You'll have one thing that looks like both. In other words, you'll have a multicore, parallel, vector machine.
And that, absolutely, positively, will happen. Larrabee, or something like it, is the future. If you hold AMD stock, sell now, because Fusion doesn't sound anything like Larrabee and is going to seem positively draconian by the time it comes out.
In some ways, these new processors will look like a Cray YMP on your desktop. It's a rough analogy but suitable for illustration. Of course there are all sorts of differences in the way the memory systems will work and that's a huge part of the performance equation.
It seems to me that Tim puts a bit too much faith in compilers. He talks about language extensions but only in CUDA-like terms of "where things will run." A compiler needs a lot of information to be able to vectorize. The user often has to provide that information in a language like C because of its loose typing, aliasing and side-effect rules.
My prediction is that some APIs will go away, but many of the low-level ones will stay because it's often faster to call into a hand-coded library than rely on the compiler to have enough information to automatically optimize the code. Eventually compilers will start pattern-matching to these APIs. Higher-level APIs will be developed to save developer time, not CPU time. They will exist almost purely for code reuse purposes.
I disagree with Tim that hardware vendors will differentiate on performance. At least, in the way he's thinking. It won't be hardware gadgets, vector length or number of pipes that matter. It's going to be the compiler, programming environment and libraries. To the extent that the hardware supports those in its ISA, hardware will matter. But the bulk of the muscles of the chip won't matter so much as their placement and utility (by the compiler). The inflection point is leading to a world where software is king.
Right on. This is the real problem with Perl. Everyone rolls their own and we end up with multiple reimplementations of the same concept done in sometimes subtly different ways.
CPAN is one of my worst nightmares. There's no peer review so one ends up looking at several different libraries that do the same thing, only they all do it differently and have varying levels of feature complexity. I just want to get work done, not evaluate libraries.
Slashdot Mirror
Oh, get off your high horse! There are plenty of very, very good elected officials. On their area of expertise (generally determined by what committees they are on), they understand a lot more than you and me. They get regular briefings. They read reports. What do you think they do all day, play golf?
We have a representative demoncracy for a reason. There's no way we call all be experts on everything. So we elect people whose job it is to study the issues and make decisions. We may not always agree with them. Sometimes we do know more than they do. But these people aren't idots. Thinking so leads one to a dangerous disregard for good governance.
Bull, Tax cuts do nothing to stimulate the economy. The rich don't need 'em and the poor need a lot more than $20 extra per month. At best that extra money goes into paying down massive personal debt.
The Republican mantra of tax cuts all the time was completely repudiated by the people. The fact that they don't recognize this is why thery're going to be in the wilderness for a very long time. Trickle down failed and Reagan was the worst U.S. president ever.
The Republican party needs a reboot. We need a healthy Republican party to be the best country we can be. I welcome a strong Republican party. This one ain't it.
I agree with most of what you've said except for the bit about those who don't pay income tax. They don't pay income tax. They do in fact pay p[ayroll taxes. So if anyone tells you people are getting free money without paying any taxes, they are straight-up lying to you.
Oh, and I completely disagree about tax cuts. Tax cuts do zilch to stimulate the economy.
Sorry, but this just doesn't make any sense. There are many codes that cannot be optimized "to the metal." Branch prediction and out-of-order execution matter. There are some codes you can schedule perfectly. Most you can't.
Now, for special-purpose stuff like graphics, it's probably easier to statically optimize, but dynamic scheduling and other hardware bits are still relevant.
Uh, no, it's not right in the article. It's in the comments. And we all know what comments are worth.
C'mon, at least try to be effective in your deliberate deception.
No, they are. You simply refuse to accept it.
And bandwith is exactly why parallel programming is hard. It's not threading or the Cell ASM that's hard. It's using the local memory effectively to maintain high bandwidth that's hard.
No. As others have pointed out, there are GPU functions on the chip. More interestingly, there is a boatload more vector stuff that's not just a bolt-on. This is beyond AVX and includes gather/scatter, mask registers and 16-element vectors.
See the Larrabee paper for the details. It's an interesting architecture, for sure.
No, the problem is the tool. Here are several reasons autoconf is to be avoided:
CMake might be interesting but it shares the generated file problem with autoconf. The thing is, we have a very nice way to execute commands and specify dependencies among them. It's called GNU make. Granted, the syntax could be cleaned up a bit but it's a very powerful tool that too many people ignore. I've written configure systems in GNU make and it's kind of nice to have the configure and build process specified in the same system with all of the proper dependencies between them.
If you really want to change things and make technology more accessible, go do some work up at your state capitol. We need technologically-savvy people in the political realm to advocate for systemic change so we don't have pockets of people left behind the technology wave.
Find a local organization already working on social justice issues, whether racial or economic. Organize a campaign to address some need in your area. Perhaps it's affordable access to broadband. Or maybe it's sufficient funding for urban schools to upgrade technology and create computing curricula.
Remember that the real issue is not technology. It's justice. All of the technology in the world will do no good unless people have access to transportation, education and health. Technology is important. Just remind yourself that getting people engaged in and comfortable with technology is going to take a lot more than some tutoring sessions.
Create a public life for yourself. Get involved in politics. You'll find that the relationships you create will allow you to move beyond your initial campaign and into making real change in all sorts of areas.
And recognize that in the work you will be changed as well.
Bravo!
I'll also add that the "paperwork and hindrances" are exactly the mechanisms we put in place to keep the public sector accountable. You see, when there is public ownership there can be public accountability. There can be no public accountability in the private sector. Privatization of public goods is a bad idea all around.
No. That is the kind of elitist BS you hear from people who like to make themselves feel better. Struggling is fine. Unnecessary struggling is wasteful. The book just doesn't fit well with the way most people learn. Most people tend to learn high-level concepts first and only later dive into the nitty gritty details.
One of the biggest beefs I have with EE programs is that they get intoi all the heavy math first without actually explaining the concepts behind it. Why we care about linear systems and all that. Who the hell wants to know about CMOS before understanding how to actually make a computer do something useful?
Simply awful. The book was written with the hairbrained notion that students should learn about transistors and CMOS first and only at the end of the semester should they actually program in a reasonable language.
I had the misfortune of working with students as a teaching assistant in that course and, predictably, students struggled. There's a reason it's a good idea to learn concepts first and then approach the implementation details later. The concepts are much more important.
The theory was that students should learn what's going on under the hood when they program. A better approach would be to require all CSE students to take computer architecture, compilers and operating systems courses.
Honestly, this stuff has been known in the HPC world for decades. What's interesting is that these troublesome bits are going to hit system-level and lower-level language programmers on everyday tasks. It's not clear to me how this stuff will affect higher-level programming, interpreted code, etc. It will almost certainly be a factor but I'm not sure there's much the programmer can do about it.
Some of the fun things we have to look forward to at the commodity level:
These (and others) all fall into the general category of "induced load imbalance." They are things the programmer doesn't directly think about; things that happen as a result of system services, CPU architecture and stuff generally out of the control of the application programmer. This is all in addition to the stuff the programmer does have control over such as data layout and the amount of work given to each thread.
Induced load imbalance is the primary reason that scaling to manycore is difficult. It requires a lot of OS work to reduce "OS jitter" to a level that is acceptable when running thousands of threads.
Here's an article on some of the scaling work HPC vendors have done with Linux.
I agree that advances in commodity memory architecture are needed to support large vectors. However, a vector register file is quite a bit different than a cache, just as a scalar register file is very different than a cache. They're architected to serve different needs.
Vector registers can help the memory system over current cache-based designs. We wouldn't have SSE registers if they didn't. Software can control the stream presented to the memory system using registers, for example. If there's enough data reuse, vector registers help tremendously.
Vector registers won't help streaming apps, but that's not what they're designed to do. A stream buffer or similar mechanism is better for that.
We are going to see longer vectors in commodity processors. Sandy Bridge will have AVX. Haswell will have something more. What's not clear is how Intel plans to architect the memory system. That will be interesting to see.
I need to correct a few things about your post.
Clusters are not supercomputers. Supercomputing is not only or even primarily about processor technology. The network and memory architecture have a larger impact. Supercomputer vendors are doing plenty of architecture innovation. They're doing it in the memory, network and I/O layers.
Supercomputers do not exist mainly to run single large apps. They are batch machines running multiple jobs simultaneously.
Supercomputer customers are not willing to pay a premium for custom processors because CPU efficiency is not the most important thing. Memory and network performance is. That's why the article focuses on the memory bandwidth problems of many-core CPUs.
What we're seeing with multi- and man-core processing is the continued commoditization of supercomputer technology. It started with the micros' adaption of strong scalar performance. The Cray-1 was not outstanding because it had vector processing. It was outstanding because it was the fatest scalar processor in the world at the time. Then the micros adopted vector processing (albeit with very short vectors, but that will change with Sandy Bridge and Haswell). Then they adopted multithreading. Now they're embracing multiprocessing.
It's the nature of the business that technology trickles down from the high end to the consumer. If you look at the timeline of processor development, commodity CPUs have gained supercomputer features about 20-30 years after their first appearance in high-end computing. What hasn't happened is commodity adoption of high-end memory, network and I/O systems because the home user does not need them.
Cray did not stream vectors from memory. One of the advances of the Cray-1 was the use of vector registers as opposed to, for example, the Burroughs machines which streamed vectors directly to/from memory.
We know how to build memory systems that can handle large vectors. Both the Cray X1 and Earth Simulator demonstrate that. The problem is that those memory systems are currently too expensive. We are going to see more and more vector processing in commodity processors.
I think hyper-individualism is one of the primary traits of the early US citizens that made this into the powerhouse country it is today.
Really? How did we do fantastic things like build the transcontinental railroad, then?
Individualism doesn't preclude helping out your neighbor either, but it is your choice whether you do and to what extent. I'd much rather give and help on my own than be forced to do so by the government.
You're confusing ideas here. Individualism is fine. It's a good thing. Hyper-individualism is bad. It does in fact preclude helping your neighbor and contributing to the common good. Not everyone has bootstraps to pull themselves up with. We used to recognize that as a country. We don't now. I am hopeful that will change with the new administration.
As for being "forced by the government," I suppose you advocate abolishing all taxes. Well, I've got news for you: the churches can't provide all the social services, infrastructure, education, labor protections, etc. that can only be provided through government, which is actually us. Government is the tool we use to collectively make decisions about what kind of community we want to live in.
There are some things that everyone owns.
There is a lot of good culture in America. We just tend to underappreciate the work of ordinary people and instead look to grand leaders to create culture, which by its nature is a grassroots phenomenon.
No. Everything in American culture that is not completely worthless was created by social rejects, people who were thrown out of American society and were left with nothing but poverty, scorn or both, and then-current American culture painted them as targets and enemies of "good American people". They had no choice but to create their own culture, untainted by hostility and shallowness of the mainstream.
That's pretty much the definition of "grassroots."
Society accepted pieces of those rejects' culture when it became commercially profitable and politically convenient -- usually at the expense of most of the original meaning. Majority of American population passively consumed that filtered, diluted and packaged culture -- be it music, art, literature, cinema, or anything else.
Oh please. Don't get all elitist on me. There's plenty of great stuff that was derided as "packaged culture" in years past. It's because we refuse to appreciate our American culture that too many people seem to think we don't have any. Culture is what a society is, for good or bad. By definition each society has one.
Just a regional name for casserole.
You need to get out way more if you think that's the epitome of anything except easy cleanup.
WHOOSH!
Also fuck you and your 'community responsibility' 'common good' BS. Code for 'do what your told' and 'pay your taxes'. I do as little of ether as illegally possible.
Uh huh. Enjoyed those Interstates lately? How about your house? Benefit from the railroads much?
Yet another nonesense libertarian rant.
Right on.
Every culture has its high and low points. Parts of our American culture are quite sick. For example, the hyper-individualism that is destroying any sense of community responsibility or the common good must be reversed if we're going to survive as a nation.
But there are also truly wonderful things that have come out of the unique mixtures of peoples here: jazz, country, rock&roll, for example, each of which influenced the others and each of which is far too pigeonholed today by people who don't care to become educated about our own cultural history.
There are all sorts of American culinary delights: Cajun feasts, Southern barbeque, Western concoctions and the epitome of gastric delight, the Minnesota Hot Dish.
We've got incredible diversity in language and idioms, from Texas hyperbole to Midwestern understatement.
There is a lot of good culture in America. We just tend to underappreciate the work of ordinary people and instead look to grand leaders to create culture, which by its nature is a grassroots phenomenon.
Wrong. There is a world of difference between a supercomputer and a cluster. The network is not simply "faster" or "higher bandwidth." These things are custom-designed to fit the type of processing a supercomputer needs to do. These are not embarrassingly parallel codes. In fact, that's why the Top 500 is bogus. It ranks computers based on their ability to run an embarrassingly parallel code. We really ought to be measuring with something like HPCC.
Actually, the analogy doesn't quite fit. Vectorization is quite a bit more comprehensible than the contortions compilers have to go through to get performance out of IA-64.
The user directives are easier, too. It's much easier for a user to say, "this loop has no cross-iteration dependencies" than for the user to mark which branches should be if-converted, which loads should be speculatively hoisted, etc. Thus for IA-64 the compiler ends up having to try to figure all this stuff out. In other words, it's much easier for the user to help the compiler understand the general parallelism available in the code than it is for the user to try to convey minute detail.
IA-64 is in many ways a house of cards. The compiler has to do a bunch of things (predication, static speculation, VLIW packing, scheduling, etc., etc., etc.) and do them all well to get performance. If it fails at one of them the code tanks. For something like Larrabee where there are multipel paradigms available (serial, vector, and multicore), you don't have to use all of them perfectly. You just have to use them well enough.
While it's easier to express parallelism, what's missing in Tim's responses is the acknowledgment that the user will have to do this.
You can read the Larrabee paper to get a good idea of the kinds of things Tim is talking about (software rendering and why it may just work).
I think the point of the article is that computing paradigms are merging. You won't have a CPU and a GPU. You'll have one thing that looks like both. In other words, you'll have a multicore, parallel, vector machine.
And that, absolutely, positively, will happen. Larrabee, or something like it, is the future. If you hold AMD stock, sell now, because Fusion doesn't sound anything like Larrabee and is going to seem positively draconian by the time it comes out.
In some ways, these new processors will look like a Cray YMP on your desktop. It's a rough analogy but suitable for illustration. Of course there are all sorts of differences in the way the memory systems will work and that's a huge part of the performance equation.
It seems to me that Tim puts a bit too much faith in compilers. He talks about language extensions but only in CUDA-like terms of "where things will run." A compiler needs a lot of information to be able to vectorize. The user often has to provide that information in a language like C because of its loose typing, aliasing and side-effect rules.
My prediction is that some APIs will go away, but many of the low-level ones will stay because it's often faster to call into a hand-coded library than rely on the compiler to have enough information to automatically optimize the code. Eventually compilers will start pattern-matching to these APIs. Higher-level APIs will be developed to save developer time, not CPU time. They will exist almost purely for code reuse purposes.
I disagree with Tim that hardware vendors will differentiate on performance. At least, in the way he's thinking. It won't be hardware gadgets, vector length or number of pipes that matter. It's going to be the compiler, programming environment and libraries. To the extent that the hardware supports those in its ISA, hardware will matter. But the bulk of the muscles of the chip won't matter so much as their placement and utility (by the compiler). The inflection point is leading to a world where software is king.
Right on. This is the real problem with Perl. Everyone rolls their own and we end up with multiple reimplementations of the same concept done in sometimes subtly different ways.
CPAN is one of my worst nightmares. There's no peer review so one ends up looking at several different libraries that do the same thing, only they all do it differently and have varying levels of feature complexity. I just want to get work done, not evaluate libraries.