No. It's discrete signal processing, similar to the signal (pulse) processing done in the brain. Stream processing is vector or data parallel processing. It's a completely different beast.
From the interview I see that Knuth thinks multithreading may turn out to be a flop. I agree and I would go even further. I don't think there is any question that the multithreading strategy used by Intel and AMD for multicore programming is a big flop already. Multithreading is the second worst thing to have happened to computing. The worst is single-threading which is what normal algorithmic programming is based on. Parallelism is the correct approach to computing. Computing should have been parallel from the start even on single-core processors. If parallelism had been emulated in a processor from the beginning, adding more cores would have been a simple evolutionary transition, a mere engineering problem. My point is that there is a better way to do parallel programming that does not involve threads at all. Cellular automata and neural network programmers have been emulating parallism for decades without threads.
Essentially, you need to have two buffers and a loop. While the first buffer is being processed, the second buffer is filled with objects to be processed in the next cycle. When the first buffer is done, swap the buffers and start over. Two buffers are used in order to prevent signal racing conditions. It's not rocket science. We just need to take it to the instruction level by changing it from a software mechanism into a hardware mechanism. In other words, the mechanism should reside in the processor, whether single or multicore. This is the correct approach to parallelism. Multithreading is going to be a complete disaster, a multi-billion dollar disaster. Google "Nightmare on Core Street" to find out why multithreading is not part of the future of parallel computing.
Gosh. You make it sound like AMD is being accused of a crime. The point I was making is that, even if there are only a few former Intel leaders at AMD, the fact remains that AMD was formed to compete against Intel on Intel's own turf, x86 compatibility. Not that there was anything wrong with that in those days. AMD did pretty well considering what they were up against. My point is that the computer world has changed drastically to the point that even Intel is getting scared, whether or not they admit it. It is time to change. The x86 line is ancient technology from the last century. Now the market is screaming for a multicore processor that is super fast, and uses a fine grain MIMD software model to run rock solid applications. In addition, the new processor must be so easy to program that everybody will want to abandon their legacy code and adopt the new model. You cannot come up with a processor like that if all you think about is how to maintain a slight me-too superiority over Intel's technology. AMD must climb up the vision pole so they can see what's beyond Intel's backyard.
I don't agree with this: it was AMD that led Intel into the world of on-die memory controllers as well as removing the front side bus from PC architecture.
These things are just evolutionary improvements to exisiting technology and it goes to prove that AMD does have excellent engineering talent, which was never really in doubt. What is needed now is a completely new technology to solve a nasty problem that threatens to bring the industry down to its knees. AMD can do it. Question is, do Ruiz and the big money people behind AMD have the huevos and the vision to step up to the plate?
Parallel processing is a new paradigm? Since when? The 1960s called, they want you to stop stealing their ideas.
What's being passed as parallel processing is not really parallel processing. In fact this is the reason that parallel programming is so hard: it's not what it's claimed to be. Switch to a true parallel programming model and the problem will disappear. Read Why Parallel Programming Is So Hard to find out why multithreading is really a fake parallelism.
Nvidia has a better chance to compete successfully against Intel because their executives do not think like Intel. AMD, OTOH, is a monkey-see-monkey-do company. Many of their executives (e.g., Dirk Meyer) and lead engineers came from Intel and they only see the world through Intel glasses. Having said that, this business of mixing coarse-grain MIMD and fine-grain SIMD cores on a single die to create a heterogeneous processor is a match made in hell. Anybody with a lick of sense can tell you that universality should be the primary goal of multicore research and that incompatible processing models should not be encouraged let alone slapped together. Programming those hybrid processors will be more painful than pulling teeth with a crowbar. Heck, breaking programs down into threads is a pain in the ass. Why would anybody want to make things worse?
The best strategy, IMO, is to work on a universal processor that combines the strengths of both MIMD and SIMD models while eliminating their obvious weaknesses. AMD needs somebody with the huevos to say, "fooey with this Intel crap! Let's carve our own market and create a completely new technology for a completely new paradigm, parallel processing". Is Hector Ruiz up to the task? Only time will tell. For a different take on the multicore and CPU/GPU issue, read Nightmare on Core Street.
As I already pointed out, the OS is what will determine how to parallelize things.
Not at all. The processor should be designed for the software model, not the other way around. All of our current processors are designed for the algorithmic software model, which that was created 150 years ago by Charles Babbage. It's time to change. Like it or not, the computer industry will be dragged kicking and screaming into the 21st century.
Although VHDL is a hardware description language, couldn't similar concepts be used to make a parallel centric computer programming language?
Excellent suggestion. This is precisely what the COSA software model is about. A pulsed neural network is my preferred metaphor for an ideal model of parallel computing. Intel and the others are on the verge of losing billions of dollars because they are already deeply committed to the hard to program multithreading model, a complete failure even after decades of research. To find out why multithreading is not part of the future of parallel programming, read Nightmare on Core Street.
You seem confused. I simply said I can do it and I know other developers who can. Just because the majority of developers can't doesn't mean it has "failed": it just means there are a lot of mediocre developers who need new tools to manage the job, which is where that $20million is being spent. AMD and Intel and others want to make it easier for you. Personally I'm quite at home writing multithreaded kernel level C, but clearly not everyone is.
IOW, you're smart and everybody else is an idiot. You sound like a pompous ass to me. So much for your solution to the parallel programming problem.
Parallel systems such as MPI have been the staple of high performance computing since the mid 90's, and there are plenty of developers (including myself) who can write multi-threaded code without breaking into a sweat, and get it right.
In that case, you should hurry and tell Microsoft and Intel to refrain from giving that 20 million they want to give to UC Berkeley and UI Urbana-Champaign to find a solution to the parallel programming problem. According to you, Microsoft, Intel, AMD and all the others are wasting hundreds of millions in research labs around the world trying to make it easy to build apps with threads. After all, you already found the solution, right? And you found an easy way to build threaded programs, right?
Needless to say I was a bit early on that prediction.
The reason is that all academic researchers jumped on the multithreading bandwagon as the basis for parallel computing. Unfortunately for them, they could never get it to work. They've been at it for over twenty years and they still can't make it work. Twenty years is an eternity in this business. You would think that after all this time, it would have occurred to at least one of those smart scientists that maybe, just maybe, multithreading is not the answer to parallel computing. Nope: they're still trying to fit that square peg into the round hole.
Both AMD and Intel have invested heavily into the multithreading model. Big mistake. Multiple billion dollar mistake. To find out why threads are not part of the future of parallel computing read Nightmare on Core Street. It's time for the computer industry to wake up and realize that the analytical engine is long gone. This is the 21st century. It's time to move on and change to a new model of computing.
Consider porting legacy systems (quite a few in this case), re-educating all those low-level programmers, and the "architecture wars," for lack of a better term,
I have. It can all be done easily.
as you try to evangelize this new platform as being better...
Is your name Reinders, by any chance? Or do you work for Intel?:-) I don't get paid for doing this. Do you? It's a labor of love. I believe in it. I just want to see a cheap, superfast and reliable supercomputer on my desk before I die. Maybe with some cool AI application in it, you know, something I can talk to in English, something that knows me. I hate keyboards.
What about multiple sockets? The problem is not parallelising to merely up to around 8 cores, but to scale up to a fully-blown multi socket and a possibly NUMA system. And with almost any penalty, you're going to incur a loss when parallelising very small, simple operations.
There does not have to be a penalty if neighboring cores share registers or even caches. The trick is to keep computations local to neighbors only. It can be done. It's a matter of instruction scheduling. Besides, even if there is a slight penalty, only a small percentage of situations will fall under the category that you describe. A QSort (or tree search function) lends itself very nicely to fine-grained parallelism, for example. Ultimately, physics will solve the Von Neumann bottleneck and cores will go directly to fast memory: no need for caches then.
Well, you know. If you don't like my copying and pasting, don't read my stuff. I mean, nobody is twisting your arm.
it's not really relevant to the real world unless we adopt a whole new (as-yet uninvented) hardware architecture.
It's going to be relevant soon enough because threads are a major pain in the ass. Add heterogeneous processors into the mix and you have a major disaster. BTW, it does not take that long to design and build a new processor anymore, even if it's a multicore processor. It can be done in months if you have a firm idea in mind.
And you're wrong about having to read all my stuff to find out that we need a whole new architecture since I mentioned in the original post that the computer industry needs to abandon Babbage's sequential model and move to a new computing model. Sorry.
The theory of fine-grained parallelism is fundamentally flawed by the fact that parallelisation itself incurs an overhead, due to locking and syncing shared data structures.
Fine-grained parallelism works fine. It works in your NVidia, SIMD-based graphics coprocessor, does it not? Locking and syncing is a problem only in a non-deterministic environment like multithreading. Fine-grained parallelism is temporally deterministic because the temporal (concurrent or sequential) order of code execution can be precisely determined.
you'll probably have to wire the registers between cores directly into each other in order to avoid the enormous overhead of an external chip.
Not really. There is a way to design a multicore processor such that only neighboring cores cooperate on related computation. It is part of the self-balancing mechanism. I can't go into detail but suffice it to say that if you keep your inter-core communication performance penalty at a fixed level regardless of the number of cores, you have a winner.
One day soon, the computer industry will realize that, 150 years after Charles Babbage came up with his idea of a general purpose sequential computer, it is time to move on and change to a new computing model. The industry will be dragged kicking and screaming into the 21st century. For over 20 years, researchers in parallel and high-performance computing have tried to come up with an easy way to use threads for parallel programming. They have failed and they have failed miserably. Amazingly, they are still continuing to pursue the multithreading approach. None other than Dan Reed, Director of scalable and multicore computing at Microsoft Research, believes that multithreading over time will become part of the skill set of every professional software developer (source: cio.com). What is wrong with this picture? Threads are a major disaster: They are coarse-grained, they are a pain in the ass to write and hard to debug and maintain. Reinders knows this. He's pushing threads, not because he wants your code to run faster but because Intel's multicore CPUs are useless for non-threaded apps.
Reinders is not an evangelist for nothing. He's more concerned about future-proofing Intel's processors than anything else. You listen to him at your own risk because the industry's current multicore strategy will fail and it will fail miserably.
Threads were never meant to be the basis of a parallel computing model but as a mechanism to execute sequential code concurrently. To find out why multithreading is not part of the future of parallel programming, read Nightmare on Core Street. There is better way to achieve fine-grain, deterministic parallelism without threads.
I don't think anyone except you said anything about threads. You may have just described exactly what the GP was describing -- point is, why should you have to break them down into individual programs yourself?
This is precisely what is wrong with the current approach. The idea that the problem should be addressed from the top down has been tried for decades and has failed miserably. The idea that we should continue to write implicitly sequential programs and have some tool extract the parallelism by dividing the program into concurrent threads is completely ass-backwards, IMO. We should start with parallel elements and build implicitly parallel modules from those elements. Nothing needs to be broken down. Hardware engineers have been doing it for years and it works.
Personally, I like Erlang, but the point is the same -- come up with a toolset and/or programming paradigm which makes scaling to thousands of cores easy and natural.
Erlang is not the answer for many reasons, otherwise the entire computer world (especially the high performance parallel research community) would have jumped on it in earnest since it's been around for quite a while. One reason is that it uses a coarse-grain approach to parallelism; you can't even parallelize a quicksort routine (an ideal candidate for parallel processing) in Erlang. Consider also that Erlang is not deterministic and has no mechanism for automatic load balancing. The same goes for all the other functional programming language approaches to concurrency.
The only problem I have yet to see addressed is how to properly test a threaded app, as it's non-deterministic.
The solution is not to use threads at all. They are not needed for parallelism. The difficulty of programming with threads is the primary reason that the multicore industry is in a panic right now. Multithreaded programs are a nightmare to maintain, especially if you did not write the original code. Ask the folks at Intel, Microsoft and AMD. And it's not for the lack of trying. Billions of dollars have been spent on making multithreaded parallel programming easy in the last two decades. They're still at it. What's wrong with this picture?
Instead, you describe, using the toolset, the problem in a way which is decomposable, and the tools spread the work over the 1000+ cores.
One day soon, the computer industry will realize that, 150 years after Charles Babbage invented his idea of a general purpose sequential computer, it is time to move on and change to a new computing model. The industry will be dragged kicking and screaming into the 21st century. Threads were not originally intended to be the basis of a parallel software model but only a mechanism for running multiple sequential (not parallel) programs concurrently. The multithreading approach to parallel computing embraced by Intel, AMD and Microsoft is a disaster in the making because the future of computing is not multithreaded. See Nightmare on Core Street for more.
The problem is when you have a single CPU-intensive task, and you want to split that over multiple processors. That, in general, is a difficult problem. Various solutions, such as functional programming, threads with spawns and waits, etc. have been proposed, but none are as easy as just using a simple procedural language.
Yes, the reason is that we are trying to fit a square peg into a round hole. Multithreading was not originally intended to be the basis of a parallel programming model but as a mechanism to run sequential (not parallel) programs concurrently. What is needed is an inherently parallel model where parallelism is implicit and sequential order is explicit. Read Parallel Programming, Math and the Curse of the Algorithm and Nightmare on Core Street for more.
it doesn't matter how fancy a name for it you find, the fact is that the equations don't add up.
In my opinion, when a theory does not agree with observation, it's the theory's fault, not nature's fault. Why is it that physicists are so reluctant to revise their theories? Why is it that they must invent every cockamamie nonsense in the book to excuse the obvious flaws in their theories? I'll tell you why. It's because a lot of people would get egg on their faces, that's why. It's all politics and human nature. Thomas Kuhn was right. I can't wait for the next revolution in physics.
Sounds more like crackpot physics to me. Only physicists can get away with crap like this. In any other field of science, this sort of voodoo bullshit would not be tolerated. I tell it like I see it. Mod me down and see if I care. ahahaha...
There are a few exceptions. The hagfish and some underground salamanders have no cerebellum. They are not known for their intelligence, by the way. Note that, among the cephalopods, the octopus and the squid are known to have a brain area that closely matches the neuro-architecture of the vertebrate cerebellum. It's safe to say that they do have a cerebellum. These are rather intelligent animals. I am not saying that the cerebellum is responsible for intelligence but it certainly helps by handling a lot of the routine automatic tasks, thus leaving more time for the cerebrum to attend to strategic functions.
My MOTOR skills are affected but my higher brain functions are completely unaffected.
Very interesting. Thank you for sharing your experience here. I don't see where we disagree. What is amazing to me is that there is a strong movement within the neuroscience community that wants to ascribe all sorts of cognitive powers to the cerebellum including language and speech processing. What you wrote essentiallly confirms (to me at least) that it's all hogwash. Look up the often cited (pseudoscientific) work of Henrietta and Alan Leiner on Google and you'll know what I'm talking about.
Slashdot Mirror
So it's stream processing.
No. It's discrete signal processing, similar to the signal (pulse) processing done in the brain. Stream processing is vector or data parallel processing. It's a completely different beast.
From the interview I see that Knuth thinks multithreading may turn out to be a flop. I agree and I would go even further. I don't think there is any question that the multithreading strategy used by Intel and AMD for multicore programming is a big flop already. Multithreading is the second worst thing to have happened to computing. The worst is single-threading which is what normal algorithmic programming is based on. Parallelism is the correct approach to computing. Computing should have been parallel from the start even on single-core processors. If parallelism had been emulated in a processor from the beginning, adding more cores would have been a simple evolutionary transition, a mere engineering problem. My point is that there is a better way to do parallel programming that does not involve threads at all. Cellular automata and neural network programmers have been emulating parallism for decades without threads.
Essentially, you need to have two buffers and a loop. While the first buffer is being processed, the second buffer is filled with objects to be processed in the next cycle. When the first buffer is done, swap the buffers and start over. Two buffers are used in order to prevent signal racing conditions. It's not rocket science. We just need to take it to the instruction level by changing it from a software mechanism into a hardware mechanism. In other words, the mechanism should reside in the processor, whether single or multicore. This is the correct approach to parallelism. Multithreading is going to be a complete disaster, a multi-billion dollar disaster. Google "Nightmare on Core Street" to find out why multithreading is not part of the future of parallel computing.
That's a pretty serious mischaracterization.
Gosh. You make it sound like AMD is being accused of a crime. The point I was making is that, even if there are only a few former Intel leaders at AMD, the fact remains that AMD was formed to compete against Intel on Intel's own turf, x86 compatibility. Not that there was anything wrong with that in those days. AMD did pretty well considering what they were up against. My point is that the computer world has changed drastically to the point that even Intel is getting scared, whether or not they admit it. It is time to change. The x86 line is ancient technology from the last century. Now the market is screaming for a multicore processor that is super fast, and uses a fine grain MIMD software model to run rock solid applications. In addition, the new processor must be so easy to program that everybody will want to abandon their legacy code and adopt the new model. You cannot come up with a processor like that if all you think about is how to maintain a slight me-too superiority over Intel's technology. AMD must climb up the vision pole so they can see what's beyond Intel's backyard.
Look, parallel processing is new for the mainstream. This is the point that Bill McColl made on his blog recently. Read Microsoft Agrees, Parallelism IS The New New Thing!
I don't agree with this: it was AMD that led Intel into the world of on-die memory controllers as well as removing the front side bus from PC architecture.
These things are just evolutionary improvements to exisiting technology and it goes to prove that AMD does have excellent engineering talent, which was never really in doubt. What is needed now is a completely new technology to solve a nasty problem that threatens to bring the industry down to its knees. AMD can do it. Question is, do Ruiz and the big money people behind AMD have the huevos and the vision to step up to the plate?
Parallel processing is a new paradigm? Since when? The 1960s called, they want you to stop stealing their ideas.
What's being passed as parallel processing is not really parallel processing. In fact this is the reason that parallel programming is so hard: it's not what it's claimed to be. Switch to a true parallel programming model and the problem will disappear. Read Why Parallel Programming Is So Hard to find out why multithreading is really a fake parallelism.
Nvidia has a better chance to compete successfully against Intel because their executives do not think like Intel. AMD, OTOH, is a monkey-see-monkey-do company. Many of their executives (e.g., Dirk Meyer) and lead engineers came from Intel and they only see the world through Intel glasses. Having said that, this business of mixing coarse-grain MIMD and fine-grain SIMD cores on a single die to create a heterogeneous processor is a match made in hell. Anybody with a lick of sense can tell you that universality should be the primary goal of multicore research and that incompatible processing models should not be encouraged let alone slapped together. Programming those hybrid processors will be more painful than pulling teeth with a crowbar. Heck, breaking programs down into threads is a pain in the ass. Why would anybody want to make things worse?
The best strategy, IMO, is to work on a universal processor that combines the strengths of both MIMD and SIMD models while eliminating their obvious weaknesses. AMD needs somebody with the huevos to say, "fooey with this Intel crap! Let's carve our own market and create a completely new technology for a completely new paradigm, parallel processing". Is Hector Ruiz up to the task? Only time will tell. For a different take on the multicore and CPU/GPU issue, read Nightmare on Core Street.
As I already pointed out, the OS is what will determine how to parallelize things.
Not at all. The processor should be designed for the software model, not the other way around. All of our current processors are designed for the algorithmic software model, which that was created 150 years ago by Charles Babbage. It's time to change. Like it or not, the computer industry will be dragged kicking and screaming into the 21st century.
Although VHDL is a hardware description language, couldn't similar concepts be used to make a parallel centric computer programming language?
Excellent suggestion. This is precisely what the COSA software model is about. A pulsed neural network is my preferred metaphor for an ideal model of parallel computing. Intel and the others are on the verge of losing billions of dollars because they are already deeply committed to the hard to program multithreading model, a complete failure even after decades of research. To find out why multithreading is not part of the future of parallel programming, read Nightmare on Core Street.
I agree. Qsort is a perfect candidate for parallelism. Check out the COSA parallel quicksort page.
You seem confused. I simply said I can do it and I know other developers who can. Just because the majority of developers can't doesn't mean it has "failed": it just means there are a lot of mediocre developers who need new tools to manage the job, which is where that $20million is being spent. AMD and Intel and others want to make it easier for you. Personally I'm quite at home writing multithreaded kernel level C, but clearly not everyone is.
IOW, you're smart and everybody else is an idiot. You sound like a pompous ass to me. So much for your solution to the parallel programming problem.
Parallel systems such as MPI have been the staple of high performance computing since the mid 90's, and there are plenty of developers (including myself) who can write multi-threaded code without breaking into a sweat, and get it right.
In that case, you should hurry and tell Microsoft and Intel to refrain from giving that 20 million they want to give to UC Berkeley and UI Urbana-Champaign to find a solution to the parallel programming problem. According to you, Microsoft, Intel, AMD and all the others are wasting hundreds of millions in research labs around the world trying to make it easy to build apps with threads. After all, you already found the solution, right? And you found an easy way to build threaded programs, right?
Sure.
Needless to say I was a bit early on that prediction.
The reason is that all academic researchers jumped on the multithreading bandwagon as the basis for parallel computing. Unfortunately for them, they could never get it to work. They've been at it for over twenty years and they still can't make it work. Twenty years is an eternity in this business. You would think that after all this time, it would have occurred to at least one of those smart scientists that maybe, just maybe, multithreading is not the answer to parallel computing. Nope: they're still trying to fit that square peg into the round hole.
Both AMD and Intel have invested heavily into the multithreading model. Big mistake. Multiple billion dollar mistake. To find out why threads are not part of the future of parallel computing read Nightmare on Core Street. It's time for the computer industry to wake up and realize that the analytical engine is long gone. This is the 21st century. It's time to move on and change to a new model of computing.
Consider porting legacy systems (quite a few in this case), re-educating all those low-level programmers, and the "architecture wars," for lack of a better term,
:-) I don't get paid for doing this. Do you? It's a labor of love. I believe in it. I just want to see a cheap, superfast and reliable supercomputer on my desk before I die. Maybe with some cool AI application in it, you know, something I can talk to in English, something that knows me. I hate keyboards.
I have. It can all be done easily.
as you try to evangelize this new platform as being better...
Is your name Reinders, by any chance? Or do you work for Intel?
What about multiple sockets? The problem is not parallelising to merely up to around 8 cores, but to scale up to a fully-blown multi socket and a possibly NUMA system. And with almost any penalty, you're going to incur a loss when parallelising very small, simple operations.
There does not have to be a penalty if neighboring cores share registers or even caches. The trick is to keep computations local to neighbors only. It can be done. It's a matter of instruction scheduling. Besides, even if there is a slight penalty, only a small percentage of situations will fall under the category that you describe. A QSort (or tree search function) lends itself very nicely to fine-grained parallelism, for example. Ultimately, physics will solve the Von Neumann bottleneck and cores will go directly to fast memory: no need for caches then.
Well, you know. If you don't like my copying and pasting, don't read my stuff. I mean, nobody is twisting your arm.
it's not really relevant to the real world unless we adopt a whole new (as-yet uninvented) hardware architecture.
It's going to be relevant soon enough because threads are a major pain in the ass. Add heterogeneous processors into the mix and you have a major disaster. BTW, it does not take that long to design and build a new processor anymore, even if it's a multicore processor. It can be done in months if you have a firm idea in mind.
And you're wrong about having to read all my stuff to find out that we need a whole new architecture since I mentioned in the original post that the computer industry needs to abandon Babbage's sequential model and move to a new computing model. Sorry.
The theory of fine-grained parallelism is fundamentally flawed by the fact that parallelisation itself incurs an overhead, due to locking and syncing shared data structures.
Fine-grained parallelism works fine. It works in your NVidia, SIMD-based graphics coprocessor, does it not? Locking and syncing is a problem only in a non-deterministic environment like multithreading. Fine-grained parallelism is temporally deterministic because the temporal (concurrent or sequential) order of code execution can be precisely determined.
you'll probably have to wire the registers between cores directly into each other in order to avoid the enormous overhead of an external chip.
Not really. There is a way to design a multicore processor such that only neighboring cores cooperate on related computation. It is part of the self-balancing mechanism. I can't go into detail but suffice it to say that if you keep your inter-core communication performance penalty at a fixed level regardless of the number of cores, you have a winner.
One day soon, the computer industry will realize that, 150 years after Charles Babbage came up with his idea of a general purpose sequential computer, it is time to move on and change to a new computing model. The industry will be dragged kicking and screaming into the 21st century. For over 20 years, researchers in parallel and high-performance computing have tried to come up with an easy way to use threads for parallel programming. They have failed and they have failed miserably. Amazingly, they are still continuing to pursue the multithreading approach. None other than Dan Reed, Director of scalable and multicore computing at Microsoft Research, believes that multithreading over time will become part of the skill set of every professional software developer (source: cio.com). What is wrong with this picture? Threads are a major disaster: They are coarse-grained, they are a pain in the ass to write and hard to debug and maintain. Reinders knows this. He's pushing threads, not because he wants your code to run faster but because Intel's multicore CPUs are useless for non-threaded apps.
Reinders is not an evangelist for nothing. He's more concerned about future-proofing Intel's processors than anything else. You listen to him at your own risk because the industry's current multicore strategy will fail and it will fail miserably.
Threads were never meant to be the basis of a parallel computing model but as a mechanism to execute sequential code concurrently. To find out why multithreading is not part of the future of parallel programming, read Nightmare on Core Street. There is better way to achieve fine-grain, deterministic parallelism without threads.
I don't think anyone except you said anything about threads. You may have just described exactly what the GP was describing -- point is, why should you have to break them down into individual programs yourself?
This is precisely what is wrong with the current approach. The idea that the problem should be addressed from the top down has been tried for decades and has failed miserably. The idea that we should continue to write implicitly sequential programs and have some tool extract the parallelism by dividing the program into concurrent threads is completely ass-backwards, IMO. We should start with parallel elements and build implicitly parallel modules from those elements. Nothing needs to be broken down. Hardware engineers have been doing it for years and it works.
Personally, I like Erlang, but the point is the same -- come up with a toolset and/or programming paradigm which makes scaling to thousands of cores easy and natural.
Erlang is not the answer for many reasons, otherwise the entire computer world (especially the high performance parallel research community) would have jumped on it in earnest since it's been around for quite a while. One reason is that it uses a coarse-grain approach to parallelism; you can't even parallelize a quicksort routine (an ideal candidate for parallel processing) in Erlang. Consider also that Erlang is not deterministic and has no mechanism for automatic load balancing. The same goes for all the other functional programming language approaches to concurrency.
The only problem I have yet to see addressed is how to properly test a threaded app, as it's non-deterministic.
The solution is not to use threads at all. They are not needed for parallelism. The difficulty of programming with threads is the primary reason that the multicore industry is in a panic right now. Multithreaded programs are a nightmare to maintain, especially if you did not write the original code. Ask the folks at Intel, Microsoft and AMD. And it's not for the lack of trying. Billions of dollars have been spent on making multithreaded parallel programming easy in the last two decades. They're still at it. What's wrong with this picture?
Instead, you describe, using the toolset, the problem in a way which is decomposable, and the tools spread the work over the 1000+ cores.
One day soon, the computer industry will realize that, 150 years after Charles Babbage invented his idea of a general purpose sequential computer, it is time to move on and change to a new computing model. The industry will be dragged kicking and screaming into the 21st century. Threads were not originally intended to be the basis of a parallel software model but only a mechanism for running multiple sequential (not parallel) programs concurrently. The multithreading approach to parallel computing embraced by Intel, AMD and Microsoft is a disaster in the making because the future of computing is not multithreaded. See Nightmare on Core Street for more.
The problem is when you have a single CPU-intensive task, and you want to split that over multiple processors. That, in general, is a difficult problem. Various solutions, such as functional programming, threads with spawns and waits, etc. have been proposed, but none are as easy as just using a simple procedural language.
Yes, the reason is that we are trying to fit a square peg into a round hole. Multithreading was not originally intended to be the basis of a parallel programming model but as a mechanism to run sequential (not parallel) programs concurrently. What is needed is an inherently parallel model where parallelism is implicit and sequential order is explicit. Read Parallel Programming, Math and the Curse of the Algorithm and Nightmare on Core Street for more.
it doesn't matter how fancy a name for it you find, the fact is that the equations don't add up.
In my opinion, when a theory does not agree with observation, it's the theory's fault, not nature's fault.
Why is it that physicists are so reluctant to revise their theories? Why is it that they must invent every cockamamie nonsense in the book to excuse the obvious flaws in their theories? I'll tell you why. It's because a lot of people would get egg on their faces, that's why. It's all politics and human nature. Thomas Kuhn was right. I can't wait for the next revolution in physics.
Sounds more like crackpot physics to me. Only physicists can get away with crap like this. In any other field of science, this sort of voodoo bullshit would not be tolerated. I tell it like I see it. Mod me down and see if I care. ahahaha...
All vertebrates have a cerebellum
There are a few exceptions. The hagfish and some underground salamanders have no cerebellum. They are not known for their intelligence, by the way. Note that, among the cephalopods, the octopus and the squid are known to have a brain area that closely matches the neuro-architecture of the vertebrate cerebellum. It's safe to say that they do have a cerebellum. These are rather intelligent animals. I am not saying that the cerebellum is responsible for intelligence but it certainly helps by handling a lot of the routine automatic tasks, thus leaving more time for the cerebrum to attend to strategic functions.
My MOTOR skills are affected but my higher brain functions are completely unaffected.
Very interesting. Thank you for sharing your experience here. I don't see where we disagree. What is amazing to me is that there is a strong movement within the neuroscience community that wants to ascribe all sorts of cognitive powers to the cerebellum including language and speech processing. What you wrote essentiallly confirms (to me at least) that it's all hogwash. Look up the often cited (pseudoscientific) work of Henrietta and Alan Leiner on Google and you'll know what I'm talking about.