Slashdot Mirror
Moore's Law Will Die Without GPUs
Stoobalou writes "Nvidia's chief scientist, Bill Daly, has warned that the long-established Moore's Law is in danger of joining phlogiston theory on the list of superseded laws, unless the CPU business embraces parallel processing on a much broader scale."
Moore's is not a law, but an observation!
I didn't realise Moore's Law was purely the driving force behind CPU development and not just an observation on semiconductor development. Surely we just say Moore's Law held until a certain point, then someone else's Law takes over?
As for Phlogiston theory - it was just that, a theory which was debunked.
Dr. Daly believes the only way to continue to make great strides in computing performance is to ... offload some of the work onto GPU's that his company just happens to make? [Arte Johnson] Very interesting .
The industry has moved away from "more horsepower than you'll ever need!" to "uses less power than you can ever imagine!" Perpetuating Moore's Law isn't an industry requirement, it's a prediction by a guy who was in the chip industry.
So, a graphics card manufacturer says that graphics cards are the future? And this is news?
But the only "law" is that the number of transistors doubles in a certain time (something of a self fulfilling prophesy these days since this is the yardstick the chip companies work to).
Once transistors get below a certain size, of course it will end. Parallel or serial doesn't change things. We either have more processors in the same space, more complex processors or simply smaller processors. There's no "saving" to be done.
considering that Moore's Law was based on the observation that they were able to double the number of transistors about every 20 months, it would be inevitable that at some point they reach a limiting factor. The factor seems to be the process size, which is a physical barrier. As the process size continues to decrease, the physical size of atoms is a barrier that they can't get past.
Obviously "NVIDIA's Chief Scientist" is going to say something about the epochal importance of GPUs; but WTF?
Moore's law, depending on the exact formulation you go with, posits either that transistor density will double roughly every two years or that density at minimum cost/transistor increases at roughly that rate.
It is pretty much exclusively a prediction concerning IC fabrication(a business that NVIDIA isn't even in, TSMC handles all of their actual fabbing), without any reference to what those transistors are used for.
Now, it is true that, unless parallel processing can be made to work usefully on a general basis, Moore's law will stop implying more powerful chips, and just start implying cheaper ones(since, if the limits of effective parallel processing mean that you get basically no performance improvements going from X billion transistors to 2X billion transistors, Moore's law will continue; but instead of shipping faster chips each generation, vendors will just ship smaller, cheaper ones).
In the case of servers, of course, the amount of cleverness and fundamental CS development needed to make parallelism work is substantially lower, since, if you have an outfit with 10,000 apache instances, or 5,000 VMs or something, they will always be happy to have more cores per chip, since that means more apache instances for VMs per chip, which means fewer servers(or the same number of single/dual socket servers instead of much more expensive quad/octal socket servers) even if each instance/VM uses no parallelism at all, and just sits at one core = one instance.
Guy at company that does nothing but parallel processing says that parallel processing is the way to go.
Moore's law has to stop at some point. It's an exponential function after all. Currently we are at in the 10^6 range (2,000,000 or so), our lower estimates for atoms in the universe are 10^80.
(80 - 6) * (log(10)/log(2)) = 246.
So clearly we are going to reach some issues with this doubling thing in sometime in the next 246 more doubles...
PATENT THAT NOW!
I'm probably being overly pedantic about this, but of course the word "law" in "Moore's Law" is almost tongue-in-cheek. There's no comparison between a simple observation that some trend or another is exponential--most trends are over a limited period of time--and a physical "law." Moore is not the first person to plot an economic trend on semilog paper.
There isn't even any particular basis for calling Moore's Law anything more than an observation. New technologies will not automatically come into being in order to fulfill it. Perhaps you can call it an economic law--people will not bother to go through the disruption of buying a new computer unless it is 30% faster than the previous one, therefore successive product introductions will always be 30% faster, or something like that.
In contrast, something like "Conway's Law"--"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations"--may not be in the same category as Kepler's Laws, but it is more than an observation--it derives from an understanding of how people work in organizations.
Moore's Law is barely in the same category as Bode's Law, which says that "the radius of the orbit of planet #N is 0.4 + 0.3 * 2^(N-1) astronomical units, if you call the asteroid belt a planet, pretend that 2^-1 is 0, and, of course, forget Pluto, which we now do anyway."
"How to Do Nothing," kids activities, back in print!
Moore's Law isn't exactly "a law". It isn't like "the law of gravity" where it is a certain thing that can't be ignored*. It's more "Moore's Observation" or "Moore's General Suggestion" or "Moore's Prediction". Any of those are only fit for a finite time and are bound to end.
* Someone's bound to point out some weird branch of Physics that breaks whatever law I pick or says it is wrong, but hopefully gravity is quite safe!
Albert P. Carey, CEO of Frito-Lay warns consumers that the continuation of the Cheddar-Dorito law and the survival of humanity ultimately relies on zesty corn chips.
Wake me up when this NVIDIA's proposed solution doesn't double my electrical bill and set my computer on fire.
Dude, this is clearly some sense of the word "infinite" of which I haven't been previously aware. A couple things: 1) atoms -> electrons -> quarks is three levels, which is not exactly infinity. 2) I'm not sure if this is what you meant, but electrons are not made of quarks. They're truly elementary particles. 3) No one thinks there's anything below quarks - the Standard Model may have some issues, but no one seriously questions the elementary status of quarks. 4) you can't do anything with quarks anyway - practically speaking, you can't even see an individual quark. They're tightly bound to each other in the form of hadrons.
I think that in practice, we're going to run into problems before we even get to the level of atoms. Lithographic processes can only get you so far - we're already into the extreme ultraviolet, so to get smaller features we're going to have start getting into x-rays/gamma rays, which have rather unfortunate health and safety issues associated with them, not to mention the difficult engineering problems involved in generating tightly focused beams. And even if you can solve that problem, you have to deal with noise introduced by electrons just leaking from one lead to another. I think 246 doublings is way, way generous.
Well, no, Moore's Law was never passed by any legislative authority, no.
As for a scientific law, 'laws' in science are like version numbers in software:
There's no agreed-upon definition whatsoever, but for some reason, people still seem to attribute massive importance to them for some reason.
If anything a 'law' is a scientific statement that dates from the 18th or 19th century, more or less.
Hooke's law is an empirical approximation.
The Ideal Gas law is exact, but only as a theoretical limit.
Ohm's law is actually a definition (of resistance).
The Laws of Thermodynamics are (likely) the most fundamental properties of nature that we know of.
The only thing these have in common is that they're from before the 20th century, really.
Perhaps nVidia's chief scientist wrote his piece because nVidia wants its very niche CUDA/OpenCL computational offering to expand and become mainstream. There's a problem with that though.
The computational ecosystems that surround CPUs can't work with hidden, undocumented interfaces such as nVidia is used to producing for graphics. Compilers and related tools hit the user-mode hardware directly, while operating systems fully control every last register on CPUs at supervisor level. There is no room for nVidia's traditional GPU secrecy in this new computational area.
I rather doubt that the company is going to change its stance on openness, so Dr. Daly's statement opens up the parallel computing arena very nicely to its traditional rival ATI, which under AMD's ownership is now a strongly committed open-source company.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Nine processors can't render an image of a baby in one system clock tick, sonny.
He doesn't say that is should be done via the GPU.
He says Intel and AMD need to focus on Parallelism. This is true.
The GPU/CPU comment was driven by the author of the article. Clearly as an attempt to drum up some sort of flame war to drive hits to the article.
Now, I would assume part of his job is to figure out how to properly do that with GPUs; however at no place is he implying only Nvidia can do this and it can only be dong on the GPU.
The Kruger Dunning explains most post on
Obviously there's a conflict-of-interest here, but that doesn't mean the guy is necessarily wrong. It just means you should exercise skepticism and independent judgment.
In my independent judgment, I happen to agree with the guy. Clockspeeds have been stalled at ~ 3Ghz for nearly a decade now. There are only so many ways of getting more per clock cycle and radical parallelization is a good answer. Many research communities, such as fluid dynamics, are already performing real computational work on the GPU, and the entire industry is shifting towards a GPGPU paradigm. Programming languages are also being written to further take advantage of parallelization. In my humble opinion, we're approaching the where every computation that can be processed in parallel will be. For what's it's worth, I actually think both Intel and AMD/ATI are doing a much better job at this than Nvidia.
The CPU industry has been developing quad cores and releasing 8 cores. But a lot of my software can't take advantage of this.
We just bought the latest version of software from one company and found that it ran a lot slower than the earlier version. I happened to stick it on a VM with only one core and it worked a lot faster.
We talked about MATLAB yesterday not being able to do 64 bit integers, big deal. I was told that their Neural Network package doesn't have parallel processing capabilities. I was like you have got to be freaking kidding me. A $1000 NN package that doesn't support parallel processing.
Well, without the slowness of windows, we wouldn't need faster computers, so there'd be nothing driving innovation.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking