Slashdot Mirror
Moore's Law Will Die Without GPUs
Stoobalou writes "Nvidia's chief scientist, Bill Daly, has warned that the long-established Moore's Law is in danger of joining phlogiston theory on the list of superseded laws, unless the CPU business embraces parallel processing on a much broader scale."
Moore's is not a law, but an observation!
I didn't realise Moore's Law was purely the driving force behind CPU development and not just an observation on semiconductor development. Surely we just say Moore's Law held until a certain point, then someone else's Law takes over?
As for Phlogiston theory - it was just that, a theory which was debunked.
Dr. Daly believes the only way to continue to make great strides in computing performance is to ... offload some of the work onto GPU's that his company just happens to make? [Arte Johnson] Very interesting .
The industry has moved away from "more horsepower than you'll ever need!" to "uses less power than you can ever imagine!" Perpetuating Moore's Law isn't an industry requirement, it's a prediction by a guy who was in the chip industry.
So, a graphics card manufacturer says that graphics cards are the future? And this is news?
But the only "law" is that the number of transistors doubles in a certain time (something of a self fulfilling prophesy these days since this is the yardstick the chip companies work to).
Once transistors get below a certain size, of course it will end. Parallel or serial doesn't change things. We either have more processors in the same space, more complex processors or simply smaller processors. There's no "saving" to be done.
considering that Moore's Law was based on the observation that they were able to double the number of transistors about every 20 months, it would be inevitable that at some point they reach a limiting factor. The factor seems to be the process size, which is a physical barrier. As the process size continues to decrease, the physical size of atoms is a barrier that they can't get past.
I don't know if you're trying to be funny or not with this post, so you earned a FAIL.
Obviously "NVIDIA's Chief Scientist" is going to say something about the epochal importance of GPUs; but WTF?
Moore's law, depending on the exact formulation you go with, posits either that transistor density will double roughly every two years or that density at minimum cost/transistor increases at roughly that rate.
It is pretty much exclusively a prediction concerning IC fabrication(a business that NVIDIA isn't even in, TSMC handles all of their actual fabbing), without any reference to what those transistors are used for.
Now, it is true that, unless parallel processing can be made to work usefully on a general basis, Moore's law will stop implying more powerful chips, and just start implying cheaper ones(since, if the limits of effective parallel processing mean that you get basically no performance improvements going from X billion transistors to 2X billion transistors, Moore's law will continue; but instead of shipping faster chips each generation, vendors will just ship smaller, cheaper ones).
In the case of servers, of course, the amount of cleverness and fundamental CS development needed to make parallelism work is substantially lower, since, if you have an outfit with 10,000 apache instances, or 5,000 VMs or something, they will always be happy to have more cores per chip, since that means more apache instances for VMs per chip, which means fewer servers(or the same number of single/dual socket servers instead of much more expensive quad/octal socket servers) even if each instance/VM uses no parallelism at all, and just sits at one core = one instance.
Guy at company that does nothing but parallel processing says that parallel processing is the way to go.
Moore's law has to stop at some point. It's an exponential function after all. Currently we are at in the 10^6 range (2,000,000 or so), our lower estimates for atoms in the universe are 10^80.
(80 - 6) * (log(10)/log(2)) = 246.
So clearly we are going to reach some issues with this doubling thing in sometime in the next 246 more doubles...
Sometimes I think that parallel programming isn't a "new challenge" but rather something that people do every day with VHDL and Verilog...
(Insert your own musings about FPGAs and CPUs and the various tradeoffs to be made between these two extremes.)
If they were smart, they'd make processors that could execute multiple threads in parallel. Personally, I'd use a flashy name such as "multi-core" or something.
Maybe it's time to amend Moore's Law to also account for a Wattage divisor. Same computing power for half the battery drain would be good...
Come on, that was funny! The poor soul obviously knew you moderators have been out for blood, oh, these last many years or so.
It's a rule, not a law. Moore's rule is based on certain assumptions, and we have push to the point where some of the assumptions are no longer valid. The limiting factor at the time of Moore was the feature size. Halving the size and scaling other parameters accordingly would result in a doubling of the switching time. However, we hit a limit with the gate oxide. The gate oxide need to be halved. Eventually you get low on the number of atoms in the gate layer. For example, when you're down to 3 atoms, you cannot divide by 2. The solution to this problem was Hf dioxide. This was also the cause/solution to the increasing static power draw. The next problem will be in the band gaps. When you shrink the width of the transistor/switch there are less and less atoms (naturally). However, you eventually get to a point where the band gap is no longer well defined. Each atoms has its own relative band gap (in a bulk crystal). The summation of atoms give a clear band gap, but there is a minimum number of atoms that create this scenario. I don't remember exactly, but 100 atoms seems to be in the back of my mind (100 could be wrong). IMHO, it is time to take parallel processing seriously.
Seriously. The headline for this should read "Moore's Law will Die Without GPUs, Says GPU Maker ."
Or, to put it another way, the GPU maker keeps invoking Moore's Law, but I do not think it means what he thinks it means. You can't double semiconductor density by increasing the number of chips involved.
I'm probably being overly pedantic about this, but of course the word "law" in "Moore's Law" is almost tongue-in-cheek. There's no comparison between a simple observation that some trend or another is exponential--most trends are over a limited period of time--and a physical "law." Moore is not the first person to plot an economic trend on semilog paper.
There isn't even any particular basis for calling Moore's Law anything more than an observation. New technologies will not automatically come into being in order to fulfill it. Perhaps you can call it an economic law--people will not bother to go through the disruption of buying a new computer unless it is 30% faster than the previous one, therefore successive product introductions will always be 30% faster, or something like that.
In contrast, something like "Conway's Law"--"organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations"--may not be in the same category as Kepler's Laws, but it is more than an observation--it derives from an understanding of how people work in organizations.
Moore's Law is barely in the same category as Bode's Law, which says that "the radius of the orbit of planet #N is 0.4 + 0.3 * 2^(N-1) astronomical units, if you call the asteroid belt a planet, pretend that 2^-1 is 0, and, of course, forget Pluto, which we now do anyway."
"How to Do Nothing," kids activities, back in print!
Of course, Nvidia says that GPUs are the answer. But then again, as a scientific software developer, I completely agree with him. In fact, I'm in the middle of a grant proposal right now to purchase a series of GPUs to complement our CPU based cluster.
Moore's Law isn't exactly "a law". It isn't like "the law of gravity" where it is a certain thing that can't be ignored*. It's more "Moore's Observation" or "Moore's General Suggestion" or "Moore's Prediction". Any of those are only fit for a finite time and are bound to end.
* Someone's bound to point out some weird branch of Physics that breaks whatever law I pick or says it is wrong, but hopefully gravity is quite safe!
It would mean that development cycles slow down, algorithmics finally win over brute force and that software quality would have a chance to improve (after going downhill for a long time).
GPUs as CPUs? Ridiculous! Practically nobody can program them and very few problems benefit from them. This sounds more like Nvidia desperately trying to market their (now substandard) products.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
What the hell does Moore's Law have to do with parallel computing?
It is concerned with the number of transistors on a single chip. Moore's Law has been dead in practical terms for a while (smaller transistors are too expensive / require too much power and cooling), which is the reason parallel computing is becoming essential in the first place.
TFA fails computer science history forever.
A GPU maker has "warned" us that the rapid pace of obsolescence will decline if we don't use more GPUs? Amazing.
So, how much are they paying you?
Aside from being a bit of a hack, there are 3 competing APIs and some of them are tied to certain combinations of operating system & hardware. I wonder why AMD or Intel haven't produced something analogous to the Cell - a mainstream multicore CPU that also contains a bunch of SPUs purpose built to blaze through data as fast as possible.
Albert P. Carey, CEO of Frito-Lay warns consumers that the continuation of the Cheddar-Dorito law and the survival of humanity ultimately relies on zesty corn chips.
Yes, we need more parallelism. But parallelism does not have to be implemented via GPUs - though a man from NVidia would say so. GPUs provide a quick-and-dirty form of parallelism which is easy to kludge onto the current PC architecture. Which, given the way the appalling x86 architecture has succeeded by pure kludgy brute force, may be the way we end up going. But actually we would be much better with an architecture that inherently supports parallelism, such as Functional Programming. I don't known whether we can overcome the inertia of the huge corporations committed to the current way of doing things, but there are much better ways. A homogeneous system is easier to use than a heterogeneous one, to start with.
Consciousness is an illusion caused by an excess of self consciousness.
if ( story.contains('Moore\'s') and story.contains('die','dead','end in'):
story.comment('Moore\'s Law is an observation not a law! and.... IT WILL NEVER DIE!!!')
The article ends with "Interestingly, though, only one company has the technology and IP needed to integrate a highly parallel GPU into a CPU... and that’s AMD." Although I like AMD and would surely like to see them getting a revolutionary "fusion" product out before anyone else, one has to ask whether the authors have looked under the hood of Intel's Clarkdale and Arrandale core i5... This shows Intel's rapidly catching up, and a neck-to-neck race may arise between their Sandy Bridge and AMD's Bulldozer. Not to mention the stubborn rumors that Nvidia's itself is developing x86 technology...
Here's some background for those of us that have been living in a cave:
http://www.brightsideofnews.com/news/2009/4/15/amds-next-gen-bulldozer-is-a-128-bit-crunching-monster.aspx
http://www.tomshardware.com/reviews/future-3d-graphics,2560-9.html
Moore's Law was an observation of a trend made in 1965 that transistor counts on an integrated circuit had doubled and redoubled over a short period of time, and would continue to do so for at least another ten years (the fact that it has done so for half a century is possibly more than Moore could have hoped for). It was based on observed data that was beyond doubt. Phlogiston Theory was not a theory in the primary definition of the word (from the Greek theria meaning observed, the analysis of a set of facts in their relation to one another). It was more a hypothesis assumed for the sake of argument or (what we now know as scientific) investigation, written as a small update to alchemy (earth, air, fire, water). Not to start an off-topic flame war, but the two are analogous to Evolution Theory and Creationism Theory.
Shiny. Let's be bad guys...
With Moore's Law broken, what will stand in the way of the Robot Apocalypse?
It doesnt suprise me since hired a bunch of ex-Transmeta engineers to work for them last year. They are more than likely working on running the GPU with a bios to boot to whatever instruction set they want on the GPU. That would completely negate Moores Law since packing cores on a chip would directly effect performance.
Wake me up when this NVIDIA's proposed solution doesn't double my electrical bill and set my computer on fire.
seriously, in the last few years Intel has produced some good CPU's with good power efficiency. contrast that with Nvidia where every generation you need more and more power to power their cards and the latest generation is something like 250W of heat. years ago we used a compaq all in one cluster server at work as a space heater, the way nvidia is going all you need to do is buy one of their cards and you can heat your house in the winter and not buy heating oil
Moore's Law works until the required elements are smaller than quantum objects. Actually, in our current state of technology and anything practical on the horizon, it works until the required elements are smaller than single atoms. Then there is no way to make stuff faster...
Sort of.
While GPUs might 'save Moore's Law', actually they just add other CPUs to each system. So more cores = more performance, and Moore's Law is still relevant.
Now, to change the entire computing paradigm to actually take advantage of parallel processing. GPUs do it, sort of, but we need entirely new operating systems, and probably new physical architecture.
Not that it matters that much to me. I'll just buy whatever is reasonably fast, and leave the bleading edge to those with more money to spend.
deleting the extra space after periods so i can stay relevant, yeah.
Dude, this is clearly some sense of the word "infinite" of which I haven't been previously aware. A couple things: 1) atoms -> electrons -> quarks is three levels, which is not exactly infinity. 2) I'm not sure if this is what you meant, but electrons are not made of quarks. They're truly elementary particles. 3) No one thinks there's anything below quarks - the Standard Model may have some issues, but no one seriously questions the elementary status of quarks. 4) you can't do anything with quarks anyway - practically speaking, you can't even see an individual quark. They're tightly bound to each other in the form of hadrons.
I think that in practice, we're going to run into problems before we even get to the level of atoms. Lithographic processes can only get you so far - we're already into the extreme ultraviolet, so to get smaller features we're going to have start getting into x-rays/gamma rays, which have rather unfortunate health and safety issues associated with them, not to mention the difficult engineering problems involved in generating tightly focused beams. And even if you can solve that problem, you have to deal with noise introduced by electrons just leaking from one lead to another. I think 246 doublings is way, way generous.
Moore's law will 'die'.
Well, no, Moore's Law was never passed by any legislative authority, no.
As for a scientific law, 'laws' in science are like version numbers in software:
There's no agreed-upon definition whatsoever, but for some reason, people still seem to attribute massive importance to them for some reason.
If anything a 'law' is a scientific statement that dates from the 18th or 19th century, more or less.
Hooke's law is an empirical approximation.
The Ideal Gas law is exact, but only as a theoretical limit.
Ohm's law is actually a definition (of resistance).
The Laws of Thermodynamics are (likely) the most fundamental properties of nature that we know of.
The only thing these have in common is that they're from before the 20th century, really.
though I rarely see the usual mistakes being made by the slashdot community.
I tried explaining to a friend of mine why it was that, in 2004 his standard desktop configuration had a CPU clocking at 2Ghz, and the standard configuration of the machines available last christmas had CPU's clocking in at 2.4Ghz (in the same price range). He seemed to think it would be in the 8-10Ghz range by now.
Perhaps nVidia's chief scientist wrote his piece because nVidia wants its very niche CUDA/OpenCL computational offering to expand and become mainstream. There's a problem with that though.
The computational ecosystems that surround CPUs can't work with hidden, undocumented interfaces such as nVidia is used to producing for graphics. Compilers and related tools hit the user-mode hardware directly, while operating systems fully control every last register on CPUs at supervisor level. There is no room for nVidia's traditional GPU secrecy in this new computational area.
I rather doubt that the company is going to change its stance on openness, so Dr. Daly's statement opens up the parallel computing arena very nicely to its traditional rival ATI, which under AMD's ownership is now a strongly committed open-source company.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
for not knowing what Moore's law is.
"ntel’s co-founder Gordon Moore predicted that the number of transistors on a processor would double every year, and later revised this to every 18 months.
well, thats half of the rule.
The Kruger Dunning explains most post on
Sure, you can add more transistors. And you can use those transistors to add more cores. But how useful will they be? That's what Amdahl's Law tells you. And Amdahl's Law is harder to break than Moore's.
GPUs only add one more dimension to Amdahl's Law: what portion of the parallelizable portion of a problem is amenable to SIMD processing on a GPU, as opposed to MIMD processing on a standard multi-core processor.
(T>t && O(n)--) == sqrt(666)
because software developers have the whole parallel software thing figured out perfectly.
Never say never. Ah!! I did it again!
Moore's law is describing the human abilities to make better processes leading to better miniaturization, leading to more precise printing of higher density transistors on smaller spaces. It is not a law that concerns natural processes, obviously -- And although it does hold true for now, it is bound to reach an end of life.
Moore's law will not be debunked, but we will surely go past it sooner or later. We cannot keep shrinking transistor size forever, as molecules and atoms give us an absolute minimum size, and upon reaching it, no law will replace Moore's - That will be it.
Nine processors can't render an image of a baby in one system clock tick, sonny.
Comment removed based on user account deletion
GO FUCK YOURSELF!
I love it, the article is about the fact that the number of transistors is out of control. So the solution is not to reduce the number of needed transistors but to make the system parallel. Sure CUDA is a better solution than simply adding more cores. However, this does not address the issue. We should look for ways to reduce the number of transistors. Maybe reassess the mathematics behind the transistors and discuss new options such as revamping the processor (like what mac did with moving to Unix).
Moore's law will not be a law, but a crusty theory that has been proven wrong. The pragmatic theory of truth will uphold and a new theory/law to take its place and we shall accept it.
"The laws of science be a harsh mistress." --Bender
Obviously there's a conflict-of-interest here, but that doesn't mean the guy is necessarily wrong. It just means you should exercise skepticism and independent judgment.
In my independent judgment, I happen to agree with the guy. Clockspeeds have been stalled at ~ 3Ghz for nearly a decade now. There are only so many ways of getting more per clock cycle and radical parallelization is a good answer. Many research communities, such as fluid dynamics, are already performing real computational work on the GPU, and the entire industry is shifting towards a GPGPU paradigm. Programming languages are also being written to further take advantage of parallelization. In my humble opinion, we're approaching the where every computation that can be processed in parallel will be. For what's it's worth, I actually think both Intel and AMD/ATI are doing a much better job at this than Nvidia.
Do not confuse laws of science and physics with those ecomonics and other fields. Murphy's Law trumps all others.
The US government have made it clear that we have no inalienable rights; any we do not defend vigorously will be taken.
Any comment made by nVidia these days (and usually in the past) is nothing more than 100% marketing. Worthless.
In related news, Issac Newton warns that Gravity will cease to exist unless physicists use his version of Calculus and not Leibniz's version. The supreme court rules that corporations CAN patents laws of the physical universe and withhold them from the general public if it benefits their profit motive, but are unsure if String Theory applies. Adam smith promptly claims full ownership of the invisible hand and uses it to bitch slap people who don't understand the difference between a physical law, a social law, and an interesting pop-science-y observation. More at 11.
The CPU industry has been developing quad cores and releasing 8 cores. But a lot of my software can't take advantage of this.
We just bought the latest version of software from one company and found that it ran a lot slower than the earlier version. I happened to stick it on a VM with only one core and it worked a lot faster.
We talked about MATLAB yesterday not being able to do 64 bit integers, big deal. I was told that their Neural Network package doesn't have parallel processing capabilities. I was like you have got to be freaking kidding me. A $1000 NN package that doesn't support parallel processing.
Excuse me moron, but how is doubling the transister count every two years in any way related to the Windows OS?
Here be signatures
Dude, you need to read up on physics. This is not a technological issue - it's not just that we need to smash stuff together harder. The Standard model predicts that quarks are it... they are true elementary particles. I'm not sure where you're getting this "turtles all the way down" thing.
Who need Moore's Law. The new hip things like the ipad work on remarkably feeble processors. (so feeble, it can't run Flash)
Well, without the slowness of windows, we wouldn't need faster computers, so there'd be nothing driving innovation.
"Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
Perhaps the OP thinks that Microsoft is particularly bad at exploiting multiple CPU environments.
It's pretty mundane to have a 64 core Unix machine.
There's a lot less history of that with PCs and Microsoft probably needs to play catch up again.
A Pirate and a Puritan look the same on a balance sheet.
He's saying that CPUs need to be massively parallel, like GPUs, to remain relevant.
He also contends that the current design of CPUs are too inefficient to compete with GPUs.
My question is: why? Programming for the GPU is tons harder than programming for multiple cores. The GPU is so efficient because it is so limited. If we were to migrate to GPUs, they would have to gain more general functionality, and lose some of their efficiency.
What's wrong with going the other way and start with a general purpose processor and make it more parallel, which surprise surprise we're already doing?
NVidia will die if it doesn't provide free software drivers to its customers.
Obviously Moore's law was doomed from the start.
Their are physical and logical reasons why it cannot continue.
one being the size of an atom, while parallel processing will help with this physical size and mass will increase.
And with the nature of Moor's law this would mean the size and weight of computers doubling every year.
another is usefulness, eventually if Moore's law was to continue, we would get to a point when we would not need any more processing power.
Troll is not a replacement for I disagree.
Given, that his law is exponential, that the universe is finite, and that the smallest possible computer to simulate the universe is the universe itself (uogac's law), Moore's prediction will flatline at some point in the future.
Actually, the word "law" in Moore's Law is very closely in agreement with the way that word is generally used in science and engineering: an observed relationship.
Examples:
Kepler's laws of planetary motion
Newton's laws of motion
The ideal gas law
The laws of thermodynamics
Hooke's law
All or just statements of observed relationships that appeared to be true at the time. Some are idealized, some are approximate, some aren't actually true. None actually explain why the observed phenomenon should be, nor do they make statements on the universality or duration of the relationship.
In 1980 8 bit computing became the norm, in 1984 16 bit computing became the norm, in 1988 32 bit computing became the norm, and then complete stupidity became the norm. AMD actually tried to rectify the situation more recently by making 64 bit computing the norm. So the industry has turned it's back on what works, tried upping core speed, didn't work, tried adding cores didn't work, what now?... parallel processing, hmm, I wonder how that's going to pan out? A 128 bit PC, how on earth would that double the performance over a 64 bit PC?
I prefer ecostereoics myself.
You are not a brain: http://books.google.com/books?id=2oV61CeDx-YC
I knew my user name would be relevant some day!
The reason GPUs are so fast at raw number crunching is the same reason that they are bad at general purpose CPU work: they delete everything from the die except what they need for doing the same instruction on many cores at once.
They don't handle random memory access, they don't branch effeciently, and most importantly, they don't do any of this for a single core: a single core executes the exact same instruction stream as 16 or more of its neighbors.
The result is much less cache, and much less instruction logic, which means many more cores per GPU. But all that extra logic is exactly what makes a CPU good at doing what it does. Running apache on a GPU?? Forget it.
Now there are certainly some interesting and esoteric things that we can do with processor and software design to make a massively parallel apache server possible, but turning the CPU into a GPU isn't one of them.
Those who do not learn from history are doomed to repeat it. The first age of parallel processing came about because individual processors where too slow so the thought was just through a bunch of cheap micros at the problem and you will get the performance of a Supercomputer of the day. Moore's Law and the killer micros offered a cheaper path to higher performance.
Today the reasons for parallel processing are different but the need is still there. Some basic Parallel Processing info.
Lots of different types of parallelism was tried but two seemyd to work reasonably well for certain types of problems.
Data parallelism, where there is lots of data and the same operations are being performed. This is also known as fine grain parallelism. The machine architecture which best fits this type of parallelism is Single Instruction Multiple Data (SIMD). GPUs are a great example of this.
The second type of parallelism is task parallelism. This is also know as course grain parallelism. The machine architecture best suit to this approach is Multiple Instructions Multiple Data (MIMD). Multicore X86 CPUs are a good example of this.
Trying to run data parallel problems on a MIMD machine cause most of the time to be spent syncronizing data instead of operating on the data. It is also very difficult to get the code right in full async implementations.
Data parallel (GPU) style of parallelism has a great deal of potental but requires a different programming approach. More like SQL the C++. You specify what actions will be taken on the data, rather then how those actions will be performed.
As long as we are talking about laws and parallelism, we should also at least mention Amdahl's law, but that's for another post.
Just my USD 0.02 worth.