Limits to Moore's Law Launch New Computing Quests

tringtring alerts us to news that the National Science Foundation has requested $20 million in funding to work on "Science and Engineering Beyond Moore's Law." The PC World article goes on to say that the effort "would fund academic research on technologies, including carbon nanotubes, quantum computing and massively multicore computers, that could improve and replace current transistor technology." tringtring notes that quantum computing has received funding on its own lately, and work on multicore chips has intensified the hunt for parallel programming. Also, improvements are still being made to current transistor mechanics.

Comments

Is this necessary?

[nebaz]

I don't really think a prize is necessary for this technology. Unlike space travel, reearch in chip design have shown to be profitable at the commercial level, and there is also no government monopoly to stifle progress in this area. Whether or not a prize is offered, faster computers and better technology are what we as consumers expect in this area, and what we will pay for.

Re:Is this necessary?

[Eravnrekaree]

Capital investment is most important with things like this. The purpose of the prize however might not be that it is necessary, but rather to accelerate development, speed it up. The benefits of course of improving technology would be significant, more powerful, faster computers.

Re:Is this necessary?

[aztektum]

It isn't a $20mil prize, it's a budget request.

Re:Is this necessary?

[Mike1024]

I don't really think a prize is necessary for this technology.

Who said anything about a prize? The PC World article talks about 'funding for research', i.e. cash given to researchers to develop new technology.

Unlike space travel, reearch in chip design have shown to be profitable at the commercial level, [...] Whether or not a prize is offered, faster computers and better technology are what we as consumers expect in this area, and what we will pay for.

It's true that a lot of commercial effort goes into current chips and the improvement thereof, but there isn't much commercial effort going into areas like quantum computing because the potential rewards are a loooooong way off. Your money is much safer invested in designing a 32-core Core2ThirtyTwo to be made in 3 years, compared to quantum computing, a technology that faces substantial scalability roadblocks and that no-one knows how to design algorithms for.

Most of the current quantum computers which have been demonstrated rely on Nuclear magnetic resonance (NMR), but it is thought this technique will not scale well - it is believed less than 100 qubits would be possible. As of 2006, the largest quantum computer ever demonstrated was 12 qubits (making it capable of such tasks as quickly finding the prime factors of a number... as long as that number is less than 4096.

In summary, promising future technologies often make poor investments because they are (a) experimental and (b) a long way off. So some funding to make research possible wouldn't go amiss.

Just my $0.02.

"If you build it, they will come..."

[syncrotic]

...is a mentality that probably won't work here.

Intel sunk billions into the development of Itanium on the premise that if they make a VLIW architecture, compiler developers will find a way to automatically extract the parallelism necessary to make good use of it. A company with the size, resources, and engineering knowledge of Intel made the mistake of assuming that a fundamental shift in thinking could be driven by money and sheer desire, but it turns out that the problem is not just hard - that would make it solvable given sufficient effort and money - it's actually impossible. Those compiler advances never materialized; you can't draw blood from a stone.

The quest for parallelism in ordinary software might just be similar. Developing tools to make this automated and easy with low overhead is akin to putting a dozen smart people in a room and saying "think up the next big idea that will make me millions." Innovation doesn't work that way; it can't be forced... and money isn't going to make the impossible into the possible.

I think we'll see a move to eight and then maybe even sixteen cores on a consumer-level chip before we see things start going back in the other direction. This will necessary mean a slowdown in the development of processors as CPU manufacturers go back to wringing every last bit of single-threaded performance out of their designs.

Thoughts?

Re:"If you build it, they will come..."

[ZorbaTHut]

While I agree with some of your points, I disagree with your details. There's no proof that compilers can't be made smart enough for that - just because it didn't happen doesn't mean it couldn't. The biggest issues with Itanium were that it was incredibly slow on "non-native" code and incredibly expensive. There was no reason for anyone to buy one without having a compiler built specifically for it, and there was no reason for anyone to spent the effort to write a compiler for it without someone having bought one.

It's possible that if Itanium had been able to execute x64 or even x86 code at a competitive speed, we'd all be using IA-64 by now (or at least hoping that new programs were recompiled with it.)

Also, I don't actually think we'll have a shift back to single-threaded apps. The fact is that most programs run "fast enough" now, even single-threaded on quadcore systems. The ones that don't (mostly games and some professional software) are frequently relatively easy to multithread. I suspect most programs will stay single-threaded, and the ones that need maximum speed will become extremely multithreaded.

The real question is

[Anonymous Coward]

How much experience is this quest worth?

patents?

[Neffirithion]

say they do get these carbon tubing and other stuff that would massively accelerate the technology worlds... Would they have patents on them as well as the 20 million? If so why have the prize? you'll just have to licence the technology from them anyway, so who ever does will be dirt rich + 20 million in pocket... If there is a hole in my thinking... please point it out to me.

~Neff

Re:patents?

[Jasin Natael]

It's not a prize. It's funding; A budget. This is the older-than-dirt story of, "If you build it, they will come!" vs. "I have a 0.01% chance of succeeding if I try to build it, so who's going to feed my family in the 99.99% probable case that I fail?"

cost per computation / 3-D Chips

[fpgaprogrammer]

Moore's law is an observation about the cost per transistor in a circuit. Making faster computation is all about transistor density and the distance signals must travel. Even after the 2-D transistor density levels off, the race will be on to make cheaper 3-D chips using wafer-bonding methods, giving us a new dimension to increase density and thus speed up computation:
http://mtlweb.mit.edu/researchgroups/icsystems/3dcsg/

And we'll still see the same exponential benefits to GOPs/$ for a long time after 3-D transistor density maxes out. The economics that drive the exponential cost-per-computation trend are more related to volume of demand which offsets high fixed production costs and less related to our ability to actually cram more transistors on a chip.

Re:cost per computation / 3-D Chips

[mean pun]

Even if we keep getting exponential growth of transistors per dollar in the coming years, the question is what to do with them. Arranging them in useful circuits is increasingly difficult because at a certain point adding cache and execution units to a processor just isn't very helpful (hence multi-core). Adding more cores is also not going to help at some point. Moreover, power dissipation can't keep growing proportionally, which means that with increasing transistor counts each transistor will have to dissipate less, which means lowering the average number of switching events per transistor, and how are we going to arrange for that?

Re:cost per computation / 3-D Chips

[fpgaprogrammer]

the hard part is, of course, how do we program it; there are plenty of applications that benefit from parallelization (graphics processing, SDR, FEM). parallelization tends to offer equivalent throughput at a lower rate of switching. we need to review whether high frequency switching is really worth the power when you have trillions of transistors in a cubic centimeter. at todays price for 1 million gate FPGA, a 1 trillion gate FPGA array would cost about $10-20M, I expect this will be down by a factor of 1000 in under 10 years. operating at 100 MHz it would be hard to not have a petaflop of computation. lower frequency requirements lets us get creative with power. power density (temperature) is directly related to the number of switching capacitance in a region. lower frequency circuits and asynchronous circuits can reduce the effect of the most major sources of switching (clock). with adiabatic logic systems you can actually use charge pumping and charge recovery to eliminate capacitive loss during switching but these circuits operate slower. when you combine asynchronous and adiabatic logic you can actually use the REQ-ACK handshake as a charge pump to power on functional units. and if you really want high frequency switching, you'll need to remove thermal energy. one of the early uses of carbon nanotubes in circuits may be as thermal channels. it's also possible to create submersible circuitry using microfluidic ducts to cool wafer-stacked chips.

Re:cost per computation / 3-D Chips

[jelle]

3D stacking of wafers with transistors is not the solution:

If someone were to try it, they better get working on methods to cool those stacks of wafers well, and ways to make the wafers cheaper...

If you make a chip with a stack of, say 10 wafers, you've also had to diffuse 10 wafers, costing, well, the same ten chips of only one wafer... Diffusing doesn't magically get cheaper when you stack the wafers afterwards. I'm sure the 'wafer-bonding' costs some dough too.

And it generates the heat of 10 chips of one wafer...

Now how much does a quad-core single-wafer chip currently dissipate? Isn't that in the range of 90-150 Watts?

Ten of those?

How many Watts does a maker use to boil a pot of water? How much a clothes-iron?

And 18 months later 20 wafers, then 40, then 80?

You get the picture...

Re:cost per computation / 3-D Chips

[Eivind]

Actually, transistors can also become more effective, and have been for decades. If not, you'd be right: Doublt computing-power would mean double power-consumption which would mean double heat-production and spell heaps of trouble.

We're still -far- away from the theoretical limits though.

Flipping a -single- bit MUST consume atleast kT joules, where T is the temperature in Kelvin and k is the Bolzmann-constant of around 10^-23.

So if your cpu runs at 300K (cooling it more won't help because then you'll spend energy for that) you can flip a physical max of sligthly over 10^20 bits. Run at 10Ghz, aproximately 10^10 Hz and you can flip a maximum of 10^10 bits every clock-cycle.

Current CPUs don't come anywhere close to being that efficient. They flip perhaps a million transistors in a clocktick which is a factor of 10000 less than they COULD be doing with a single watt, and they spend not ONE watt but more like 50W to do that.

Still, the limits are visible: We can, theoretically, up cpu-efficiency by a factor of 100.000, but a factor of a million looks, imho, physically impossible.

This sounds like a lot, but consider that if we keep doubling every 2 years, we'll hit the hard wall in 20 - 30 years. And long before we get close to the hard wall we'll be in a pretty steep terrain.

Unless we go with reversible computing in which case all bets are off and entirely new limits apply.

Killer app?

[sakdoctor]

And what would be the killer app that needed all that extra power?
Moore's Law might be linear but who's to say that demand for processing power is also... ...scratch that, Microsoft just released a new operating system. The minimum spec is 640 quantum cores.

Re:Killer app?

[mangu]

And what would be the killer app that needed all that extra power?

The short answer is all the applications that run in these computers.

I can think of at least two applications that are often in the news: protein folding and physical simulations of continuous media, like weather and climate, aerodynamics, water, oil, and gas flow in porous rocks, etc.

But I think the future applications for personal supercomputers haven't been invented yet. We don't have the brains to predict what super-human artificial intelligence will be like.

Re:Killer app?

[master_p]

How about:

1) parallel search
2) accurate text translation
3) accurate human speech rendering
4) raytracing for 3d graphics
5) advanced physics in 3d applications
6) more dynamic programming languages
7) better video and audio decompression
8) much faster compression
9) ultra fast large WORD document repagination
etc

Moore's Law is bullshit.

[Xiph1980]

Moore just happened to make a prognosis that transistordensity would double every 2 years.
It just happened to work out that way. We're about to reach a point where current transistors won't cut in anymore. At such a point we'll either stagnate because we can't make a smaller process than 10 nanometer and we can't find a different functional tech, or we'll make an enormous jump in performance because we'll find something in a different field, be it optics or nano-tubing, that does make processors a lot faster.

Moore's law isn't a law, and should never have been called that way. It's merely a prognosis.
microprocessor technology is driven by the market. If the general consumer thinks their pc is fast enough, manufacturers will focus on energy-efficiency to sell more cpu's, and speed will start to be a secondary concern.

Re:Moore's Law is bullshit.

[Tablizer]

That does not make it "bullshit", but just less than a mathematical or economic certainty. It's an observation that has held up.

Breakin' da law, breakin' da law...

[Anonymous Coward]

So, is Moores Law a law or the quota the industry need to meet?

Officer : "Sir, I'll have to arrest you for breaking Moore's Law"

Intel exec : "Oh noes!"

Re:Moore's Law is really about price-performance

[billstewart]

Sure, Moore expressed his observations in terms of transistor density, and the speed factor of 1-2 years has varied a bit over the last few decades, but what it's really about is price-performance of technology in a positive-reinforcement market. If you want to sell more chips, you either have to make them faster or cheaper or both, unless you're the only player in an underserved market.

So the expensive fast chips get faster to sell to customers with the need for speed, and the production technology gets refined to make more chips cheaper at a given speed, so the currently-fast speeds get cheaper, and the currently-cheap chips get faster, but on the other hand you do spend more capital on each new generation of fab plant.

And as the chips get faster, the software makers use up the available speed, and as the software makes machines slower (but more useful, or more friendly, or more popular), the customers want faster chips or bigger memories or bigger disks or all of the above.

The big threats to Moore's Law right now aren't so much that we're running into the edge of silicon technology, but that Microsoft Vista is sufficiently unsatisfactory that people aren't buying it unless it ships on their new laptops, so there's less demand for faster machines, and also that gamers are playing more MMORPGs, where faster CPUs and graphics chips don't make as much difference in game capability as they do with standalone games (but even so, a cutting-edge graphics card costs more than a business-class desktop computer.)

On the other hand, virtualization (which is pretty much the reinvention of time-sharing) is pushing the business sector toward doing new and exciting technology for clustering storage, and at least creating some demand for RAM, and using up some of those multi-core CPUs even though they're buying fewer of them. And we're starting to hit environments where the cost of electricity for cooling and power exceeds the cost of the CPU itself, so price-performance is starting to get measured in watts/bogomips, rather than just dollars/bogomips.