Limits to Moore's Law Launch New Computing Quests

tringtring alerts us to news that the National Science Foundation has requested $20 million in funding to work on "Science and Engineering Beyond Moore's Law." The PC World article goes on to say that the effort "would fund academic research on technologies, including carbon nanotubes, quantum computing and massively multicore computers, that could improve and replace current transistor technology." tringtring notes that quantum computing has received funding on its own lately, and work on multicore chips has intensified the hunt for parallel programming. Also, improvements are still being made to current transistor mechanics.

Comments

"If you build it, they will come..."

[syncrotic]

...is a mentality that probably won't work here.

Intel sunk billions into the development of Itanium on the premise that if they make a VLIW architecture, compiler developers will find a way to automatically extract the parallelism necessary to make good use of it. A company with the size, resources, and engineering knowledge of Intel made the mistake of assuming that a fundamental shift in thinking could be driven by money and sheer desire, but it turns out that the problem is not just hard - that would make it solvable given sufficient effort and money - it's actually impossible. Those compiler advances never materialized; you can't draw blood from a stone.

The quest for parallelism in ordinary software might just be similar. Developing tools to make this automated and easy with low overhead is akin to putting a dozen smart people in a room and saying "think up the next big idea that will make me millions." Innovation doesn't work that way; it can't be forced... and money isn't going to make the impossible into the possible.

I think we'll see a move to eight and then maybe even sixteen cores on a consumer-level chip before we see things start going back in the other direction. This will necessary mean a slowdown in the development of processors as CPU manufacturers go back to wringing every last bit of single-threaded performance out of their designs.

Thoughts?

Re:"If you build it, they will come..."

[ZorbaTHut]

While I agree with some of your points, I disagree with your details. There's no proof that compilers can't be made smart enough for that - just because it didn't happen doesn't mean it couldn't. The biggest issues with Itanium were that it was incredibly slow on "non-native" code and incredibly expensive. There was no reason for anyone to buy one without having a compiler built specifically for it, and there was no reason for anyone to spent the effort to write a compiler for it without someone having bought one.

It's possible that if Itanium had been able to execute x64 or even x86 code at a competitive speed, we'd all be using IA-64 by now (or at least hoping that new programs were recompiled with it.)

Also, I don't actually think we'll have a shift back to single-threaded apps. The fact is that most programs run "fast enough" now, even single-threaded on quadcore systems. The ones that don't (mostly games and some professional software) are frequently relatively easy to multithread. I suspect most programs will stay single-threaded, and the ones that need maximum speed will become extremely multithreaded.

patents?

[Neffirithion]

say they do get these carbon tubing and other stuff that would massively accelerate the technology worlds... Would they have patents on them as well as the 20 million? If so why have the prize? you'll just have to licence the technology from them anyway, so who ever does will be dirt rich + 20 million in pocket... If there is a hole in my thinking... please point it out to me.

~Neff

cost per computation / 3-D Chips

[fpgaprogrammer]

Moore's law is an observation about the cost per transistor in a circuit. Making faster computation is all about transistor density and the distance signals must travel. Even after the 2-D transistor density levels off, the race will be on to make cheaper 3-D chips using wafer-bonding methods, giving us a new dimension to increase density and thus speed up computation:
http://mtlweb.mit.edu/researchgroups/icsystems/3dcsg/

And we'll still see the same exponential benefits to GOPs/$ for a long time after 3-D transistor density maxes out. The economics that drive the exponential cost-per-computation trend are more related to volume of demand which offsets high fixed production costs and less related to our ability to actually cram more transistors on a chip.

Re:cost per computation / 3-D Chips

[fpgaprogrammer]

the hard part is, of course, how do we program it; there are plenty of applications that benefit from parallelization (graphics processing, SDR, FEM). parallelization tends to offer equivalent throughput at a lower rate of switching. we need to review whether high frequency switching is really worth the power when you have trillions of transistors in a cubic centimeter. at todays price for 1 million gate FPGA, a 1 trillion gate FPGA array would cost about $10-20M, I expect this will be down by a factor of 1000 in under 10 years. operating at 100 MHz it would be hard to not have a petaflop of computation. lower frequency requirements lets us get creative with power. power density (temperature) is directly related to the number of switching capacitance in a region. lower frequency circuits and asynchronous circuits can reduce the effect of the most major sources of switching (clock). with adiabatic logic systems you can actually use charge pumping and charge recovery to eliminate capacitive loss during switching but these circuits operate slower. when you combine asynchronous and adiabatic logic you can actually use the REQ-ACK handshake as a charge pump to power on functional units. and if you really want high frequency switching, you'll need to remove thermal energy. one of the early uses of carbon nanotubes in circuits may be as thermal channels. it's also possible to create submersible circuitry using microfluidic ducts to cool wafer-stacked chips.

Re:Killer app?

[mangu]

And what would be the killer app that needed all that extra power?

The short answer is all the applications that run in these computers.

I can think of at least two applications that are often in the news: protein folding and physical simulations of continuous media, like weather and climate, aerodynamics, water, oil, and gas flow in porous rocks, etc.

But I think the future applications for personal supercomputers haven't been invented yet. We don't have the brains to predict what super-human artificial intelligence will be like.