Limits to Moore's Law Launch New Computing Quests

tringtring alerts us to news that the National Science Foundation has requested $20 million in funding to work on "Science and Engineering Beyond Moore's Law." The PC World article goes on to say that the effort "would fund academic research on technologies, including carbon nanotubes, quantum computing and massively multicore computers, that could improve and replace current transistor technology." tringtring notes that quantum computing has received funding on its own lately, and work on multicore chips has intensified the hunt for parallel programming. Also, improvements are still being made to current transistor mechanics.

Comments

"If you build it, they will come..."

[syncrotic]

...is a mentality that probably won't work here.

Intel sunk billions into the development of Itanium on the premise that if they make a VLIW architecture, compiler developers will find a way to automatically extract the parallelism necessary to make good use of it. A company with the size, resources, and engineering knowledge of Intel made the mistake of assuming that a fundamental shift in thinking could be driven by money and sheer desire, but it turns out that the problem is not just hard - that would make it solvable given sufficient effort and money - it's actually impossible. Those compiler advances never materialized; you can't draw blood from a stone.

The quest for parallelism in ordinary software might just be similar. Developing tools to make this automated and easy with low overhead is akin to putting a dozen smart people in a room and saying "think up the next big idea that will make me millions." Innovation doesn't work that way; it can't be forced... and money isn't going to make the impossible into the possible.

I think we'll see a move to eight and then maybe even sixteen cores on a consumer-level chip before we see things start going back in the other direction. This will necessary mean a slowdown in the development of processors as CPU manufacturers go back to wringing every last bit of single-threaded performance out of their designs.

Thoughts?

Re:"If you build it, they will come..."

[ZorbaTHut]

While I agree with some of your points, I disagree with your details. There's no proof that compilers can't be made smart enough for that - just because it didn't happen doesn't mean it couldn't. The biggest issues with Itanium were that it was incredibly slow on "non-native" code and incredibly expensive. There was no reason for anyone to buy one without having a compiler built specifically for it, and there was no reason for anyone to spent the effort to write a compiler for it without someone having bought one.

It's possible that if Itanium had been able to execute x64 or even x86 code at a competitive speed, we'd all be using IA-64 by now (or at least hoping that new programs were recompiled with it.)

Also, I don't actually think we'll have a shift back to single-threaded apps. The fact is that most programs run "fast enough" now, even single-threaded on quadcore systems. The ones that don't (mostly games and some professional software) are frequently relatively easy to multithread. I suspect most programs will stay single-threaded, and the ones that need maximum speed will become extremely multithreaded.

cost per computation / 3-D Chips

[fpgaprogrammer]

Moore's law is an observation about the cost per transistor in a circuit. Making faster computation is all about transistor density and the distance signals must travel. Even after the 2-D transistor density levels off, the race will be on to make cheaper 3-D chips using wafer-bonding methods, giving us a new dimension to increase density and thus speed up computation:
http://mtlweb.mit.edu/researchgroups/icsystems/3dcsg/

And we'll still see the same exponential benefits to GOPs/$ for a long time after 3-D transistor density maxes out. The economics that drive the exponential cost-per-computation trend are more related to volume of demand which offsets high fixed production costs and less related to our ability to actually cram more transistors on a chip.

Re:cost per computation / 3-D Chips

[fpgaprogrammer]

the hard part is, of course, how do we program it; there are plenty of applications that benefit from parallelization (graphics processing, SDR, FEM). parallelization tends to offer equivalent throughput at a lower rate of switching. we need to review whether high frequency switching is really worth the power when you have trillions of transistors in a cubic centimeter. at todays price for 1 million gate FPGA, a 1 trillion gate FPGA array would cost about $10-20M, I expect this will be down by a factor of 1000 in under 10 years. operating at 100 MHz it would be hard to not have a petaflop of computation. lower frequency requirements lets us get creative with power. power density (temperature) is directly related to the number of switching capacitance in a region. lower frequency circuits and asynchronous circuits can reduce the effect of the most major sources of switching (clock). with adiabatic logic systems you can actually use charge pumping and charge recovery to eliminate capacitive loss during switching but these circuits operate slower. when you combine asynchronous and adiabatic logic you can actually use the REQ-ACK handshake as a charge pump to power on functional units. and if you really want high frequency switching, you'll need to remove thermal energy. one of the early uses of carbon nanotubes in circuits may be as thermal channels. it's also possible to create submersible circuitry using microfluidic ducts to cool wafer-stacked chips.

Killer app?

[sakdoctor]

And what would be the killer app that needed all that extra power?
Moore's Law might be linear but who's to say that demand for processing power is also... ...scratch that, Microsoft just released a new operating system. The minimum spec is 640 quantum cores.

Re:Is this necessary?

[Mike1024]

I don't really think a prize is necessary for this technology.

Who said anything about a prize? The PC World article talks about 'funding for research', i.e. cash given to researchers to develop new technology.

Unlike space travel, reearch in chip design have shown to be profitable at the commercial level, [...] Whether or not a prize is offered, faster computers and better technology are what we as consumers expect in this area, and what we will pay for.

It's true that a lot of commercial effort goes into current chips and the improvement thereof, but there isn't much commercial effort going into areas like quantum computing because the potential rewards are a loooooong way off. Your money is much safer invested in designing a 32-core Core2ThirtyTwo to be made in 3 years, compared to quantum computing, a technology that faces substantial scalability roadblocks and that no-one knows how to design algorithms for.

Most of the current quantum computers which have been demonstrated rely on Nuclear magnetic resonance (NMR), but it is thought this technique will not scale well - it is believed less than 100 qubits would be possible. As of 2006, the largest quantum computer ever demonstrated was 12 qubits (making it capable of such tasks as quickly finding the prime factors of a number... as long as that number is less than 4096.

In summary, promising future technologies often make poor investments because they are (a) experimental and (b) a long way off. So some funding to make research possible wouldn't go amiss.

Just my $0.02.

Moore's Law is bullshit.

[Xiph1980]

Moore just happened to make a prognosis that transistordensity would double every 2 years.
It just happened to work out that way. We're about to reach a point where current transistors won't cut in anymore. At such a point we'll either stagnate because we can't make a smaller process than 10 nanometer and we can't find a different functional tech, or we'll make an enormous jump in performance because we'll find something in a different field, be it optics or nano-tubing, that does make processors a lot faster.

Moore's law isn't a law, and should never have been called that way. It's merely a prognosis.
microprocessor technology is driven by the market. If the general consumer thinks their pc is fast enough, manufacturers will focus on energy-efficiency to sell more cpu's, and speed will start to be a secondary concern.