DARPA Looks Beyond Moore's Law

ddtstudio writes "DARPA (the folks who brought you the Internet) is, according to eWeek, looking more than ten years down the road when, they say, chip makers are going to have to have totally new chip fabrication technologies. Quantum gates? Indium Phosphide? Let's keep in mind that Moore's Law was more an observation than a predictive law of nature, despite how people treat it that way."

Comments

Stacked chips

[temojen]

perhaps stacked wafers with vertical interconnects might help... I'm not sure how you'd dissipate the heat, though.

What about diamonds?

[GreenCrackBaby]

This diamond article in Wired 'http://www.wired.com/wired/archive/11.09/diamond. html' seems to indicate that Moore's law is sustainable for much more than ten more years.

Besides, I've been hearing about the death of Moore's Law for the last ten years.

Diamonds are no longer a girl's best friend

[beacher]

I saw this article about new diamond manufacturing techniques and it's an interesting read. Having diamond based processors looks like a viable technology in the near future and heat dissipation is one of the major reasons that they're considering diamond.

I'm just worried about what my wife will say when the diamond in my machine is bigger than the one on her finger...
-B

Parallel Computing

[lawpoop]

When we absolutely cannot put anymore transistors on a chip, we will start making computers that are massively parrallel. In the future, you will have a desktop computer that will have 2, 4, 8, 16, etc chips on them.

All these other things they are talking about are vaporware. Parallel computing is here and in use now.

Re:Parallel Computing

[Abcd1234]

Sounds like a nice idea for the desktop or for certain classes of research, but there will always be a place for massive computational capacity on a single chip since there is a large class of computing problems which are not easily parallelizable, and hence can not take advantage of parallel computing.

Incidentally, there is also a limit to how fast your parallel computer will get... it's call the bus. If you can't build high speed interconnects, or if your software isn't designed well (not as easy as it sounds!), you will inevitably have problems with the system bus becoming overtaxed. Heck, this is already a problem in our primarily single-CPU world.

Re:Parallel Computing

[HiThere]

Yes, but the current parallel computers have huge performance costs...they can easily spend over half their time coordinating themselves on many kinds of problems. Of course, on well structured problems there probably isn't a better solution possible.

Two major answers occur to me:

Answer one is that we figure out how to automatically decompose problems into independently solvable threads.. a quite difficult problem.

Answer two is that we build special purpose parallel processors to handle parallelizable tasks efficiently (sound processors, visual cortexes, etc.) and use them as/from demons to maintain environmental awareness. Then we divvy everything we can into separate threads. Dedicate one CPU to coordinating between processes (with possibly hot-switchable backups). And do the best we can.

Answer two seems to be "sort of" the approach that biological design takes...at least in mammals.

Get rid of C!

[Temporal]

Not many people know it, but one of the problems holding back processor technology today is the way programming languages are designed. Languages like C (or C++, Java, Perl, Python, Fortran, etc.) are inherently serial in nature. That is, they are composed of instructions which must be performed in sequence. However, the best way to improve the speed of processors is to increase parallelization; that is, make them do multiple things at once. And, no, threading isn't the answer -- threading too large-scale, and can only usefully extend to 2-4 parallel processes before most software has trouble taking advantage of it.

Think about this: Why is video graphics hardware so much faster than CPU's? You might say that it is because the video card is specifically designed for one task... however, these days, that isn't really true. Modern video cards allow you to write small -- but arbitrary -- programs which are run on every vertex or every pixel as they are being rendered. They aren't quite as flexible as the CPU, but they are getting close; the newest cards allow for branching and control flow, and they are only getting more flexible. So, why are they so much faster? There are a lot of reasons, but a big one is that they can do lots of things at the same time. The card can easily process many vertices or pixels in parallel.

Now, getting back to C... A program in C is supposed to be executed in order. A good compiler can break that rule in some cases, but it is harder than you would think. Take this simple example:

void increment(int* out, int* in, int count)
{
for(int i = 0; i < count; i++)
out[i] = in[i] + 1;
}

This is just a piece of C code which takes a list of numbers and produces another list by adding one to each number.

Now, even with current, mostly-serial CPU's, the fastest way to perform this loop is to process several numbers at once, so that the CPU can work on incrementing some of the numbers while it waits for the next ones to load from RAM. For highly-parallel CPU's (such as many currenty in development), you would even more so want to work on several numbers simultaneously.

Unfortunately, because of the way C is designed, the compiler can not apply such optimizations! The problem is, the compiler does not know if the "out" list overlaps with the "in" list. If it does, then the compiler has to do the assignments one-at-a-time to insure proper execution. Imagine the following code that calls the function, for example:

increment(myArray + 1, myArray, count);

Of course, using the function in such a way would not be very useful, but the compiler has to allow for it. This problem is called "aliasing".

ISO C99 provides for a "restrict" keyword which can help prevent this problem, but few people understand it, even fewer use it, and those who do use it usually don't use it everywhere (using it everywhere would be too much work). It's not a very good solution anyway -- more of a "hack" if you ask me.

Anyway, to sum it up, C generally requires the CPU to do things in sequence. As a result, CPU manufacturers are forced to make CPU's that do one thing at a time really, really fast, rather than lots of things at the same time. And, so, since it is so much harder to design a fast CPU, we end up with slower CPU's... and we hit the limits of "Moore's Law" far earlier than we should.

In contrast, functional languages (such as ML, Haskell, Ocaml, and, to a lesser extent, LISP), due to the way they work, have no concept of "aliasing". And, despite what many experienced C programmers would expect, functional languages can be amazingly fast, despite being rather high-level. Functional languages are simply easier to optimize. Unfortunately, experienced C/C++/Java/whatever programmers tend to balk at functional languages at first, as learning them can be like learning to program all over again...

So, yeah. I recommend you guy

Re:Moore's law is already ending

[Elladan]

Well, except for games.
And anything that uses 3D.

Games and 3D make heavy use of FPU, but it's interesting to note that as time goes on, more and more of the heavy lifting FP work is being offloaded to the graphics processor.

Given a few more generations, most of the FPU work in todays games may actually be executed in the GPU.

Of course, this doesn't actually change anything, since tomorrow's games will just put that much more load on the CPU for physics processing and such!

And audio/video playback and work.

Video codecs are essentially all integer based. Audio codecs often use the FPU, but they really don't need to - fixed point implementations tend to be just as fast.

And image editing.

The vast bulk of image editing work tends to be integer-based, or easily convertible to integer-based.

And some spreadsheets.

Spreadsheet math calculations aren't really performance-related in any sense. 99.9% (remember, your statistics may be made up on the spot, but mine are based on sound scientific handwaving!) of the time a spreadsheet spends is in fiddling with the GUI, which is primarily an integer operation activity.

That said, the parent poster's point sort of goes both ways. It's true that the FPU unit is heavily underutilized by most things outside of games, so it's not an unreasonable idea to strip it out and let the FPU be emulated in software or microcode or whatnot.

However, that won't necessarily really help. Modern CPU cores are better able to manage their power routing than previous ones, so having an FPU on there doesn't necessarily cause any trouble. The CPU may be able to disconnect power to the FPU when it's not in use, thus making the whole thermal issue something of a moot point in this respect. If it doesn't generate heat, it's just a matter of wasted silicon - and silicon's becoming quite cheap!

In fact, the FPU is an example of good design in CPU's, really. It's not too hard to fit a lot of computation units on one CPU core these days, hence having multiple ALU and FPU computation units being driven by complicated pipelining and SIMD engines. The difficulty is making efficient use of them - note the trouble getting SMP to work efficiently, and the whole idea of hyperthreading. While the FPU may get fairly low utilization, it is fantastically faster at floating point computation than the integer cores are, and putting a couple on a chip is thus generally a pretty good idea.