Slashdot Mirror


Sun Unveils Direct chip-to-chip Interconnect

mfago writes "On Tuesday September 23, Sun researchers R. Drost, R. Hopkins and I. Sutherland will present the paper "Proximity Communication" at the CICC conference in San Jose. According to an article published in the NYTimes, this breakthrough may eventually allow chips arranged in a checkerboard pattern to communicate directly with each other at over a Terabit per second using arrays of capacitively coupled transmitters and recievers located on the chip edges. Perhaps the beginning of a solution to the lag between memory and interconnect speed versus cpu frequency?"

40 of 185 comments (clear)

  1. It will be running java by holzp · · Score: 5, Funny

    therefore the speeed increase will be unnoticable.

  2. Timing? by afidel · · Score: 3, Interesting

    I wonder if this release might have been pressed forward a bit to squelch some of the talk about Sun losing their will to innovate after Bill Joy left.

    --
    There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    1. Re:Timing? by Usagi_yo · · Score: 5, Insightful

      No. What you don't understand or realize is that Bill Joy actually left 2 years ago, when he "retired" into distinguished senior engineer, from CTO. This latest move by Bill Joy, full retirement is merely a continuation of that. At least thats how I see it.

  3. terrabit by lanswitch · · Score: 2, Funny

    Does "terrabit" mean that it will be made of pieces of the earth?

  4. No registration by Anonymous Coward · · Score: 5, Informative
  5. Replacing Network-on-Chip/System-on-Chip by KarmaPolice · · Score: 4, Interesting

    This could prove very interesting as the speed usually drops when "leaving the chip" to do communications. There has been alot of research to develop protocols to ease on-chip communication when several ICs are combined on a single chip. If Suns technology can stand the test, NoC/SoC products could reduce it's time-to-marked dramatically...smaller and faster devices for everyone!

    BTW: I didn't RTFA since it requires (free) reg.

  6. I suppose this will be patented... by Thinkit3 · · Score: 3, Funny

    Or maybe Rambus is already fixing to sue them.

    --
    -Libertarian secular transhumanist
  7. Fast today Slow Tomorrow by Anonymous Coward · · Score: 5, Interesting

    That is the nature of the beast.

    Remember how excited you were to get your hands
    on a 386 machine?

    The thrill of your first encounter with a 286 screamer?

    Upgrading to 16k from 4k on your TRS-80?

    Your first disk drive for your Apple 2?

    It's all relative.

    So enjoy

  8. SUV of chip interconnects? by Atomizer · · Score: 3, Funny

    Whatever, I think this will end up being the SUV of chip to chip conections. ;)

  9. Link via Google (no Reg. Required) by chrestomanci · · Score: 4, Informative
  10. IANAEE (I am not an electrical engineer) by Peridriga · · Score: 4, Insightful

    This might be the obvious question but, why hasn't anyone done this before?

    It seems obvious, the end of chip has pins. The chip it will eventually connect to has pins. Instead of having 20 trace lines to the next chip why not redesign them so the out/inputs of both line up to reduce the complexity of the design.

    Anyone wanna fill in my mental gap for me?

    1. Re:IANAEE (I am not an electrical engineer) by Jah-Wren+Ryel · · Score: 5, Insightful

      It has been done before, probably the most recent incarnation is hypertransport from AMD. The only difference at the 50,000ft view is that the speeds and feeds are faster. This is an evolutionary step, not revolutionary or innovationary,

      --
      When information is power, privacy is freedom.
    2. Re:IANAEE (I am not an electrical engineer) by jhines · · Score: 2, Informative

      It has been done.

      The DEC PDP11/03 aka LSI-11 was implemented as a multi chip (4 + 1 rom) CPU. The 5 chips were placed right next to each other.

      This chip set was also setup by others with the UCSD Pascal "p-code" as the instruction set.

      Other CPU in the series had MMU, and additional instructions in additional chips.

    3. Re:IANAEE (I am not an electrical engineer) by Usagi_yo · · Score: 2, Interesting
      Alot of reasons. We'll start with large complex chips. No, the pins aren't at the end of the chip, they are underneath the chip. BGA, CGA, LGA, ball grid, colume grid or land grid array.

      Then we'll go to ... well, they sorta did. They just enclosed it into one big chip.

      Then of course Heat and cooling and power requirments.

      Then onto manufacturability and repairability. Don't want to have a $15k board that has to be thrown away whenever there are problems with it. You do want to be able to repair it.

      Next on to glue logic or glue componants such as current limiting resisters, pull ups, pull downs, bypass and decoupling capacitance.

    4. Re:IANAEE (I am not an electrical engineer) by Anonymous Coward · · Score: 5, Informative

      You can't simply just remove the circuit board to achieve better speeds, you need to eliminate the need for the pad that converts internal logic to what we currently use externally. That is what Sun is claiming they have done.

      Sun's technology is not simply soldering to pins directly together (as you suggest), which is effectively the same thing as wiring through a circuit board. The high speed, low drive strength, low-voltage drivers have to go through pads that convert the internal signal to a slower, high drive strength, high voltage driver, that will yield a reliable connection to the next chip. I'm not an expert in this area, but Physics just gets in the way. There are capacitive issues, and interconnect delay issues.

      Sun is claiming to use capacitive coupling (put the pins really close together, but don't physically connect them.) This way they don't have to drive the external load of the pin/board connection, and are claiming they will be able to scale this down to a pad that will be able to switch faster than existing physical wire connected pins. Which means they believe they can make this technology work with lower drive stengths.

      They still have a ways to go. Notice that the P4 has faster connections using existing techology. Sun did a proof of concept, and claim they can speed it up 100x. So they haven't _proved_ that this will operate faster yet. They still have many things to overcome to make this viable, including how to make a mass production/assembly process. It's going to be a few years. At least.

    5. Re:IANAEE (I am not an electrical engineer) by fitten · · Score: 2, Informative

      So you think DRAM access time is 40ns (or so) because of the system bus?

      L1 cache typically found on today's processors and DRAM are two different things with different design targets. Pick up a VLSI book.

    6. Re:IANAEE (I am not an electrical engineer) by eXtro · · Score: 5, Informative
      There are two seperate metrics that define the speed of memory. Latency, which is what your 40 ns refers to, and bandwidth. Large caches address the latency problem as you stated. If you want to transfer more bits per cycle you're restricted due to signal integrity issues related to the bus, so the parent post is also correct. You can increase the width of the bus, up to a point, and get a small scalar increase in bandwidth. To go beyond this you need to address signal integrity problems.


      Sending fast edges over a bus is difficult because the signal degrades:

      • inter-signal interference: Each parsel of information spreads due to the RC nature of the bus so that it takes up more than a period, thus interfering with the next packet.
      • cross-talk: Each wire on the bus is fairly tightly coupled with it's neighbours, so switching activity on one wire affects it's neighbours.
      • transmission line effects: Package connectors, bends in circuit traces etc all create impedance mismatches. This causes reflections which degrade the signal.


      If your dataset fits into the cache well, which is often the case for PCs, then a cache can fix most of your problems. If you're dealing with datasets that span gigabytes or terabytes and your application can't be subdivided such that processing and memory can be constrained per cpu then your cache doesn't assist you very much.
  11. Imagine a Beowulf Cluster... by tekiegreg · · Score: 2, Funny

    of these, well that's kind of the point actually :-)

    --
    ...in bed
  12. Perhaps a physical base for Neural Network? by BlankStare · · Score: 3, Interesting

    I wonder if this hardware computing model could provide the first real base for Neural Network computing? As far as I know, any neural network is currently emulated on linear processing machines.

  13. FINALLY! by JoeLinux · · Score: 5, Interesting

    Someone gets it. As an Electrical Engineer-in-training, I was always frustrated with people who got these big bad processors and wondered why their improvement was minimal.

    They never quite grasped that the biggest bottleneck is between the processor and memory.

    My EE instructor always said that they could improve performance by doing one simple thing: make the interconnects on the motherboard between the motherboard and RAM rounded instead of cornered. You could then increase bus speed as you wouldn't have magnetic loss at the corners like you do now.

    You fix that, and you can see a SUBSTANTIAL improvement in performance. The only thing that can be done beyond that is to get a Platypus drive (Solid state "Hard Drive" made from Quikdata made from DDR RAM). Then you reduce your access time to your hard drive from milliseconds to nano/microseconds.

    1. Re:FINALLY! by Jeff+DeMaagd · · Score: 2, Informative

      I think the fourty five degree corners is about as much of a compromise as one can get without being too expensive with routing.

      Another problem is that the speed of memory itself isn't that great unless you want to spend a _lot_ of money, to the tune of $50-$100 per megabyte as we see in advanced processor caches, and the faster it is, the more very power inefficient it becomes, maybe to a sizeable fraction of a watt per megabyte.

    2. Re:FINALLY! by hackstraw · · Score: 2, Interesting

      Other people "get it". If you go to UVA, you might want to talk with Dr. McCalpin, and take a look at the stream memory benchmark.

      Memory bandwidth is a bottleneck, not the biggest. It depends on the application. Sometimes an app is CPU bound, disk bound, network bound, or memory bound (or graphics card bound if 130FPS is too slow for your eyes). Also, chip-to-chip interconnects will not change the memory bandidth issue, because if the data does not fit on the chips or thier cache, then its going in memory.

      Also, this is for scaling. Think beowulf. For those machines, the data must go from the cpu -> memory -> network interface -> switch -> network interface -> memory -> cpu. Yes, bandiwidth is an issue, but there is also latency, which would be very low with these kinds of chips.

      A side note, I work with a guy that soldered chips together like this which had native parallel processing instructions about 20 years ago.

    3. Re:FINALLY! by chrysrobyn · · Score: 5, Insightful
      Someone gets it. As an Electrical Engineer-in-training, I was always frustrated with people who got these big bad processors and wondered why their improvement was minimal.
      They never quite grasped that the biggest bottleneck is between the processor and memory.

      Don't get too frustrated with this. There will always be people who don't understand something fundamental to your training. That's why you're trained, to understand these non-obvious fundamentals. Now that you understand a CPU has to be fed data in order to process it, it's obvious, but a PHB wouldn't necessarily come to that conclusion on his own.

      My EE instructor always said that they could improve performance by doing one simple thing: make the interconnects on the motherboard between the motherboard and RAM rounded instead of cornered. You could then increase bus speed as you wouldn't have magnetic loss at the corners like you do now.
      You fix that, and you can see a SUBSTANTIAL improvement in performance. The only thing that can be done beyond that is to get a Platypus drive (Solid state "Hard Drive" made from Quikdata made from DDR RAM). Then you reduce your access time to your hard drive from milliseconds to nano/microseconds.

      Your EE instructor will tell you lots of things that can help performance. For example, making the L2 cache be the size of main memory. Just because it helps performance doesn't make it worth the price. Rounded edges on the PCB are not easy to accomplish and their benefits may not be outweighed by the added price-- even for exceptionally high end servers. Without looking at the math, I would like to toss "10% performance adder, 50% cost adder" out into the air, and say that most people would rather save the dough. Another factor to consider is reliability. Intuition suggests to me that reliability would go up without sharp edges, but intuition also tells me that modelling board coupling on a 4 layer board would be a real pain in the ass, to say nothing of a server class 6 or 8 (or higher) layer board if you have to model curved structures. You might not find an easy way to capitalize on your wonderful curved wire performance. Not only do you have to worry about your slowest path, but your quickest one can't arrive so quickly that the other chip can't sample the previous output.

      Take care in your classes when you use the word "only". Taking advantage of our wonderful next generation 64 bit processors and multiple gigs of RAM, we could conceivably copy the contents of the hard drive to main memory (especially if we are only concerned with 1-4 gigs of data in a low cost solution). Here, we get the enhanced bandwidth of main memory instead of having to kludge through the southbridge, PCI controller, IDE/SCSI to RAM interface and back.

      There are many things that improve system performance-- and the system is the only thing that matters-- rounded wires and SSDs (solid state drives) are only the beginning. Depending on the application, a larger L3 cache may make more difference, or a wider faster CPU to CPU interface, or a pair of PCI controllers hanging off the southbridge for twice the bandwidth, or integrating the northbridge onto the CPU, or ...

      The best engineering advice I can give you is that the answer is always, "It depends". You'll spend the next 5-30 years of your life learning how to answer the followon question "Depends on what?". Almost everything has advantages and disadvantages and there are few absolutes.

      The "Someone gets it" and "They never quite grasped" attitude may get you in trouble. Being proactive and explaining and educating instead will likely be more effective.

    4. Re:FINALLY! by ingenthr · · Score: 2, Insightful

      Quite right. Sun gets this quite well. Look for the articles on the Niagara processor. People always look at it as an SMP on a chip and try to compare it to hyperthreading or the stuff that IBM has done. However, what Sun is doing is 'fast switching' between threads based on stalls for memory access. This is another way of solving the problem you mention.

      If the software and programming model are capable (and most software run on Solaris is) of exploiting this, you effecively trade off bandwidth (easy to obtain) for latency (very, very hard to obtain).

      It's cool stuff-- I'm looking forward to seeing it released as a product.

  14. Is this new? by 4im · · Score: 3, Insightful

    Sounds a lot like the ol' Transputer (was from INMOS), of course faster. One could also think of AMD's HyperTransport. So, again, except maybe for the speed, I don't see much innovation here.

    If only people could remember that "terra" has something to do with earth, "tera" is the unit...

    1. Re:Is this new? by Ella+the+Cat · · Score: 2, Insightful

      It doesn't sound like the Transputer to me. Sure, they resemble each other in that you can build a 2D array of chips from them by design, but you miss (or inadvertently downplay) that the innovation occurs in the fundamental electronic engineering issue of what happens in the bits of circuitry that drive the pins/pads - the transputer used asynchronous links and conventional pins and a nice but conventional memory interface, the Sun chip is doing something new, or if not new, seldom seen and highly promising. Ivor Sutherland knows his stuff.

  15. or it might not by penguin7of9 · · Score: 3, Insightful

    Placing large numbers of chips adjacent to one another has obvious problems with heat and power, in particular when running at those speeds. That, rather than interconnect technology, is probably the main reason we still package up chips in large packages.

    This might be useful for placing a small number of chips close together, in particular chips that may require different manufacturing processes.

    1. Re:or it might not by Derivin · · Score: 2, Insightful

      Heat will definatly be an issue, but much less power will be required. The majority of the power required by chips is used to push data on and off the chip. It takes alot of poser to move a signal from a 25 micron PCB path.

      This technology (if it pans out) will mostlikly enter teh private sector in cell phones, DVD players and other small consumer electronics that have a very large number of units produced.

      Silicon wafer production has always had one major problem. Impurities. The ability to use more of the waffer to produce smaller chips that can later be 'put back together' in arrays that may not be any larger than the origional single chip solution has the potential to be much cheaper to manufacture in mass quantity.

      Granted this is part of the theory behind 6 Sigma, which does not always work out.

  16. Hard to say what's new here by kent.dickey · · Score: 2, Informative

    The article is a bit vague as to what the innovation really is.

    The article immediately made me think of multi-chip modules. Multi-chip modules is an idea which never really caught on in the industry (except for IBM), and I'm not sure how Sun's innovation isn't just a take-off along that idea. Multi-chip modules have failed due to costs since much has to go right to get a multi-chip module that works.

    Any practical chip-to-chip connectivity scheme had better have a good rework scheme. If it doesn't, it's just boutique technology that will not affect the industry overall.

    Having worked on chips with multi-gigabit pins, a huge problem is resynchonizing the signals. Creating a receiver to align one pin's data with 15 neighbors at 3GHz takes a whole lot more logic space on the die than a small driver (or receiver). The auxiliary logic basically makes shrinking the final driver FET almost meaningless.

    Modern chip design is a constant trade-off between features and cost. And what's cheap is what everyone has been doing for years (or is an evolution of that).

    1. Re:Hard to say what's new here by William+Tanksley · · Score: 2, Informative

      They could probably do something similar with arrays of laser diodes beaming out the edges of the chips

      Definitely. That would be electromagnetic coupling. Sun's using capacitive coupling, using only the E field. Last week we saw an article on a company using inductive coupling (magnetism) for short-distance data links (in their first product, a wireless earset).

      EM is long-range (drops according to the inverse square) but very hard to convert to and from electricity.

      M is short-range (inverse sixth power), relatively easy to convert, not easy to interfere with, but bulky and directional.

      E is short-range (inverse sixth), slightly harder to convert, not bulky, but easily interfered with.

      Sun's choice here is perfect: this application doesn't need (or want) the range of EM, and can't afford the mass and volume of an inductor. OTOH, the ease of interference is easily dealt with because once we know the geometry and composition of the board, we know the shapes the e-fields will have.

      I really like the fact that we've had two nicely orthogonal stories just so close together.

      -Billy

  17. Bad math? by Quixote · · Score: 4, Insightful
    I hate it when the hype overshadows the technical details. Here's a snippet from the article:

    By comparison, an Intel Pentium 4 processor, the fastest desktop chip, can transmit about 50 billion bits a second. But when the technology is used in complete products, the researchers say, they expect to reach speeds in excess of a trillion bits a second, which would be about 100 times the limits of today's technology.

    If a P4 is already doing 50 Gbps (as they say), and this uber-technology will allow 1Tbps (which is 20x a P4's 50Gbps), then how is that "100x the limits of today's technology" ?

    <shakes head>

  18. Sun may be ahead in other areas, too by Mr.+Ophidian+Jones · · Score: 4, Interesting

    Normally I don't pimp Sun, but here's something that makes me think they still have a finger on the pulse of things:
    Read about plans for Sun's "Niagra" core

    I understand they hope to create blade systems using high densities of these multiscalar cores for incredible throughput.

    There's your parallel/grid computing. ;-)

  19. This reminds me of by BackSpace · · Score: 2

    the Transputer. It had 4 available hardware connections and the description of the way the different processors communicate is very similar to what is described by the article.

    Of course to take maximum effect of this communication speed in general parallel applications, main memory access would have to be improved. I'd guess these things will have huge on-chip caches.

  20. More on the broader project by leery · · Score: 4, Informative

    IANAEE either, but this made a little more sense to me after I read this Inforworld article, which talks about two other aspects of Sun's DARPA-funded project: clockless "asynchronous logic", and building processors with interchangeable and upgradable modules. They absolutely need these busless "proximity" interconnects for the processor modules to communicate at close to on-chip speeds, and the clockless architecture lets them get rid of the bus. Or vice versa... or something like that.

    Working prototype computer about six years away, according to the article.

    --
    "This is not a sig." -- R.
  21. Transputer dusted off and presented as new? by Anonymous Coward · · Score: 2, Informative

    As usual with alot of Computer Science, this appears to be just an old idea reinvented...the Transputer...and about time too!

  22. Already been done with SERDES by StandardCell · · Score: 4, Informative

    If you look at a modern evaluation board with gigabit SERDES or SERialization-DESerialization (e.g. the 3.125Gbit/s differential signal pair per channel), the trace routes are typically rounded, with no square corners. This is done to reduce the effective impedance along the line which needs to be carefully controlled. They also run in parallel closely-routed pairs because it's typically a differential signal. Actually looks a bit like a set of minature train tracks without the railroad ties.

    In fact, multichannel SERDES is the next real interconnect technology. It's used in Infiniband, HyperTransport, PCI Express, Rambus RDRAM and in 10 Gb/s Ethernet (usually as 4x3.125Gbit/s channels as a XAUI interface between optical module and switch fabric silicon with 8b/10b conversion). There are even variants, such as LSI Logic's HyperPHY, that are deployed specifically for numerous high-bandwidth chip-to-chip interconnections. The problem that is cropping up is that the traditional laminate PCBs are becoming the limiting factor in increasing per-channel connectivity, to the extent that 10Gbit/s per channel speeds are next to impossible on these boards due to the lack of signal integrity. There has been some experimentation for very short hops on regular boards, as well as using PTFE resins to manufacture the boards themselves, but it's precarious at best.

    As for Sun's technology, it's interesting but I don't know how much it will catch on or how feasible it will be. It creates packaging issues and requires good thermal modelling and 3-D field modelling to account for expansion and contraction through the operating temperature range and the presence of nearby signals, which could affect the integrity of the signals.

  23. Re:It will be running java - The Best Comment ! by SlashingComments · · Score: 2, Funny

    Great ! I am so happy to see that there are some real programmers exist who see the truth. I have seen our Sun E3500 with 8 CPUs felt like a pentium pro with java shit running on it. But it was management's vision ... what we can do. I just procured the servers and pretended that I am doing social work by giving Sun more money.

    --

    - People who believe other people have no right to live, got no right to live ...

  24. you don't grasp what a hard disk is by Anonymous Coward · · Score: 2, Insightful

    You need more training. Or less ego.

    Look at a recent P4 motherboard for 45 degree traces. Look at any previous motherboard with RAMBUS (even an Nintendo 64 from November 1999) for curved traces.

    It's not so much a question as knowing about something as it is implementing it. If it isn't affordable, it isn't worth it. Because if it isn't affordable, you might be able to buy two affordable ones for the same price. And you're going to have trouble beating the performance of two systems with one.

    Finally, to make a hard drive from RAM is to totally lose track of the idea of what a hard drive is. Hard drives are supposed to be slower but they make it for it with lower cost per megabyte. Instead of a RAM drive, just put more RAM in your machine, it will use it as a disk cache/backing store and get you all the performance you want.

    Also, at 120us per command register access, you really cannot initiate any transfer over ATA in under 0.75ms.

  25. Increase yield? by mmol_6453 · · Score: 2, Interesting

    ...in particular chips that may require different manufacturing processes.

    Or at least portions of more complex circuits where part of the circuit may not warrant the added cost of SOI, 90nm, or strained silicon.

    But then, those divisions are already made. AMD, for one, is working on recombining those parts. As an example, consider AMD's putting the memory controller on the CPU die.

    I am curious, however, as to whether you could have more than one silicon die in the same ceramic casing. This would let you combine different parts made using different techniques. The CPU and L1/L2 caches could be on a high-cost process, with the L3 cache and memory controller being built with cheaper processes.

    Placing more emphasis on cost savings than on performance, you could build the L1 instruction cache with a slower process than the L1 data cache. Or you could leave all of L1 on the same process and split L2 into instruction and data caches, with different processes.

    If you wanted to ramp up CPU speed without a major hit to your performance, you could reduce the feature size of the core and L1 caches, and put L2 on a slower, more reliable process. That way, you could ramp up core speeds without having as much worry about yield loss from cache failure.

    Helluva way to increase yield, and it gives the designer a LOT of options.

    --
    What's this Submit thingy do?
  26. Re:Chip to Chip technology? by randyest · · Score: 2, Informative

    Isn't that called a trace? Or another fancy name would be a lead? I think that there are people with prior art...

    No, a trace is a flat wire stuck to (or etched from) a printed circuit board. This invention (process, really, see below) obviates the need for PCB's between (at least some of the) chips. A lead is a wire, not stuck to a PCB, such as the input connections to most oscilloscopes and test equipment.

    I don't get it either. You want to make memory access faster and faster, so you put it closer and closer to the cpu. Eventually the bus length reaches 0, as the two chips are physically adjacent. So what?

    As with many great inventions, the difficulty is not so much thinking of what needs to be done, but in actually doing it cost-effectively. System designers have been trying to use the idea of optimized interconnect (sometimes called "integration", as in LSI, VLSI, etc.) but it has remained cost-prohibitive in most cases (notable exceptions include the Pentium Pro and some ATI mobility products, but these are more desperation moves than anything, since margins drop on multi-chip "chips", and they had to do it to get the needed result even though the costs were higher than normally tolerable.)

    So, sure, light bulbs are obvious, as are cars, space shuttles, computers, etc. The hard part is making them possible technically and economically.

    Hope that helps you two understand why the ASIC-design industry is pretty damn excited and anxious to license this technology (if we really can do this as cheaply as they claim).

    --
    everything in moderation