3D DRAM Spec Published

Lucas123 writes "The three largest memory makers announced the final specifications for three-dimensional DRAM, which is aimed at increasing performance for networking and high performance computing markets. Micron, Samsung and Hynix are leading the technology development efforts backed by the Hybrid Memory Cube Consortium (HMC). The Hybrid Memory Cube will stack multiple volatile memory dies on top of a DRAM controller. The result is a DRAM chip that has an aggregate bandwidth of 160GB/s, 15 times more throughput as standard DRAMs, while also reducing power by 70%. 'Basically, the beauty of it is that it gets rid of all the issues that were keeping DDR3 and DDR4 from going as fast as they could,' said Jim Handy, director of research firm Objective Analysis. The first versions of the Hybrid Memory Cube, due out in the second half of 2013, will deliver 2GB and 4GB of memory."

Comments

Every other iteration of ram tech is a dud

[sinij]

Just like Star Trek movies, every other iteration of memory tech is a dud. I will just wait for holographic crystals.

Still waiting...

[Shinare]

Where's my memristors?

Re:Still waiting...

[fyngyrz]

Your memristors are with my ultracaps, flying car, and retroviral DNA fixes. I think they're all in the basement of the fusion reactor. Tended to by household service robots.

Oh noes

[ArcadeMan]

Where I have seen 3D silicon before?

Re:nothing new here

[Thagg]

I was working at SGI at the time, late 1991. The cheapest way to buy expansion memory was to buy Indigo's and throw out the rest of the computer. SGI was just feeling the first tickles of the commoditization of computer hardware, and was looking for ways to make their components unique (and keep them expensive.)

Re:And for faster performance

[ArcadeMan]

Mac users won't see any difference in 5 years... wink wink

Posted from my Mac mini.

Latency?

[gman003]

Massive throughput is all well and good, very useful for many cases, but does this help with latency?

Near as I can tell, DRAM latency has maybe halved since the Y2K era. Processors keep throwing more cache at the problem, but that only helps to a certain extent. Some chips even go to extreme lengths to avoid too much idle time while waiting on RAM ("HyperThreading", the UltraSPARC T* series). Getting better latency would probably help performance more than bandwidth.

Re:Latency?

[harrkev]

I have a passing familiarity with this technology. Everything communicates through a serial link. This means that you have the extra overhead of having to serialize the requests and transmit them over the channel. Then, the HMC memory has to de-serialize it before it can act on the request. Once the HMC had the data, it has to go back through the serializer and de-serializer again. I would be surprised if the latency was lower.

On the other hand, the interface between the controller and the RAM itself if tighly controlled by the vendor since the controller is TOUCHING the RAM chips, instead of a couple of inches away like it is now, so that means that you shold be able to tighen timings up. All communication between the RAM and the CPU will be through serial links, so that means that the CPU needs a lot less pins for the same bandwidth. A dozen pins or so will do what 100 pins used to do before. This means that you can have either smaller/cheaper CPU packages, or more bandwidth for the same number of pins, or some trade-off in between.

I, for one, welcome our new HMC overlords, and hope they do well.

Re:Latency?

[Cassini2]

This technology will not significantly affect memory latency, because DRAM latency is almost entirely driven by the row and column address selection inside the DRAM. The additional controller chip will likely increase average latency. However, this affect will be lessened because the higher bandwidth memory controllers will fill the processors cache more quickly. Also, the new DRAM chips will likely be fabricated on a denser manufacturing process, with many parallel columns, which will result in a minor improvement in speed.

All told, this new technology will not change the fact that modern CPU's spend about 50% of their clock cycles waiting for data.

Re:Latency?

[hamster_nz]

This change of packaging allows greater memory density, and maybe higher transfer bandwidths. It will not alter the "first word" latency much, if at al.

Signal propagation over the wires isn't the problem, it is the way all DRAM works is.

- The DRAM arrays have "sense amplifiers", used to recover data data from the memory cell. The are much like op-amps, To start the cycle both inputs on the sense amplifier are charged to a middle level,
- The row is opened, dumping any stored charge into one side of the sense amplifier.
- The sense amplifiers are then saturate the signal to recover either a high or low level.
- At this point the data is ready to be accessed and transferred to the host (for a read), or values updated (for a write). It is this part that the memory interconnect performance really matters (e.g. Fast Page mode DRAM, DDR, DDR2, DDR3).
- One the read back and updates are completed then the row is closed, capturing the saturated voltage levels back in the cells.

And then the next memory cycle can begin again. On top of that you have have to add in refresh cycles, the rows are opened and closed on a schedule to ensure that the stored charge doesn't leak away, consuming time and adding to uneven memory latency.

Re:Latency?

[doublebackslash]

Pointer chasing is the cannonical example. Trees, linked lists of every flavor, maps, many many more.

Even if your memory accesses are aligned you will still start to stream cache misses as soon as you are operating beyond the limits of cache, or start bouncing between cores and/or threads (snooping is cheap, but it isn't free and by the time you get there another thread might have kicked out your data).

Then there is synchronization between threads. Fences aren't free (far far from it, though some can be cheaper than others)

Some practical examples are rays tracers (objects scattered all around memory), XML parsers (relatively huge objects and more. Love or hate it XML is everywhere), precise garbage collection scatters certain objects around memory, and compression.
That is just off the top of my head, but you get the idea. Not everything is contigious. Even when it is you can easily stream misses a rate collosally higher than they can be served.

Re:And for faster performance

[TheRaven64]

Most CPU vendors do. This has been the standard way of shipping RAM for mobile devices for a long time (search package-on-package). It means that you don't need any motherboard traces for the RAM interface (which reduces cost), increases the number of possible physical connections (increasing bandwidth) and reduces the physical size. The down side is that it also means that the CPU (and GPU and DSP and whatever else is on the SoC) and the RAM have to share heat dissipation. If you put a DDR chip on top of a Core i7, then one or the other (or possibly both) would be too hot to function. There are quite a few interesting experimental architectures that mix execution units and RAM on the same die, because the power cost of moving data between RAM and CPU is starting to be important. It's also often cheaper (in terms of both time and power) to recompute intermediate results than fetch them from main memory for workloads such as image processing.

Re:Dram

[fuzzyfuzzyfungus]

So when can people running ddr1 or ddr2 expect to get some multilayer chips that vastly increase memory bandwidth in older systems?

Given that, for PC applications at any rate, the memory controller is built into either the motherboard or the CPU, there is likely to be a bottleneck there in any case. There would have been no reason for designers of memory controllers of the era to spec them out with the expectation of more than modest improvements.

Also, this '3D memory' stuff includes a memory controller with the DRAM dice stacked on top. To what, exactly, in a DDR2-using system are you going to connect a fancy new memory controller?

If you were a real high roller with a big cluster full of multi-socket hypertransport based systems or something, somebody might be moved to build some very, very, high performance memory modules that occupy CPU sockets; but that's a serious edge case. Most systems(even new ones) simply don't have a spare bus fast enough to hang substantially-faster-than-DDR3 RAM from.

Re:And for faster performance

[harrkev]

HMC does not need to sit on top of a CPU. HMC is just a way to package a lot of memory into a smaller space and use fewer pins to talk to it. In fact, because of the smaller number of traces, you are likely to be able to put the HMC closer to the CPU than is currently possible. Also, since you are wiggling fewer wires, the I/O power will go down. Currently, one RAM channel can have two DIMMs in it, so the drivers have to be beefy enough to handle that posibility. Since HMC is based on serdes, it is a point-to-point link that can be lower power.

I am sure that at speed ramps up that HMC will have its own heat problems, but sharing heat with the CPU is not one of them.

Re:nothing new here

[jandrese]

Nobody ever accused SGI of sane pricing.

Re:Dram

[jandrese]

The overall design reminds me a lot of Rambus. It saved pins and had excellent sustained throughput, but memory latency suffered.

Memory is far more complex than you imagine.

[hamster_nz]

If you think that modern memory is simple send an address and read or write the data you are much mistaken.

Have a read of What every programmer should know about memory and get a simplified overview of what is going on. This too is only a simplification of what is really going on.

To actually build a memory controller is another step up again - RAM chips have configuration registers that need to be set, and modules have a serial flash on them that holds device parameters. With high speed DDR memory you have to even make allowances for the different lengths in the PCB traces, and that is just the starting point - the devices still need to perform real-time calibrate to accurately capture the returning bits.

Roll Serial Port Memory Technology!

Re:And for faster performance

[ackthpt]

the CPU vendors need to start stacking them onto their die.

In 5 years your systems will be sold with fixed memory sizes, and the only way to upgrade is to upgrade CPUs.

Stacked vias could also be used for other peripheral devices as well. (GPU?)

IBM tried this with the PS/2 line. It fell flat on its face.

Re:And for faster performance

[gagol]

I would like to see 4GB on die memory with regular DRAM controller for "swap" ;-)

Re:And for faster performance

[kaws]

Hmm, tell that to my upgraded Macbook. I have 16gb of ram in mine. On the other hand, you're probably right that it will take a long time for the upgrades to show up in apple's store.

Re:And for faster performance

[forkazoo]

To be fair, if somebody tried to sell something as locked down as the iPad is during the period when IBM first released the PS/2, it would have also flopped. The market has changed a lot since the 1980's. People who seriously upgrade their desktop are a rather small fraction of the total market for programmable things with CPU's.

Ultracaps

[fyngyrz]

Um... yeah. No. I appreciate that what you have are considerably better than regular caps, but they're nowhere *near* the performance of what we keep being offered. Nanotube infused designs with power to weight ratios around that of batteries, graphene designs, etc. There's a huge wealth of applications waiting for them to hit somewhere around those marks. Electric cars, actual car battery replacements, cellphone power supplies that never die, backup systems for the house with peak powers far in excess of anything we have now but with comparable storage... the ultracap "breakthroughs" are as regular as any other kind (memristors, etc.) and the consistent no-show of actual commercially available units is also consistent. It's the flying car of electronic components, sigh. High voltage, high capacity, high vapor factor, lol.

Believe me, I've been following the whole ultracap thing for a while. I even keep an eye on EEStor, which I can assure you has been a stupendous exercise in fruitless waiting. As a ham with a full boat of offline powered goodies and the beginnings of a household able to run off backup systems, and more than a little willingness to buy an electric car, actual availability of ultracaps in what I call "the battery range" would truly light me up.

But that carrot is well and truly still out on the stick.

Re:Ultracaps

[jkflying]

TI has a new range of super-low-power embedded chips which use FRAM, they are using it to replace flash and get faster writes, lower power consumption and higher write cycles before failure, so there's one new tech which made it to market and might become more popular over the coming years as it gets cheaper.

And even current-gen ultra-capacitors have a similar or better *power*/weight ratio as a battery - I'd like to see a 30g battery which can give 30A at 600V without damage to itself. It's the *energy*/weight ratio which is a killer - that 30A spike doesn't last long enough to be useful for the types of applications we currently use batteries for.

Re:Dram

[QQBoss]

Back in 1997, it was determined that ~90% of the benchmarks and customer applications (provided to us for testing purposes, the NDAs were amazing) used on PowerPC were completely dominated by cache misses. That means that if we knew how many times the processor touched a bus (data easily obtained in real time), we could be accurate to within 5% of what the performance would be using a spreadsheet calculation (Thanks, Dr. Jenevein) vs running the apps on a cycle accurate system simulation which could take weeks to develop a meaningful profile. Every time the caches got bigger, the code to solve customer problems would get proportionally bigger. That hadn't changed in 2007 and isn't anticipated to change by 2017. There are edge cases, but until people are satisfied with continuing to play Lode Runner instead of Crysis N, it won't matter for the mass market.

That doesn't mean that CPUs don't need to get bigger/faster, but it does mean that there is a meaningful limit on performance relative to the cache size, the calculation of which is probably left to an exercise for the student in H&P's Computer Architecture.

Re:And for faster performance

[Issarlk]

Since those are 3D chips, does that mean Apple's price for those RAM will be multiplied by 8 instead of 2?