Ask Slashdot: Is the Gap Between Data Access Speeds Widening Or Narrowing?

New submitter DidgetMaster writes: Everyone knows that CPU registers are much faster than level1, level2, and level3 caches. Likewise, those caches are much faster than RAM; and RAM in turn is much faster than disk (even SSD). But the past 30 years have seen tremendous improvements in data access speeds at all these levels. RAM today is much, much faster than RAM 10, 20, or 30 years ago. Disk accesses are also tremendously faster than previously as steady improvements in hard drive technology and the even more impressive gains in flash memory have occurred. Is the 'gap' between the fastest RAM and the fastest disks bigger or smaller now than the gap was 10 or 20 years ago? Are the gaps between all the various levels getting bigger or smaller? Anyone know of a definitive source that tracks these gaps over time?

Comments

The gaps are getting smaller

[NotInHere]

The distance between the "fastest" and "slowest" gets larger and larger, but the gaps are getting smaller because things like SSDs fill them.

Does it matter?

[i.r.id10t]

Does it matter? Fast CPU, fast RAM, fast disks is like having no speed limits on every race track in the world - but in order to get from track to track you have to go on the interstates or perhaps back country roads (PCI bus, etc). Sure, each component is fast and getting faster, but the way those components connect to each other hasn't changed all that much...

Re:Does it matter?

[Anonymous Coward]

I disagree, the interconnecting paths may not be keeping up as fast but look back at the original ISA bus or before that S100. The PC/AT 16 bit slots more than doubled the speed of the original ISA. Specialized video busses were all the rage not that long ago. PCI has grown to become PCIX and multi lane. For us greybeards, that a serial bus operated faster than a parallel bus remains one of those great mysteries.

Re:Does it matter?

[Wycliffe]

Yes. It matters. Price also matters. If the price/performance gap is eliminated then major architectural changes are both possible and likely. We're already seeing this with tape vs harddrive where people are just archiving to an external harddrives instead of tape. Think about what would be possible if RAM was cheaper than harddisks and also non-volatile. Then everything could be in ram and even "saving to disk" becomes a thing of the past. Likewise, if SSDs becomes just as fast as ram, then why differentiate between ram and harddrive at all, just have "memory" which can be used or reused however you like.

Re:Does it matter?

[godrik]

That is so untrue. What are you talking about!?
PCI-express is much faster than the late AGP, PCI and ISA buses, you get over 15GB/s from one GPU to an other one. SATA and SCSI have had a good run, but are being remplaced by PCI-express for high throughput devices. In clusters, Infiniband decreased latency massively compared to ethernet, and gives you bandwidth of over 100Gb/s.

So surely, the interconnect is a factor slower than devices, but that's pretty much always has been the case.

Re:Does it matter?

[sexconker]

fast disks

I showed your mom my fast disks, but she only had an ISA port.

Re:Does it matter?

[drinkypoo]

I had RAM on the 8-bit ISA bus back in the PC-1 days, but back then the CPU was so slow that wasn't a serious impediment. Today, however, we have specialized (and seriously fast) memory buses, and we have massive interconnects between the CPU and the chipset, and we have large numbers of PCI-E lanes. High-speed SSDs are designed to sit right on a useful number of PCI-E lanes. These days a PC actually does have massive bandwidth, in a way it never did before; it was crippled between basically the time PCs got into double-digit MHz speeds and the time when PCI-E became the prevalent bus.

In my current PC, the only stuff connected to a PCI bus (which is chained off a PCI-E bus) is IEEE1394 and some of my USB ports. Everything else connects directly to PCI-E, down to my audio hardware. I have overclocked dual-channel memory... yet benchmarks prove that this makes basically no performance difference. (Still, it's got an XMP profile, why not use it?) Bandwidth is not something which concerns me in my PC any more.

Re:Does it matter?

[Khyber]

" The bus is more often the bottleneck than anything connected to it in modern computing systems."

Not even close. Quite often it's the underlying architecture itself causing the bottlenecks.

Example, Intel's latest and greatest Xeons FUCKING SUCK. Why? Because their internal architecture to deliver data across cores is gimped beyond belief. You can run 2 CPU x 4 GPU, 4 CPU x 2 GPU, but you can't do 4 CPU x 4 GPU. Meanwhile, I've got far older AMD systems that run 4 CPU x 4 GPU without a problem.

What are you really asking?

[jeffb (2.718)]

I'm not sure what a historic timeline of these ratios (not "differences", please) would gain you.

These ratios can have a big impact on what algorithms and implementations you choose to maximize performance. I suppose if, say, the ratio of RAM to disk speed increased by a factor of 10 over the decade before last, then decreased back to its original ratio in the last decade, it might be worth trawling through some old papers (or old source trees) to revisit lessons learned in the earlier period -- but that seems like a bit of a stretch.

If you're just curious, it shouldn't be too hard to generate timelines of CPU cycle speeds, cache and RAM latencies and bandwidths, disk performance, and so on. But really, each of those has enough factors that a simple "ratio" would probably conceal more than it illuminates.

Writing a paper

[Anonymous Coward]

for a CS or IT class?

Is the Gap Widening Or Narrowing?

[fleabay]

Yes, yes it is.

Re:WTF post, come on kids

[jeffb (2.718)]

Ancient scrolls of dubious provenance hint darkly that DDR4 was not the first inhabitant of the RAM slots we consider so permanent. Debased cultists still sometimes mutter chants mentioning "PC100", or even uncouth syllables such as "korr"...

Re:WTF post, come on kids

[Anonymous Coward]

This could literally be answered with three google searches. '2015 l2 cache speed'

This /. article, plus one called "Casino lock flooring [...] play casino online" which 404s in Norwegian when you click on it.

'ddr4 speed'

As it turns out, a whole bunch of really technical reviews on DDR4 memory, plus 'scope/test gear for testing DDR4 bus access. At least partially, potentially, useful, if you're prepared to wade through a bunch of dense stuff.

'2015 ssd speed'

A whole bunch of MacBook reviews/unpaid ads, followed at the end by a Toshiba and a Kingston paid ad (sucks to be them, should have paid more).

So, ummm, like most LMGTFY trolls who think they're way smarter than they are, the actual results are far less useful than just asking someone who knows what they're talking about for the answer. (Whether /. counts as that is an entirely different matter, though there's usually at least one person who appears to know what they're talking about and will answer the question usefully and honestly without being a smug stuck up prick.)

lolwut?

[kuzb]

"Everyone knows that CPU registers are much faster than level1, level2, and level3 caches."

I'd argue that most people don't even know what a CPU register is, never mind what it's faster than.

Re:lolwut?

[freeze128]

And apparently, now everybody knows that 15 minutes can save you 15% or more on car insurance. Well, Screw that! We need to get the word out. CPU registers are faster than level 1 cache! Suck it, Geico.

Too complicated to answer

[gman003]

Originally, there was CPU registers, and memory. Then there was registers, memory, and disk. Then there was registers, SRAM cache, memory, and disk. Then there was registers, L1 cache (on CPU), L2 cache (on mobo), DRAM, and disk. Then the L2 moved onto the CPU. Then there was L3. Then SSDs were added between RAM and disk. Now some chips have an L4 cache on the CPU package (but not the CPU die).

Oh, and there's a difference between latency and bandwidth. DRAM latency has not significantly improved over time, particularly compared to DRAM bandwidth.

And with multiple cores, some levels are core-specific while others are not. You can even have a bizarre situation where L1 cache is per-core, L2 cache is shared between two cores, and L3 cache is per-CPU (in SMP setups, that means main RAM is the first level shared among all cores).

Re:Too complicated to answer

[Blaskowicz]

And IBM has made external memory controllers with 16MB L4 cache in each of them, called "Centaur".

RAM latency is not getting much faster

[hkultala]

The latency of RAM is improving very slowly, only something like 2x-4x improvement in last 20 years.

Only the bandwidth of the memory is growing faster, and that's just because they have been putting more dram cells in parallel, always doing bigger data transfers and having faster memory bus.

Same is true for hard disk drive speed, the rotation speeds dictates the random access latency and the rotation speed of average hard disk has only gone up from 4200 or 5400 to 7200 rpm in the last 20 years, meaning only 1.7 or 1.33 times improvement in random access latency

  Though replacing hard disks with flash-based SSD storage has improved latency by a huge margin.

The gaps are still there.

[Anonymous Coward]

20 years ago main memory was 10-14 ns, instruction cycle time was 2-4ns (Cray)

Guess what? it still is.

Memory has grown, it has gotten cheaper.

What HASN'T changed? Access to memory. That is how Cray got its speed - instead of a single port to memory, it used a crossbar switch - 4 ports for each processor. 1 instruction bus, 2 input data busses, and one output bus; even I/O got its own port to memory; all with overlapping address/data cycles.

The effect was that all of main memory worked at the speed of cache, thus the CPU had no need to waste silicon on cache memory - and the entire system ran full speed.

What slows down the current systems? Memory access. Most systems only have a single port to main memory. Some servers and "high performance" desktops have dual ported memory. Yet even dual ported memory access is slow when you have to share it among 4/8 cores... plus I/O (which isn't dual ported). Interrupt latency on PCs is really horrible. Still only 15 IRQs? and have to share them? No direct vectoring? Forced interrupt chain actions? Even the old PDP 11 with ONE interrupt request line allowed direct interrupt vectoring (64 basic vectors) to reduce overhead.

There hasn't been much innovation in architecture in over 20 years.

Re:The gaps are still there.

[business_kid]

I am a dissenter from Moore's law. There are physical limits and we are near them. There is a physical limit on fab size, charge time, propagation time, signal speed, current and heat. There are also pricing issues: Would you pay $300 extra for faster ram? It could probably be done by using cache. I doubt if users would pay.
If we could fit the entire system on a chip, Then you could speed up. But choice goes out. No choice at all.

Shed some light

[transfire]

This article can shed some light on it: http://www.dba-oracle.com/t_hi... Looks like RAM is the laggard.

Calculating is easy

[guruevi]

You can look up the specs easy.

Back in the 80286 days, there was not even an L1 cache however the memory and ISA bus ran at CPU speed 8-20Mhz. Hard drive latency was ~65ms.

In the 80486 days L1 cache was introduced and L2 was sometimes available in (very) expensive modules. I remember buying 256kb for the same price as 16MB RAM. The L1 caches ran (if I remember correctly) at CPU speed, 1 cycle. However the bus speed started to slow down compared to the CPU. The VLB ran at CPU bus speed ("local" bus) and was often used for graphics but PCI (an inferior bus) ran at 33MHz so for anything over 33MHz, we started needing dividers. The RAM ran at 80-120ns so it started being slower than the CPU bus. Hard drive speeds were however up to ~30ms.

In the Pentium age memory slowed even farther compared to the CPU bus. Now it took several cycles to access memory, buses ran even slower (still PCI mostly, eventually PCI-X (133MHz?) until PCI-e (serial buses running) came along. Hard drive speeds went up to ~15ms

In modern age, L1 caches have slowed even further requiring 4 cycles for L1 cache and up to 30 for L3 caches. RAM is even slower access with bus speeds about a quarter of a single CPU but sometimes 16 CPU's need to share those lanes. Peripheral bus speeds however have gone up and PCIe 3.0 is now directly integrated into CPU 80486 VLB-style. Hard drives have latencies of 10ms (we have a mechanical issue there) still but even cheap SSD's can go down to ~1-2ms.

Re:WTF post, come on kids

[HiThere]

Yes, he was ruling himself out as unable to answer. So am I. And it would take a *LOT* more than a Google search to answer. I lean towards agreeing with the people who cite bus speed as the limiting factor, but I'm not sure, and there could certainly be special circumstances where something else was the limit.

I *do* know that it's not an easy question to answer, and that any answer is going to depend for its correctness on a presumed workload. (Some things are CPU bound, and don't even use much RAM. Other things are IO bound, and make you think your disks are thrashing. Most things are somewhere in between.)

But the original question was "has the gap between fast-small memory and slow-large memory gotten larger, smaller, or stayed the same. Even that's an oversimplified question, as it doesn't deal with persistent RAM. (I'm dubious about the value of that, but some people used it to advantage in the days of core-memory.) Also ignored were the questions of relative cost. If you pay ENOUGH there are lots of exotic technologies...and I have no idea of the tradeoffs.

So much better to get the answer from somebody who actually knows the area. It's not simple.