Understanding Bandwidth and Latency

M. Woodrow, Jr. writes "Ars has a very eye-opening article on the real causes of bandwidth latency and why we should just drool endlessly over maximum throughput issues. In particular, I think the author's look into the PowerPC 970 and the P4's frontside bus is interesting considering how we're constantly being told by marketers that more speed is always going to translate into massive performance gains. The issue is, of course, far more complex, and this article does a good job of thinking about the problem from an almost platform agnostic point of view."

Comments

Re:Who cares?

[The Optimizer]

One thing I have found though, and it may apply to your work, is that when the process is computation intensive, particularly 80-bit precision FPU intensive, then the FPU processing time is sufficently large enough to mask variations in memory designs. That is, you're FPU speed bound not memory bound.

Here here!

[aztektum]

I work begrudgingly in a CompUSA store and the customers are suckered by our sales guys, yet I need the $ so I just suck it up and feel dirty.

I just wish this information was more well distributed, and that people would actually research what they're getting into. They treat it like clothes shopping, they stop in and take something that looks cool home, but if you're plunking down a big wad of money you should research, sadly they don't, then they're pissed when they realize a week later they were suckered. And since I work customer service counter I get to play whipping boy.

(On a truly sad note this one customer swore at me and said it was "Horse shit..." that we didn't carry Dell even though I said it wasn't our decision to make...)

Ultra 320 SCSI

[Bullseye_blam]

Yes, while the theoretical rate is much faster than PCI (as you noted), I believe that these cards are designed for 64-bit PCI slots, which you can see by the chart (which only lists fast/wide PCI) is 4x faster. A standard 64-bit slot running at 33 mhz (the speed at which most 32-bit slots run) is twice as fast as standard PCI.

So actually, Ultra-320 SCSI is the shit. ;)

Re:It can be done better with self-modifying code

[joto]

Nope. Most modern processors have separate data and code caches. So when you write self-modifying code, the data-cache must be flushed to memory, and the code-cache reloaded. In the meantime, the pipeline will be stalled, and the instruction decoded (important for x86) must start all over again. Pentium Pro handles this automagically, other processors may need special tricks.

So self-modifying code is rarely important (and of course very hard to write/maintain). Code with dynamic compilation (e.g. jvm) is possible to write in a sane way, and can give potentially large speedups. Of course, this goes for C as well. Sometimes for an inner loop, it's better to write a C-program at runtime, compile it, and load it as a dynamic library instead of having lots of parameters to the function. Of course, that is much more heavyweight than what the JVM does. It would be nice to have a portable alternative. But actually modifying that code afterwards is really hard (and inherently non-portable).

Of course, there are some uses for self-modifying code that can be made quite safe, and simple to understand. E.g. Knuth's MMX uses self-modifying code to store the return address in procedure calls. (I believe that was quite a common thing to do when making FORTRAN compilers back then...).

On the x86, such tricks are relatively easy, because the x86 tends to almost always have instructions available where you can store a full 32-bit pointer/integer in the opcode (whereas most RISC architectures will not). But you will not get a speed benefit by using it, as explained above in the first paragraph.

Re:Who cares?

[Tim C]

How many of you have found computer speed to be a major impedence on your productivity?

I have.

I write server-side Java code for a living at a web agency. Until recently, I had a P3 450 with 384meg of RAM, and it was too damn slow. I develop in JBuilder, and deploy my code using Resin (see caucho.com), and for complex sites it could take literally minutes for the server to start, then a couple more minutes per page in debug mode for the pages to be parsed and compiled. You're looking at 10-15 minutes to check to see if a one-line bugfix has worked, and hasn't had any unexpected side-effects, etc. That's 10-15 minutes of waiting, waiting, waiting, clicking a link, waiting, waiting, waiting, entering some details and hitting submit, waiting, waiting....

Extremely frustrating, especially if you're working late, especially when you know that 30-45 minutes spent going to the nearest high-street electrical retailer will buy you a machine 4 times as fast.

That's all before we even get on to the responsiveness of JBuilder...

I now have a P4 1.9GHz with 3/4gig of RAM, and the difference is incredible. JBuilder is much more responsive (feels like native code rather than Java most of the time), compile-run-test-debug cycles are much reduced, etc. This has a knock-on effect - people are generally happier and less fresutrated, stress levels are lower, there's less swearing (one or two of my colleagues regularly vented steam by hitting their PCs and swearing loudly) - work is all-round more enjoyable.

The bottom line is that anything that means that I spend less time waiting for things to compile, or startup, or whatever, is a good thing. Do not make the mistake of thinking that you'll never have a use for all that power; you'll find a use. The laser, for example, was sat around in research labs for years before anyone thought of anything practical to do with them. Now, practically every PC has at least one, not to mention hi-fis, DVD players, etc.