Visualizing System Latency
ChelleChelle writes "Latency has a direct impact on performance — thus, in order to identify performance issues it is absolutely essential to understand latency. With the introduction of DTrace it is now possible to measure latency at arbitrary points; the problem, however, is how to visually present this data in an effective manner. Toward this end, heat maps can be a powerful tool. When I/O latency is presented as a visual heat map, some intriguing and beautiful patterns can emerge. These patterns provide insight into how a system is actually performing and what kinds of latency end-user applications experience."
Back in the day, working at an instrumentation company as a mechanical guy, I stopped to watch the senior electronic design engineer who was doing something that looked interesting. He had an old persistence-type storage oscilloscope hooked up to the rack-mount computer for a new instrument system and was watching the scope display, which was producing some fascinating patterns. Knowing f'all about this stuff but intrigued, I asked him to explain what was happening. He explained (and I'll butcher the explanation with layman's terms) that he was using d/a converters on the high and low bytes of the program address? to drive the x and y axes of the scope, and watching to see where, in the software, that the processor was spending much of it's time. He pointed to a hot spot on the scope display and said that this was where he would concentrate on optimizing his code. Fwiw, I thought that was pretty cool.
These visualizations are used to condense the information gathered on one second intervals from running systems. Any graph of substantially advanced material is going to require explanation until you understand what is being measured, how it is being graphed, and how this information translates in real world performance.
Of course a casual reader from the net needs to read text to understand what is going on. These aren't sales figure pie-charts and shouldn't necessarily be accessible for uninformed parties.
On another note.. Do you think casual readers would have any more success interpreting the raw data files? Anyhow, I am interested in the technique as it is not one I am currently using. With a little practice this may be a good at a glance technique.