Are Website Performance Metrics Still Relevant?

jackvalko asks: "Recently Keynote Systems began an upgrade of Transaction Perspective, one of their performance measuring products. The data collected is used by the Executive staff at the dotcom I work for as a means to evaluate our customer experience. Now that they are almost done, we've noticed a better than 40% reduction in response time of our site. While I'm happy that our performance 'looks' good, this major change has given us pause to question the statistical relevance of the data that is being collected and the somewhat arbitrary nature of performance metrics collection in general. And, of course, how we message all of this to upper management. So I put it to you, Slashdot! Do you find performance metrics relevant to your customer experience? How do you manage expectations and change to upper management? And, most importantly perhaps, are organizations that collect this data still relevant on The Internet as it exists today?"

Comments

Ask your customers!

[asc4]

I know this is anathema to most big businesses, but you say you work for .com, so maybe this isn't so far out in left field. What is the point in paying Keynote big bucks to tell you what your page load times are? Great, you've improved your performance 40%. Does it matter? No one here can tell you, Keynote can't tell you. Only your customers can tell you what about your site makes them happy and what frustrates them. So ask them!

Oh I don't know...

[davidu]

When you serve out a website that pushes in excess of 200,000,000 pageviews a day (yes I really do, dynamic even!) then you start to use a lot of metrics to try to gauge how your code changes and network changes affect things. There are lots of companies that specialize in this sort of thing. This runs from the application level all the way down to the network level, sometimes even the transport layer if you buy their marketing. :-)

Certainly from the network (RouteScience, Internap, etc) to the application (Zend, Urchin, Webtrends) there are all kinds of companies willing to provide you metrics and solutions, usually at a high cost.

The trick is to build your own metrics and then find or build the products you need to solve your bottlenecks or improve your user experience.

Just some ideas, not sure if it is the answer you want.

OT: And since I refuse to post in the "how do I build a mail server for 1mm users" thread I'll just say it here. "What the fuck dude? First of all, 1mm users isn't all that large these days. SMTP and POP3 are no brainers. IMAP is a bit harder but not really. Check out perdition and maildir and qmail or postfix or even sendmail if you know it well. This shit has been done over and over and over again. There is nothing even remotely complex in providing mail stores for 1mm users. THERE ARE FUCKING HOW-TO's to do it even. Okay, </rant>"

And I'm out. ;-)

-david