Are Website Performance Metrics Still Relevant?

jackvalko asks: "Recently Keynote Systems began an upgrade of Transaction Perspective, one of their performance measuring products. The data collected is used by the Executive staff at the dotcom I work for as a means to evaluate our customer experience. Now that they are almost done, we've noticed a better than 40% reduction in response time of our site. While I'm happy that our performance 'looks' good, this major change has given us pause to question the statistical relevance of the data that is being collected and the somewhat arbitrary nature of performance metrics collection in general. And, of course, how we message all of this to upper management. So I put it to you, Slashdot! Do you find performance metrics relevant to your customer experience? How do you manage expectations and change to upper management? And, most importantly perhaps, are organizations that collect this data still relevant on The Internet as it exists today?"