Slashdot Mirror
How Microsoft Degrades Their Users (In a Good Cause)
blackbearnh writes "We all know that slow Web pages drive users crazy, but where is the boundary between too slow and too simple? As Microsoft's Eric Schurman points out, the fastest-loading page of all is a blank one, but it's also the most useless. In an interview with O'Reilly Radar leading up to his appearance at the Velocity Conference, Schurman talks about his experiences working on some of Microsoft's highest-volume sites, including the home page and Live Search. In particular, he discusses how Microsoft will selectively degrade the performance of pages to small sets of users so that they can see how various amounts of delay at different times and places affect user behavior. 'In cases where we were giving what was a significantly degraded experience, the data moved to significance extremely quickly. We were able to tell when we delayed people's pages by more than half a second, and it was very obvious that this had a significant impact on users very quickly. We were able to turn off that experiment. The reasoning... was it helps us make a strong argument for how we can prioritize work on performance against work on other aspects of the site.' He also talks about what it's like to be one of the most often-targeted DDoS sites on the planet."
In this, there is also the possibility of becoming complacent and ill-tuned to the needs of your users. Taking Google as an example, they keep their services in a perpetual state of beta, always in testing, never reaching a final v1. This type of reliance on constant feedback from customers may work for a short while, but unless the product reaches a state of relative stability (in terms of both not crashing and also not changing) the users will typically find some other software to use.
You just disproved your own point.
Experimenting by delaying a pageload for 500ms is worthy of ethical considerations? Would you like to sue Microsoft for emotional damage? Too many people are afraid of doing anything these days.
You're discussing the wrong topic. "graceful degradation" != "selective worsening".
In Gmail, the degradation is helping you cope with a poor connection. In Microsoft's experiment, the degradation is hurting you and is imposed by a coin flip.
That would probably completely invalidate the results though, for two reasons. First, the sorts of people who would opt into that wouldn't at all be representative. (It would take an unusual person to even find the opt-in, let alone volunteer for a degraded experience knowingly.) Second, knowing about it would be way too likely to affect how the people behaved.
You could get halfway by saying "would you like to help us do research" or something like that, without saying in what way, which would reduce these problems, but not completely.
I think this would only actually be a problem if anyone used LIVE search.