Slashdot Mirror

← Back to Stories (view on slashdot.org)

Customer Feedback Surveys Could Be Considered Harmful (easydns.org)

Posted by BeauHD on from the scratch-beneath-the-surface dept.

Longtime Slashdot reader Stunt Pope writes: Customer Feedback surveys are now near-ubiquitous, subjecting us all to near-Black Mirror-esque pursuit to "rate your experience" for everything from going to the bank to ordering a pizza. Thanks to The Curse of Goodhart's Law, all of these surveys are beyond useless and even damaging. Mark Jeftovic writes in a blog post: "The shop/hire-rate-reward feedback loop has become baked-in to some systems. Many live marketplaces incorporate these feedback transactions into ratings, which then become a score which then impacts future prospects of whomever is being rated. And that's where the trouble starts. There is a point where this stops being useful and the knock-on effects of a ratings system predicated on feedback results becomes counter-productive. That point is when the ratings become targets. When a company decrees 'All customer feedback ratings must score a minimum of X, or else...' the company has just commenced the process of invalidating and corrupting all useful information to be gleaned from that feedback/survey process. A label which captures this concept is 'Goodhart's Law' -- after economist Charles Goodhart, who posited in essence that 'when a measure becomes a target, it becomes useless.'"

3 of 196 comments (clear)

  1. Most forms of metric are like this by Zephyn · · Score: 5, Insightful

    " A label which captures this concept is 'Goodhart's Law' -- after economist Charles Goodhart, who posited in essence that 'when a measure becomes a target, it becomes useless.'"

    I've seen a similar effect in places where I've worked. A poorly defined metric that is used to rate employee performance will suddenly become the primary focus of the job, instead of actually doing the job.

  2. I've seen it in action by npslider · · Score: 5, Insightful

    An experience at a local biz in town with a customer service rep... Was told that anything less than 5 out of 5 on his customer review is considered a bad review, and he all but begged me to give him 5-stars.

    He was so overly friendly it was past creepy. I felt conflicted: he did a good job, but I felt I was rating for his sake, not to give an honest assessment of how well I was served by him.

  3. Re:agreed by Nkwe · · Score: 5, Insightful

    And as a person filling out surveys who knows (just a little) about math and statistics, I think of ratings on a bell curve. On a 1-10 scale almost nothing is actually a 1 or a 10. On that scale I would rate a 5 as average service and give a 7 or 8 to what I think is well above average service, 9 would be excellent service. You would only get a 10 if there was no possible way to do any better under any circumstances and you completely exceeded all of my expectations. Unfortunately people get dinged if they don't get all 10s. Sucks to be you if I have to fill out your survey.