Slashdot Mirror
Fixing Steam's User Rating Charts
lars_doucet writes: Steam's new search page lets you sort by "user rating," but the algorithm they're using is broken. For instance, a DLC pack with a single positive review appears above a major game with a 74% score and 15,000+ ratings.
The current "user rating" ranking system seems to divide everything into big semantic buckets ("Overwhelmingly Positive", "Positive", "Mixed", etc.), stack those in order, then sort each bucket's contents by the total number of reviews per game. Given that Steam reviews skew massively positive, (about half are "very positive" or higher), this is virtually indistinguishable from a standard "most popular" chart.
Luckily, there's a known solution to this problem — use statistical sampling to account for disparate numbers of user reviews, which gives "hidden gems" with statistically significant high positive ratings, but less popularity, a fighting chance against games that are already dominating the charts.
The current "user rating" ranking system seems to divide everything into big semantic buckets ("Overwhelmingly Positive", "Positive", "Mixed", etc.), stack those in order, then sort each bucket's contents by the total number of reviews per game. Given that Steam reviews skew massively positive, (about half are "very positive" or higher), this is virtually indistinguishable from a standard "most popular" chart.
Luckily, there's a known solution to this problem — use statistical sampling to account for disparate numbers of user reviews, which gives "hidden gems" with statistically significant high positive ratings, but less popularity, a fighting chance against games that are already dominating the charts.
As much as people forget, Valve are not in this to make gaming a better place. They're there to make gobs of money, and have been rather successful at doing so thus far.
Valve makes gobs of money *because* they make gaming a better place.
Valve is the sole reason why I'm probably not buying a console this generation, or maybe ever again. Gaming is so much better on PC these days that it just doesn't make sense to lock yourself into the console market anymore. And that's all on Valve.
So you might be cynical and say Valve only cares about money, but the fact of the matter is that in order to generate that money they need a healthy market. Their interests are aligned with ours.
Considering the sort of talent they hire and have hired in the past, if they truly wanted to fix things, they'd be fixed. If they're not, they either don't consider it important or have a reason for not fixing it.
Maybe they've just got better things to do. Valve has their hands on so many pies that I'm sure they could double their workforce tomorrow and still keep everyone occupied.
The new recommendations system is still new, I'm sure it's under close scrutiny and updates will be coming. But as someone else said, it'll happen on Valve Time.