Slashdot Mirror
Valve Finally Takes On Steam User Review Score Manipulation (eurogamer.net)
Valve is taking a step to stop developers from artificially inflating review scores on Steam. From a EuroGamer report: Valve just changed the way Steam user reviews work -- and it's certainly set the cat among the pigeons. In May, Valve updated Steam so that it highlighted recent reviews on games. The thinking behind this change was sound: it wanted to better show the current state of a game, many of which evolve quickly as developers issue updates. Now, though, Valve is changing the default review score that shows up at the top of each product page -- the one developers and potential customers put so much stock in -- so that it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.
... it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.
Although certainly a valiant effort, one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores. Perhaps that's a negligible portion of the total, but for some games, it may not be. For example, Humble frequently puts up indie bundles for a few dollars, including games that many people wouldn't necessarily buy individually on Steam (because of, for example, the lack of reviews). But at $10 for two games you want and three you've never heard of, you figure, why not? If you end up liking one of those games, your review won't matter... again making it difficult for hidden gems to get a foothold.
one of my initial thoughts as well.
I suppose a better way to deal with the problem is to throw out reviews that are tied to a clearly inactive steam account.
A person who actually uses steam will have recorded play histories and times. A bullshit ratings inflation service will have hundreds of dummy accounts that they use to inflate ratings with, and little to nothing else. If those accounts need actual play history, especially recent play history (given valve's stated goals with this to capture changing ratings over time), then the cost of these ratings inflation services will balloon.
That suggests an idea that they should be doing already, with data they already have access to: rather than providing a single rating score (or even two with "recent" and "overall"), provide a graph of average rating vs. time played. If the average score among people who've played it less than 20 minutes is 4 stars, but the average score among people who've played it two hours is 2 stars, that's a lot more indicative of rating inflation and what the real game is like... Conversely, if the average score among short-term users is low, but the score shoots up among people who stick with it, that may indicate a difficult learning curve that most people give up on, or may indicate that it's a niche title only for users really into that genre, etc., etc. Either way, it would be very useful information to have.