Slashdot Mirror
Valve Finally Takes On Steam User Review Score Manipulation (eurogamer.net)
Valve is taking a step to stop developers from artificially inflating review scores on Steam. From a EuroGamer report: Valve just changed the way Steam user reviews work -- and it's certainly set the cat among the pigeons. In May, Valve updated Steam so that it highlighted recent reviews on games. The thinking behind this change was sound: it wanted to better show the current state of a game, many of which evolve quickly as developers issue updates. Now, though, Valve is changing the default review score that shows up at the top of each product page -- the one developers and potential customers put so much stock in -- so that it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.
... it does not include reviews written by those who obtained the product through a Steam key. What this means is that reviews penned by those who got a game after backing it on Kickstarter, for example, or via a developer's website, do not affect the Steam user review score. Again, the thinking behind this change is sound. Valve knows that some developers were gaming the system -- that is, they were giving keys to friends or shady paid services in exchange for positive reviews.
Although certainly a valiant effort, one unintended result is that it will ignore reviews from people who purchase keys via Humble Bundle or other third-party stores. Perhaps that's a negligible portion of the total, but for some games, it may not be. For example, Humble frequently puts up indie bundles for a few dollars, including games that many people wouldn't necessarily buy individually on Steam (because of, for example, the lack of reviews). But at $10 for two games you want and three you've never heard of, you figure, why not? If you end up liking one of those games, your review won't matter... again making it difficult for hidden gems to get a foothold.
Good. Fuck humble bundle for selling out and selling steam keys instead of DRM-free games.
I got burned on the recent Sierra bundle, because why the fuck would old-school games NOT be DRM-free? I didn't even look for the steam/DRM-free icons, because there was no reason for me to assume anything other than DRM-free. I already HAVE all of them DRM-free, FFS!
I immediately realised after I had paid that they were only selling keys and not games, and demanded my money back.
one of my initial thoughts as well.
I suppose a better way to deal with the problem is to throw out reviews that are tied to a clearly inactive steam account.
A person who actually uses steam will have recorded play histories and times. A bullshit ratings inflation service will have hundreds of dummy accounts that they use to inflate ratings with, and little to nothing else. If those accounts need actual play history, especially recent play history (given valve's stated goals with this to capture changing ratings over time), then the cost of these ratings inflation services will balloon.
That suggests an idea that they should be doing already, with data they already have access to: rather than providing a single rating score (or even two with "recent" and "overall"), provide a graph of average rating vs. time played. If the average score among people who've played it less than 20 minutes is 4 stars, but the average score among people who've played it two hours is 2 stars, that's a lot more indicative of rating inflation and what the real game is like... Conversely, if the average score among short-term users is low, but the score shoots up among people who stick with it, that may indicate a difficult learning curve that most people give up on, or may indicate that it's a niche title only for users really into that genre, etc., etc. Either way, it would be very useful information to have.
Indie games and Humble Bundles. There are plenty of Indie games that don't have shit for reviews until they are included in a bundle because they simply got lost in a crowd. With these changes even if the majority says its a really fun game their opinion won't count if it was part of a Humble Bundle which is just bullshit, especially when a lot of indie games get shit on at release because of bugs that may have been fixed ages ago so those old reviews no longer reflect the current game.
If they want to do this with rinky dink sites or keys coming straight from the devs? Well and good but the HBs are pretty damned well known and hugely popular so I don't think they should be affected by this rule as it kills a lot of legit reviews from counting.
And for anybody that says "it shouldn't count because the HBs are cheap" well why stop there? Why not ban those reviews from Steam sales counting in the overall score and only count those that paid full price?
ACs don't waste your time replying, your posts are never seen by me.
If the devs promised all of heaven and earth to kickstarter, but only deliver a decent game ... original backers are going to slam it.
I'd argue that is information relevant to the review though because those same devs may be continuing to claim they will deliver amazing new features etc. in the future and so those contemplating a purchase should know that they have a history of not delivering.
The best solution is to come up with what you think is a better algorithm and then display BOTH results. This way people can judge for themselves which score is most useful in a particular circumstance. In fact this would give you some idea which games devs were manipulating their reviews which is very useful information if you are considering a purchase.
I love No Man's Sky. I haven't played it at all, but it's responsible for this, which is probably more amusement than I've gotten out of some games I've actually bought.