Slashdot Mirror
Designing a MMORPG Feedback System
Gamasutra is running one of their highly enjoyable 'soapbox' pieces, looking at possible ways you could implement a feedback or ranking system for Massively Multiplayer Online Game players. From the article: "When playing an MMORPG, I should be able to give a positive, neutral, or negative rating to anyone who has been in my group for more than thirty minutes. Negative ratings could be characterized via a multiple-choice list of common gripes (i.e. 'loot theft', 'abusive language', etc) -- a feature now built into the Xbox Live feedback system. However, it isn't clear that a good feedback system requires this level of depth; there's an argument to be made for simplifying the process as much as possible."
Just a Slashdot-style moderation system, where a player can be modded up and down (one vote per person, which can be up/neutral/down) according to that player's action.
These mods will "expire" after a couple of days so that players can start a new leaf.
Virtual Betting on Facebook for non-geeks.
Karma is king.
liqbase
well the problem with most rating systems used is that they could be abused... this sounds good though, since you would need to be in a party for a while...
FYI - this was tried in Ragnarok Online but it was taken off due to abuse. People would pay (in game) money for positive points, and you'd have people who would stalk you and give negatives at every opertunity. You could only give 1 point per day, but it was still abused by people having multiple characters on one account or multiple accounts. It opened people up to a lot of threats as well, as a planned implementation of the system would have been more chance to drop equipment when you died.
By reading this, you have given me brief control of your mind.
"I understand concerns that a reputation system might drive away paying customers who get bad ratings..."
:o)
Has bad Karma ever driven a Slashdotter away?? These types of community style initiatives usually work out pretty well and generally add to value to the experience but I don't think it would drive me over the pay/don't pay edge.
Homer: Facts are meaningless, you can use facts to prove anything that's remotely true!
Any such player-run system would be too easy to abuse. If you put restrictions in place to prevent the abuse, people will game their way around them.
For example, a guild could have all of its members give each other high ratings. Or they could band together to give poor ratings to someone that pisses them off.
Now that you mention it, I do believe we did.
:-)
The result of the conversation was that a rank system could be implemented similar to that of the Military ranks. Rather than working your way up some sort of point system, you'd get your promotions directly from your CO. (That's Commanding Officer for those of you who aren't familiar with the term.) High ranks would be seeded by the "Admiralty" (i.e. The people running the game), thus kicking things off. And if you really don't want to follow the whole rank system, you can always privateer.
At least, it seems to work in the context of Star Trek and military SciFi. YMMV elsewhere.
Javascript + Nintendo DSi = DSiCade
I play Gemstone, an older (to put it lightly) text-based MMORPG. In it, someone's "popularity" or whatever comes from interacting with other characters etc. Having a game mechanic to tell you if someone was a jerk to someone else seems very very strange, and would definetly be out of character. So, for "role playing" games I don't feel like that is something that's useful or good. On the other hand, if the game has on objective, and you are trying to "win" WoW or whatever, I guess having that feedback system could save you time and frustration. I guess one way to rationalize this in character is to have some little cottage where you would go in and greet a jolly fat man in a little red suit who keeps track of who's "naughty" or "nice".
You should use a distributed trust metric system like Advogato. This allows you to develop a hierarchical system for rating peers in a community.
Nooface
In Search of the Post-PC Interface
"/target: age?"
And I'm good to go.
An excellent idea
Yes its a way to keep track of the people who play but also it builds up a stigma that a big red X shall be assigned to me by anyone who is unhappy with my performance... and what is the life cycle of such an X and the what if the good fairy comes by and gives you a + for whatever reason, which do I believe and then what about the entire lables thing.. Isn't that what we are trying to get away from in the digital realms? Away from lables and classifications... man sounds like school and citizenship. Class clowns were notorious for bad citizenship but a hoot to watch!
I should be able to give a positive, neutral, or negative rating to anyone who has been in my group for more than thirty minutes
Hypothetical situation. I form a group, recruit people who want to do (insert dungeon here). Halfway through the run I realize one of the member is incompetent. I remove him and flag him "incompetent". Out of spite he returns the favor and flags me something negative. Yet I've only done what was best for the success of the run, I wasn't using abusive language, I didn't do anything wrong.
Another problem is when there are games where you can reach max level fast. Blacklist the assholes all you want, they will remake as different characters. UO murderer system is perfect example where griefers would level throwaway characters and PK until they got caught. Rinse, repeat.
Another problem is with raid guilds. From experience, raid guilds will recruit the biggest assholes on the server under two condition: They obey the raid leaders and have insanely high play time. Raid guilds don't care if you are an asshole. They care only if you can contribute to their success.
One of my gripes with WoW is the inability to quickly flag someone as a gold-seller, user of inappropriate language (sexist, homophobic, racist, etc.), exploiters, etc.
Too often will I receive a whisper from a character trying have me visit a gold for $ site. I will report them but the process is not stream-lined, it takes a few minutes. If multiple people could quickly flag this character as a gold-seller then blizzard reps would be able to investigate and perma-ban the most abusive accounts.
How do I level up?
You're an immobile computer, remember?
Especially the fucking rule about abusive language. It's lame.
'Nuff' asked.
it's called /yell in Ironforge. Seriously though, A MMORPG is such because it's an immersive world where people not only quest for 'phat lewtz' but spend a good amount of time sitting around, idling and chatting. I don't need a feedback system to tell me that X is a ninja or that Y is an excellent healer or tank that I would be happy to have in my party, I know from experience and communication. Jeez it almost sounds like handing out gold stars.
Plus a system like that is open to massive abuse, especially when you put it into the hands of a bunch of 14 year old boys who decide that they don't like a particular player. Sounds like another shitty rep grind. no bloody way.
Promote Charity on Myspace, Show Your Colours!
The key, then is to not give out absolute ratings, but rather relative ones. The rating you personally see for people you haven't rated yourself should be based on how they're rated by people you yourself have rated highly. This goes a long way towards preventing abuse by effectively negating the ratings assigned by people you don't like.
"A++++++++++, great raider. Would group with him again."
"Perfect, now hand over the Scimitar of the Wolf, or my whole guild will give you retaliatory feedback."
https://www.eff.org/https-everywhere
Any company that is able to guarantee a favorable experience in an online game is going to make a lot of money. It's not just MMORPGs that need this, but FPS titles as well. How many times have you played a team-based FPS (like Counter-Strike), only to run into some asshole who thinks that shooting you in the back constantly is "funny"? Many games have tried changing the game mechanic to prevent this from happening (for example, if you cause enough team kills, you get booted from the server), but a lot of those mechanics can be gamed. Or it doesn't do anything about the fact that you had to be TKed three times before the griefer got booted, etc.
... and vice versa)
Much of the problem comes from the fact that griefers find entertainment in doing the griefing. I happen to know someone who regularly did this in Battlefield games. I asked him, "Why are you such an asshole in the game?" His answer was because he's not very good at them, but he just finds it hilarious the feedback/anger he gets from those he griefs. I've seen the same thing in WoW, where notorious spawn campers, who feed on lower level newbies, delight in the grief that they cause and will even post screenshots on a website of their "conquests".
I've always thought that the Slashdot moderation system is a good start for such a feedback mechanism in games. Obviously, though, it needs a lot of extra work to make it appropriate for a game (and TFA brings up many good points). Better yet, some company needs to be able to make it an overall game service that can be used across a broad spectrum of games. If you're an asshole in one game, you're probably going to be an asshole in another game. If not, then hopefully it's a deterrent that if you like game X, you can't be an asshole in game Y since it'll affect your karma/rating.
You can even extend it beyond just "I don't want to play with griefers". I know role playing enthusiasts would like to be able to rate folks as a "good RPer" or a "bad RPer", so they know who to RP with. Or perhaps ratings like, "Good at PvP" versus "Bad at PvP". (I know plenty of folks who I'd love to group with, when tackling regular monsters/dungeons in WoW, but I know they're horrible when it comes to actual PvP combat
I, for one, would gladly pay for such a service, and I'm sure I'm not alone. Of course, there'd be a huge backlash from those folks who ARE the assholes to begin with, and also from folks who are generally good players, but object to having labels being slapped on them and being rated constantly (much like the popularity contest that is high school).
-- jchenx
What about a system like Fable had? Not necessarily changing your appearance, but your fame (or infamy) was automatically calculated by the game, and NPC's react differently based on your deeds. Say the system is based on points; playing all the way through cooperatively in a quest gave so many positive, healing teammates, etc. On the flip-side, ninjalooting etc. would be caught by the game mechanic and weighed accordingly. This "karma" system would be automatic and players could say, view a few stats about a player's reputation, without any necessary player input. And then there would always be the opportunity for a player to redeem their poor reputation.
--- Though lovers be lost, love shall not; And death shall have no dominion -Lem
None of this will matter until there are real consequences to anti-social behavior. That requires a couple things. First, you need the people running the game to care enough to *publicly* boot people for being asshats on a regular basis. Secondly, once that is done, you need a way to prevent the asshat from just opening another account or creating another character.
I've thought for a long time what the MM market needed was a third-party identity broker. Call it 'Good Guys, Inc'. You apply for a GGI account. GGI does a check, verifies you are who you say you are, address, SSN, whatever, and issues you a GGI #.
You buy Worlds of Evercrack, and enter the CD key that came with the game into your GGI account. You login to a GGI-enabled game server, and it uses the gamekey to lookup your GGI account and verifies that that key belongs to a member in good standing. Or, if you've been banned (from that game, or anther game, or whatever they want to check), it boots you or forces you to play on a server comprised of other asshats.
Not infallible, but maybe if it booted the jerks (and ninja-looters, and bots, and pro farmers) from not only that game, but potentially other games that used the service as well, things might start to improve. Or at least segregate the people who want to play the game from those that want to beat up on other players.
Feedback systems can always be exploited in some way. Nobody would want to give feedback, even if the a$$hole in the group deserves it, in fear of getting negative feedback in return. You'd have every player ask for positive feedback after every dungeon run, which would be annoying, and it would be hard to say "you did alright, but not quite good enough to deserve a positive rating" to mediocre players you got along, but didn't really have a great time, with. Honestly evaluating people may offend them.
eBay's simpler than a social game. On eBay, you're happy if you get your stuff in the mailbox on time, or if the buyer pays you prompty - that's more or less all there's too the interaction. In a MMORPG, there are way more variables which make up how happy you are with somebody, and so the scale is much finer. If you'd keep the MMORPG feedback system simple (with, say, negative, neutral, and positive), then everyone will expect positive feedback (like on eBay), and the system wouldn't give much indication as to how good (skilled, polite, helpful etc.) the players really are. Make it too complex, and you'll have the problem with players getting pissed off at not getting MAX/MAX points at the end of every group. Also, you'd probably end up having guilds unbiasly give each other feedback, as somebody else already said...
A system like this could be helpful. People would know and remember who the ninja looters are, but it could also be abused (false "ninjalooter!!" accusations could be made" - although maybe this could be solved like on eBay, with a function for being able to comment on some feedback one has received to defend ones self against such false accusations). In the end, word of mouth of who the nice guys and bad guys are usually gets around, although not to everyone and not very quickly.
Developing a feedback system is probably viewed as merely a nice little bonus feature by developers, and thus isn't something MMORPGs have. It's hard to predict how well such a system will perform and how useful it really will be in practice, so implementing it is a bit risky.
Just in MMORPGs? I think we need to begin taking more actions to rate people we interact with in everyday life.
Once again, I brink up Eve Online on a subject under games...I swear I don't work for them, tho. :P
They have two great means of this. 1, players can set standings for how they view any player, corporation (think guild), alliance, faction, or what have you. The can rate some(one/thing) on a scale of -10 to +10. At any time you can look at another character's standings to see what PC's, NPC's, Corps, factions, alliances, etc., like or dislike them. Also, committing illegal acts in secure / semi-secure space results in a decreased standing with Concorde (NPC Police), which gives a highlight to their nick. Eventually, if that standing gets too low, the police will shoot on sight if the person is found in too high a security zone, depending on how bad the standing is.
The second way that you can deal with people you don't like is with bounties. If you've got the dough, you can place a bounty on somebody's head and it will be visible to all players that they are wanted. If they have good (or not horrible, at least) standing with Concorde, shooting them will still be illegal in secure space, but for the right price...well, you get the idea. You would think that this feature would be heavily abused, but in fact it's not. It's not especially difficult to get a bounty off your head (just have somebody kill you...note however that clones get expensive, so it's only worth it if the bounty is higher than the cost of your clone), but it is still a moderately effective tool.
Another interesting note is that some have mentioned loot thieves. As of last expansion, if you loot something from somebody else's kill, they have kill rights on you for the next hour, regardless of security level. And last of note, unless you are in the most secure (and therefore, least profitable / exciting) sectors, it is not overly difficult for somebody to kill you if you piss of the wrong person. Oh, and did I mention PC corps can declare war on each other, and thereafter kill opposing corp members, regardless of security level? Mmmyesh. All in all, the game deals beautifully with asshats.
Unpleasantries.
Slashdot karma has no particular affect on my enjoyment of the website.
If I'm playing a game, and I have a red flashing light over my head that says "LOOT THIEF", that, however, will impact my opportunities within the game, and my enjoyment of it -- especially if I'm paying money for the privilege.
The problem is that no part of a game 'regulates' gameplay, as if from heaven -- it only changes the nature of the game being played. Lots of people will be delighted to discover the "apportioning blame" sub-game ('tag' would be the basic archetype).
Basically, different people think different things are good and/or bad in another person's playstyle.
Examples:
You are a person that likes to take their time going through a dungeon/instance/raid. You want to be thorough. However, you aren't able to find a group with your regular friends, so you do (oh, the horrors!) a PUG. The people you get stuck with are of the OMGZERS HURRY UP! mode. You stay with them, because you don't want to get a horrible score for leaving too early. Your play styles clash too much for it to be a friendly group, and 5 other people give you a negative rating.
You are in a group, and you normally play a laid back style. Your wife is sick, but is letting you play. You get in a group, and an hour into the session with a PUG, you wife tells you she is really ill, and wants to go to the hospital. Half the group is adult and understands. The other half gives you a negative mark for leaving before it is over.
You are playing a game that allows PvP. Your party encounters some griefers that manage to kill you all after you had gotten along easily for the last hour. The other party members want to gank back right away, but you want to finish the mission. They rate you all negatively.
3 examples, all possible. There are hundreds more. And most folks just aren't capable of giving an honest opinion. Some 14yr old little pissant isn't going to care that your wife is pregnant and about to have the baby a month earlier than expected. You ruined their gaming experience by having the audacity to have a family emergency. They are going to rate you negatively.
Also, you could stack the ballot box, in effect. Don't like another player? Get friends to group with that person, and slam them.
Lastly, people can and will create griefer toons. Most games allow some levelling to be done solo, if not all. In WoW, I could solo a druid or other toon to lvl 10 in less than 6 hours. I can hit 20 in less than 20 hours. I can then go about with that character and make other players live's miserable. Then delete the toon and start again. Too much rep on that server? Pick another one.
I don't see it working. It's too easy to change your identity. And there's no way I want other folks knowing my account name. They don't need it, but if they get it, someone will eventually attempt to hack your account.
Word of mouth on the forums works. Heck my WoW realm forum was full of "This person is a ninja-looter" threads, and people started to keep list of confirmed ninja-looters. And all it would take is one group all saying "Toon A ninja-looted", and even if the lied, you were now blackballed.
So, why bother giving another avenue to grief, cause problems, and the like?
Wasn't this tried in The Sims Online, where it resulted in the community being run by a mafia of players coercing people into following their orders through threats of negative feedback?
A safegard against this kind of behaviour is easy. Set it to where they have to be grouped with you for a period of say, 15 rl minutes, before they can give feedback. Since most raids take MUCH longer, it could be a deturrent. You can also set a threshold to 1 rating per day, per account. In other words, you may not leave more than 1 feedback for a particular account, on a particular account. On top of that, when someone is kicked from a raid/group/party/whatever, the leader may then be presented with the option of leaving feedback, and the leader and group be exempt from being rated buy the kicked member (e.g. if you are kicked, you may not leave feedback for the group experience. Of course, in order to be fair about it, you can't let the whole group gang up and rate the kicked player negativey, so the only rating that player gets is from the leader.
I think that would help keep the revenge rating to a minimum.
I've been running an online game for five years now (see .sig). During that time I've experimented with several feedback systems.
None of them work as they should. The problem is that there is too much abuse in the system. People who want to game a system will always find a way to do so, and it is very hard to design a system that is resilient against that.
Simple example: Everyone can give everyone else a score from 0 (worst) to 10 (best).
Theory: While there are jerks who give bullshit scores, it will average out.
Real life: Most people won't care enough, even if it's just one click. The jerks will dominate the system because if the "fair" average is 5, and one jerk gives you 0, it takes five scores of 6 to counter that. Since jerks are active and normal people are lazy, your chances of getting those five scores are lower than your chances of getting another 0.
Same with more refined system. I had a system that didn't use scores but keywords like "reliable", "cheater", etc. running for a while. That, too, was abused. People were labeled "cheater" not because they were actually suspected of cheating, but because some jerks didn't like what they did. Heck, several of the game designers were labeled cheaters.
It's a really, really difficult problem. Trust flow systems are the only thing I believe would work, but they give a very subjective view (essentially what your friends think of someone) and a lot of people don't really understand them so coding and interface that everyone can use is another tough problem. Aside from the fact that the entire area is pretty much in beta stage still and there's very few code available, so unless you're a math and coding wizard and can create your own trust flow system, that's not something really available right now.
At the moment, what I do is having no system. I rely on the human beings in the game to be able to handle things themselves, using their brains as tools that are infinitly better than anything I could ever code.
Assorted stuff I do sometimes: Lemuria.org
in wow ther is a freind list and an ignore list.. people you dont like go on the ignore list... its that simple.
I thought it a bit interesting to compare this to how things worked out in EQ.
When EQ first opened, i noticed alot of jerks, and i wrote their names down. As i levelled up, these jerks all dropped out.
You see, you needed to be in a group. And if someone asked to be in a group, and someone didn't like them, they weren't let in. Deprived of levelling opportunities, they dropped out.
By the time you were in the 40's (this was when the max was 50), even being in the wrong guild would be enough to black flag you. Our guild would watch an applicant (grouping with them while playing an unguilded alt), to see if they were friendly.
This all grew from the 'forced grouping' that EQ had. The more solo friendly a game is, the more jerks you will find in the game.
-Eric
SJW: Someone who has run out of real oppression, and has to fake it.