Online Reputation Is Hard To Do

Symblized writes "A new article from InformationWeek argues that not only does the Web need ways to verify identity, it also needs better ways to measure reputation . The article uses Digg, Wikipedia, and eBay as examples and muses whether their models could be applied more widely. There's also a profile of Opinity, a company that tried to introduce a reputation system and didn't make it. Choice quote from a source in the article: 'The idea of a transferable, semantic reputation is identity nirvana.'"

Comments

Trust is the currency

[Raindance]

"Trust is the currency of the participation age." -Jonathan Schwartz

This is the $64,000 question. Building a reputation/trust system is very difficult. Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

I'm volunteering at Citizendium, which is another possible datapoint. We're assuming that real names and respecting verifiable expertise will allow us to benefit in some fashion from existing scholarly reputation systems, and to build a more cohesive community.

Eventually, I think it'll be feasible to layer reputation and credentials (for sites that care) on top of a system like OpenID. People will be able to choose what reputation/credentials to share with which site. Information that you want to follow you (e.g., "I have a BA in Math from UCLA" or "I have excellent karma on Slashdot") will follow you across sites.

But yeah, it's a very difficult problem. Figuring it out is a big, potentially very lucrative issue.

Re:Trust is the currency

[zCyl]

Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).

That's because Slashdot's system puts only minimal emphasis on individuals, and very high emphasis on selected adjectives of value. Comments do not simply get moderated up or down, but have to be moderated with a chosen adjective, such as "insightful", "informative", "funny", etc. This really helps keep people's heads on straight, especially with the presence of meta-moderation, because people then have to agree on what these words mean. The end result is that posts are usually moderated in close proximity to these labels.

The karma attribute is used only as an accessory to this content-based moderation, to provide some inertia to the community's character. It's not really a reputation centered system.

And what do you buy with that currency?

[khasim]

It seems that everyone trying to "solve" this "problem" doesn't know what they're trying to achieve.

So what if you can make a perfect pseudonym identification system? What does that achieve for you? What do you accomplish beyond that?

Does it really matter to anyone else if your Slashdot 'nym can be verified to match your 'nym's on a dozen other boards? Who really cares if you have excellent karma on Slashdot?

Re:And what do you buy with that currency?

[Original Replica]

In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere. Just like if you act like an asshat at the company picnic, it effects you back in the office and possibly gets back to your friends at home. And yes, I think if someone is a carebear in WoW then they are a more trustworthy eBay seller, and someone with intelligent /. posts is more likely to contribute intelligently to Wikipedia.

Re:And what do you buy with that currency?

[Sigma 7]

In this anonymous online world, this would be an attempt to establish character. So in the future, acting like an asshat for fun in formus would relfect on you elsewhere. I'll play devil's advocate here...

Let's say you comment on the theory concerning the use of demolition explosives on one of the world trade centers - pointing out that the collapse of the WTC doesn't look like other building demolitions, or that the "symettric demolition" claim is incorrect.

However, the conspiracy theorists on the site are extremely fanatic about their theory (as opposed to a more moderate site that tries to investigate properly.) As a result, you receive a large quantity of negative feedback that attaches itself to your online reputation.

Other things that can affect you would be playing RTCW:ET, where you get kicked from a server for n00bism as you didn't dodge the three panzers that get fired into your local area (because another player thought you should have.)

And my personal favourite - just claim you support Bush. Your reputation would instantly tank.

Re:And what do you buy with that currency?

[Eivind]

Yes it matters. A lot.

Online, a lot of clues that are present in the real world are absent. We instinctively assign some level of trust to people we meet, for different purposes, based on a lot of variables, some of them we're aware of, others are subconscious.

You let your neighbour have your house-key to water the flower. But you wouldn't do that with *ANY* kind of neighbour.

You let someone babysite your kids. You let a friend borrow $50 'cos his credit-card is broken. You wouldn't to everyone, it's a matter of trust.

You trust someones judgement on some issue -- because you know that they are experts in the field and have a track-record of good judgement.

Being able to build trust in a pseudonym, and being able to prove that you are that pseudonym is very useful. It allows people to trust you who wouldn't otherwise.

To avoid abuse, it is nessecary that *you* have complete control over what aspects of your trust you share with which people and which companies.

So, what do you want to achieve ? World Domination offcourse ! *grin* No seriously, a million little small things, each of which may be unimportant, but the sum could be huge. Some examples used *today* include:

• If you've got excellent reputation, Ebay-buyers generally won't mind paying first, getting the item(s) afterwards. This would be quite risky -- except you know that the seller has sold 471 things before on similar terms, and -zero- of his customers complained.
• Hospitality-club use a trust-system to allow you to let complete strangers sleep over at your house, or vice versa, with a much reduced risk of any unwanted problems. Sure, *you* may not know this person, but it helps if 50 other people do -- especially if 5 of those are your friends.
• Slashdot use a trust-metric to let people with a track-record of sane comments be sligthly more visible.

In a universally networked world (which we're rapidly approaching anyway) with strong trust-systems, you could stop a complete stranger on the street and ask to borrow his car -- and he'd actually consider it. He wouldn't know *you* but, he'd be able to know a lot *about* you -- if you choose to share it with him.

Online reputation? All reputation is hard to do

[Colin Smith]

It's why we have exams, professional organisations, CVs, brands, social networking etc etc etc.

We use reputation all the time and no-one has come up with a single reliable, coherent way of measuring it. You just try to get a decent builder.

Re:To clarify that ...

[dkf]

What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).

You achieve all the same benefits without the problems just by having your own reputation/moderation system.

I suspect that there will be a lot of correlation between sites, as people who act like asses in one place are more likely to do so elsewhere. On the other hand, I also think that what will actually emerge is that the sites that correlate cluster in groups: if you have a good rep on site A, you're more likely to have a good rep on site B and a bad rep on site C, and you probably won't frequent D at all. Given all that, someone with a good search engine and a lot of cleverness will be able to start mapping out what these virtual communities actually look like and other sociological/anthropological stuff like that.

But will sharing reputation systems help sites? I think so, eventually, but not for many years yet. Until the reputation clusters have been found, sharing is as likely to introduce needless pollution of reputation systems as it is to enable reputations to be built up quickly. (Could there be a single reputation cluster? Maybe, but I suspect not; people are too inclined to divide the world into "us" and "them" for it to work out.)

One thing that might come out of reputation research is that it might become possible to use the reputation clusters to predict, from someone's interests and interactions, which sorts of sites they'd like to visit. OK, that does sound backwards, but it should guide people to where they won't want to make a total fool of themselves on a regular basis (yes, even the griefing pranksters; after all, when amongst the fools the foolish are sages and the wise foolish.) It may also eventually be possible to join the reputation clusters up, but using negative links (so reputations on sites for followers of Xenu who believe in ID and the supremacy of feng shui of placement of feeding bowls for their chihuahua will negatively reinforce reputations here) but I doubt that will help any time soon. There's a revolution waiting to happen here, but since it really involves lots of people, it'll take time to brew.

On the other hand, it is sensible to start working out what technological steps are required to enable specific bipartite reputation sharing, as well as looking at how to build sane single-site reputation systems. For example, slashdot's is pretty good in that it isn't easy to totally game the system while being mostly self-regulating, but can it be bettered without input of data from outside sites? If it can't be greatly improved, how difficult will it be to export the system to other sites? (It's late: I'm sure you can think of other aspects, but I can't right now.)