Slashdot Mirror
Online Reputation Is Hard To Do
Symblized writes "A new article from InformationWeek argues that not only does the Web need ways to verify identity, it also needs better ways to measure reputation . The article uses Digg, Wikipedia, and eBay as examples and muses whether their models could be applied more widely. There's also a profile of Opinity, a company that tried to introduce a reputation system and didn't make it. Choice quote from a source in the article: 'The idea of a transferable, semantic reputation is identity nirvana.'"
"Trust is the currency of the participation age." -Jonathan Schwartz
This is the $64,000 question. Building a reputation/trust system is very difficult. Honestly, Slashdot is one of the better examples of this (Slashdot's moderation system does alter the flow of the discussion but it does get a downright reasonable signal-to-noise ratio vs other online communities).
I'm volunteering at Citizendium, which is another possible datapoint. We're assuming that real names and respecting verifiable expertise will allow us to benefit in some fashion from existing scholarly reputation systems, and to build a more cohesive community.
Eventually, I think it'll be feasible to layer reputation and credentials (for sites that care) on top of a system like OpenID. People will be able to choose what reputation/credentials to share with which site. Information that you want to follow you (e.g., "I have a BA in Math from UCLA" or "I have excellent karma on Slashdot") will follow you across sites.
But yeah, it's a very difficult problem. Figuring it out is a big, potentially very lucrative issue.
I think Wikipedia is a site that really needs to somehow integrate the reputation of it's contributors into the articles. I haven't kept up with the structural changes they've made in the past couple years, but a lot of the editing work seems to be undoing trolling and vandalism, and also participating in edit and revision wars. I could be wrong at this point.
But if wikipedia had a reputation system ( other than just being banned or allowed ), they might automate contributions from reputable authors ( and check on the actual contributions later), while authors who are less reputable may have their contributions queued for review before they are published.
Furthermore, a casual user would be able to have a more savvy understanding of the reputability of any article or section of an article if it is tagged with the reputation of its' author.
Reputable authors might be able to also tag the contributions of others, such that the text or information itself gets a reputation. That would help users make a judgement about the validity of information on Wikipedia.
Instead of pushing the mechanics of the actual editing of articles behind the scenes, and just presenting a 'final' article to the end-user, let's formalize the process and enfranchise users into the process of judging the validity of articles.
Computers are useless. They can only give you answers.
-- Pablo Picasso
The only reason online reputation is hard is because online identification is hard. Once you're past the identification and privacy issues you could go Google: your single/central/one point rated identity, linked with all your accounts from all over the place which should give you some sort of a global and more specified ranking (karma on ./, trustworthiness on ebay, whatever rating/googlerank on google/amazon) for people to search for.
Even still, it is hard to rank someones reputation based on a numbers system. In alot of forums I post on, I am a regular poster with a high rank. However, alot of people have an issue with me because of my free speech (the beauty of the internet). Is there really a standardized way to determine reputation? It really has to do with the context. If you are on a programming forum, you may rank someone based on their aptitude for a specific language, or their problem-solving skills. Conversely, if you are on a political debate forum, ones reputation may be based on how fluently their opinions are expressed.
The problem is that everyone wants to know right now who is trustworthy and who is not.
Building a reputation takes time, often a lot of time. Amazon's reputation is built on several years of good service, good web design, and overwhelmingly positive customer experiences.
Facebook and Digg don't have that track record, and until they do will not enjoy the same level of trust.
Any system designed to give a stamp of approval needs only one mistake to become untrustworthy. Unless it can be nearly 100% foolproof it won't be effective. And given the number of supposedly trustworthy businesses who are anything but, I'd say that rating reputation is not likely to happen soon.
Three Squirrels
It seems that everyone trying to "solve" this "problem" doesn't know what they're trying to achieve.
So what if you can make a perfect pseudonym identification system? What does that achieve for you? What do you accomplish beyond that?
Does it really matter to anyone else if your Slashdot 'nym can be verified to match your 'nym's on a dozen other boards? Who really cares if you have excellent karma on Slashdot?
I haven't used my name for posting on the Internet since 1997 when I realized that dejanews.com would keep my newsgroup postings forever (even that was with a somewhat random e-mail address). I literally don't have any internet presence with my real name unless it's inadvertent (ie. a news release from my employer) but the good thing is that my name is so common I would be hard to find anyway.
So in it's place, I created a whole shitload of false identities that I post under, one of them about 10 years old now. Mainly on forums and newsgroups for work purposes, etc. If you searched for this particular identity, you would probably fine hundreds of posts (including many on slashdot) some of them truthful, some of them fake, with various opinions of topics.
Every few years I will discard an identity or create a new one, for various reasons. I even have a fake lj blog that I've created just for the purpose of having that sense of "credibility", just in case I need it. I usually update that every few weeks, with something that I read on someone else's blog, but changing the words around just enough so that I can't be googled and exposed as a fake. I make sure each identity has a different way of typing, different levels of typos or capitalization, etc. I don't think you would be able to properly gauge the "credibility" of this person at all.
I doubt I'm unique and there are probably scores of people doing the same thing. As internet users get more and more sophisticated, how will internet credibility really be gauged unless you actually meet someone face-to-face? I was even contemplating getting a pay-as-you-go cellphone with no traceability (paid with cash at a store in a different city than where I live) just in case I needed to talk with someone offline. I'm doubtful you can really establish credibility to the point where it's better to just assume that everyone is lying and be on the guard all the time.
Indeed! Here's my own anecdote on that: I recently tried editing an article on a certain Posada Carriles, a man whom the Cuban government and Wikipedia call a "terrorist".
I was browsing the Cuban government site Granma where they had a list of what they called evidence against Posada. One item was an AK-47 rifle, another item was a box of 5.56mm ammo for that rifle. It doesn't take much of gun expertise to know that NATO ammo doesn't go into an AK-47, and I tried to put that in a paragraph criticizing the accusations against Posada. I don't know the guy, for all I know he could really be a terrorist, but you aren't going to convict anyone in a civilized court of law with that kind of "evidence".
I was thoroughly flamed by someone about that. It seems that Cuban government sympathizers are carefully patrolling any critical statements about the dictatorship. If Wikipedia had a reputation system, the commies would mod me down for presenting a balanced view in their rant against Posada, but I would recover my karma through my other contributions. OTOH, fanatics would find it too troublesome to fake an interest in subjects other than their favorite and their karma would suffer from that.
The article claims to be about reputation but mostly talks about the various "identity" efforts out there. Yes, a reputation is associated with an identity, but most of the identity systems being promoted focus on real identity rather than pseudonyms which you can choose to associate with yourself or not.
t
There is a paradox to those systems -- the easier they are to use, the more they will get used -- and demanded. We'll go from a web where most web sites can be used casually, with no "sign on" (single or otherwise) to a web where far more sites demand you use the single sign on and thus have an account, because it's easy for them to ask.
This paradox is described at http://ideas.4brad.com/paradox-identity-managemen
Has it been over a year since you last donated to the Electronic Frontier Foundation
So you're saying that it would help filter out a majority of the "complete mumpty".
That's a possibility. But it would be even easier to just use Slashdot's reputation/moderation system on your own site. That would solve the "complete mumpty" problem while also solving the problem of someone with excellent karma for his programming knowledge posting his conspiracy theories on your site.
And it automatically tunes itself to your audience.
Not really. Check back on the "creationism vs evolution" stories here.
What would be considered "good stuff" on one site (or even by one moderator) would be considered ignorant drivel on another site (or by a different moderator).
You achieve all the same benefits without the problems just by having your own reputation/moderation system.
It's why we have exams, professional organisations, CVs, brands, social networking etc etc etc.
We use reputation all the time and no-one has come up with a single reliable, coherent way of measuring it. You just try to get a decent builder.
Deleted
And so what? Is all of this really so important? I find it fascinating that so many people on so many sites care more about their "reputations" than what they post.
Does it? Sometimes I don't WANT my "good" reputation to follow me. I like acting like a goon on something awful and like a lolcat-loving ding-dong on fark and like a...well...never-you-mind-like-what on consumptionjunction and 4chan.
When (and where) I want to be serious, I am. Others see it quickly enough too. It doesn't take long at each site I join for people to realize that I'm a "good poster". Honestly, it isn't complicated. Stay on topic, write well, be helpful, and the rest follows. Such has been my pattern over the years at sitepoint, namepros, webhostingtalk, and even here.
\Perhaps it's because I'm old
\\And still use slashies
\\\(reversed because slashdot doesn't like 'em forward for some reason)
Nothing is inexplicable; only unexplained -Tom Baker, Doctor Who
The advantage of OpenID is not replacing typing in a username and password with a URL. It is not having to sign up with Joe Random's blog site in order to post a one line comment. I don't want to give out my email address to every site I want to contribute to. Sure I could make up fake info for the site, but most of the time even that's too much work to bother with.
OpenID has problems yes, but there are technical solutions for all the ones I know of, including the redirect/proxy one you highlight. I've yet to see another solution to the identity problem that doesn't involve a centralised trust authority.
I was going to post much the same thing, so it's refreshing to see someone else thinking that way.
The thing about Slashdot's karma is that it creates groupthink. As you've said, too many people care about it instead of just posting what they think. So they post what they perceive to be the popular opinion, even if that's not what they really think, or even if it's contrary to what they really think.
Frankly, I think groupthink is a bigger problem than even the goatse links. Groupthink is where rational information exchange dies. If you look at the worst bible-thumping communities, or at rabid theocracies, or at the worst excesses of history, and some of its biggest mistakes too, almost all were based on groupthink. Take a million people who individually think "jeez, X is stupid and evil" and put them in a big group where they think that everyone else is fundamentally and rabidly pro-X... and watch them all start chest-thumping for the very thing they secretly despise. Just to get brownie points with the rest of the gang.
When a whole village went and cheered about one of them being burned at the stake as a witch (for bonus points when everyone knew it's a bogus charge and the real reason is something like: widow without sons inherits some land, some rich guy wants her land), that was groupthink. "OMG, I can't let the other ones even think I'm not a rabid fundie. Why, my popularity would go down."
At the risk of tempting Goodwin's law (although it's not a comparison): when a few million Germans cheered about invading the USSR, that was groupthink too. "OMG, I can't let the others think I'm not patriotic."
And in our own times, when you look at such things as bible-thumping communities, or at the broken high-school culture where being smart is uncool and being an airhead is the apex of fashion... guess what? That's groupthink too. Once the ball got rolling, even kids who do understand that their future job does depend on it... still go and insult the nerd, because that's what brings them karma points with the rest of the group.
So, to cut a long story short, I actually _don't_ want that kind of global karma. I actually _want_ people to come forth and say what they think, and not what they think would be popular in that community. I want people to actually come forth and say stuff like "this war is bogus" or "the PATRIOT act is unconstitutional" and not devolve into sheep thinking "OMG, I can't have it follow me for the rest of my life that I'm not patriotic or that maybe I have something to hide". Even if it's something as unimportant as a games forum, I actually want people to come forth and tell me the bad parts about it, so I can make an informed decision. I don't want more of them to think "OMG, if I say anything bad, I'll come out as a troll." Etc.
A polar bear is a cartesian bear after a coordinate transform.
Identity is really the easiest part of the problem.
But reputation is easy as well. The problem with most proposals is that they are focused on organizational reputation rather than personal reputation.
Reputation, however, is relative and contextual. We don't need Slashdot vouching for us, we need people in our own address book / social network. Then we can vouch for our friends/family in various ways ("this person isn't a spammer", "this person knows a lot about cars", etc.).
But the real power of a personal reputation system is that it is transitive. If I trust that Alice is not a spammer, and Alice trusts that Bob is not a spammer, I can to some degree also trust Bob, and so can my friends, etc. A few degrees of Kevin Bacon there and you've got a real system.
Such a system allows for anonymity as well. I don't need to use my real name if I can generate some other identity and foster trust in some other community. As long as the identity token itself is secure, they don't need to know my name, they just need to know I'm not a troll, I'm insightful (hint hint), etc.
My vision of such a system would use SMTP as the transport mechanism for requesting and relaying trust between parties. Mail agents would handle the requests automatically, like calendar-enabled mail programs do now, and it is a fully-distributed system. Mail clients would also cache trust from their own "friends," like DNS, to better respond to requests.
This degrades well, since the emails can contain manual instructions for those whose mail clients don't have this feature. Or their Internet providers can help with server-based responses, so the mail client doesn't even need to be involved in most cases.
With such a system, spam would mostly be a thing of the past. I can limit incoming email to only people in my Address Book, people in theirs, etc. out to some limit of degree. Chances are, that will quickly encompass everyone likely to want to send me a legitimate email, and bounce away people with no legitimate friends (spammers). The system would self-correct when accounts are compromised or people unwittingly trust spammers, and if a friend of mine is too naive and adds spammers to his list constantly, I can stop trusting his list.
We really do need a ubiquitous identity-trust system, something that uses existing protocols to share trust and integrates with IM, email, online forums, auction sites, etc. But the problem itself isn't that hard.