Slashdot Mirror
Distributed Trust Metrics?
"I've done some googling for systems that might work in a distributed fashion but turned up nothing. I'd happily register a key with an authority (ideally a distributed one, think supernodes rather than centralized structure) and have it verify my identity. Then, at each website participating in the trust network, I can provide my identity upon registration. As people moderate me and my comments, this feedback is applied to my profile both locally and network wide. The idea is that I may be all wet when it comes to tractors, but relatively well read on politics and technology (i.e.: my overall trustworthiness would be a 7, with a 3 on misc.rural, a 8 on slashdot.org and a 10 on poliglut.org). Now readers of my commentary have a more reasonable way of judging my trustworthiness on both a local and a global scale."
Unfortunately, I feel that in most cases such a small sample size would render virtually any impersonal/algorithmic trust metric unhelpful or at best unreliable. I think it would be best to implement something simple and human-powered; maybe like an extension of the Slashdot zoo system? This works very effectively as is for shutting out regular trolls (or even just ACs, if that floats your boat) and would probably be just fine without the moderation systems. Set friends and foes, allow comment scores and let a few simple rules do the rest!
Bash script for FP whores
Small groups of the most active good posters doing the moderation. Find people that regularly share your views or at least make intelligent comments? Ask them if they'd be interesting in helping moderate. Spreading the load between a few people should make things easy to manage.
1) require e-mail response verified accounts to post
:)
2) enable the ability to 'bozo-bin' someone: their account can be made so that they can still post, and they can see their posts, but noone else can. Most bozos won't even know they've been binned, and thus will not try to create a new account to get around it. Think of it as a honey pot for trolls.
3) Check for bozos all coming from the same domain - likely the same bozo who has realized he's been binned, and has created a new email address from (probably) his own domain - so bin all accounts from that domain.
That should cut down on the vast majority of problems, I'd think. Also, with a 'small' site, as you say, moderation doesn't work well. Well, with a SMALL site, you don't _need_ moderators to handle the load, so that should work out well, right?
Block users for ridiculously long amounts of time when they troll. If they troll repeatedly after the ban(s) simply use a redirect statement to send them automatically to http://www.tubgirl.com or even better, www.slashdot.org.
And its Web of Trust might help you. Let the social net filter the bad stuff.
It works very well on small samples, IMHO. In fact, I believe a Web of Trust doesn't scale in the thousands, or hundreds of thousands, because of the dilution of the metric.
Also, since you run a political website, a Web of Trust can help to "cluster" similar points of view.
Buy a Nintendo DS Lite
Actually, I just this morning read this article by Clay Shirky which covers your points exactly.
In summary, Technical and Social issues are inextricably linked, and what you're really looking for is a group of people to take on a governmental role for your website(s).
/* affect != effect */ void affect(int *thing,int effect) { *thing += effect; }
Here's an idea I had a few days ago that I think would work very well... a Bayesian moderation system. Add or subtract points based on some kind of bayesian filter. Keep traditional user moderation, but use it to train the filters.
Username taken, please choose another one.
It is sometimes difficult to distinguish what is exactly a troll and what is not, therefore it is very very bad in my opinion to completely block someone from your site. I have on occassion been given the "Troll" mod on /. I don't agree with it, because my intention was not to troll, things can be taken the wrong way. I'm not trying to justify my previous posts but i do think it is necessary to keep everyone in the system. Just use a point based system and give mod privledges to a few trustworth users. You don't need metamod or anything fancy. Just keep it open and free (linux free).
so *noone* from @aol.com would be allowed to post on these forums ever again... ok, sounds great
Linkfilter FAQ
/. you can change your settings so that funny & troll are +1 and informative and interesting are -1 and, in doing so, completely change the content it provides.)
Really though, the system doesn't matter much. You have two choices, make it the way you like it or make it so customizable that the content forms itself in the manner that matters most to the reader. (For example on
Personally, I reccomend deciding now if you want a clique that agrees with you or an open site filled with content that offends you. Well defined goals are always a good thing.
No Zen is good zen
What happens when this is larger? Someone has to manage the keys to the sites (for updating the registry of trust levels) so that not just any site can update a user's rating. Who decides what is an appropriate site? Why can't I create my own site and boost my level so my comments show up on other sites with similar content?
Require accounts. Allow people to keep individual lists of users from whom they don't want to hear. Allow users to subscribe to each othere's lists.
I forget what 8 was for.
Affero: Rating & Reputation Service for Online Works
Kuro5hin commentary on the below story
This story on Kuro5hin entered a living hell when the sites being discussed sent their members to go get accounts and start stuffing the ballots.
Even controlled moderation would fail simply because the signal/noise ratio at the bottom level becomes so out of whack that no one wants to dive in the sewer on the chance of finding a diamond.
No Zen is good zen
As you suggest, a distributed, global (federated) identity would make this all a lot easier and work a lot better. Persistent profile information is powerful and offers many advantages to citizens, corporations and all those middlemen, but can lead to serious privacy abuses if the information is not securely - and absolutely - controlled by the profile owner.
The fact that global identity is so valuable has not escaped the eye of marketing departments everywhere, and there are several projects aimed at establishing global identities for consumers. (Note that I say "consumers" here rather than "citizens" as the systems being designed generally only see you as a consumer and nothing more. Therefore, since there's nothing to buy on Slashdot or Poliglut, you probably wouldn't exist on those systems.)
There are two main problems with the currently proposed systems: Passport, designed by Microsoft, is a wholly centralized system. (The only thing good about this is that your profile data in Passport is not in danger of being bought by Microsoft!) The other system is Project Liberty, a system being put together by a scary consortium of BigCos. EPIC has a good, short paper on the privacy considerations of Liberty here.
There's a new group in this area working...
I'm a member of the tech group and suffice to say we're looking at a very hard problem. One of the key insights into this work is that we don't need to build a global namespace. Not only is that hard (viz PKI) but it's not even what people really want. Rather, people belong to groups and have local names for people within their groups. As people from other groups get introduced into one's local group, they either get local names or become known as "xyz from 123 group". While global URIs may exist to uniquely point to every object in the universe, they are generally difficult to manage and use.So where does one go? First, of course it has to be open source. PGP's web of trust, Ron Rivest's and Carl Ellison's SPKI/SDSI, and Matt Blaze's Keynote all offer secure local name spaces and even integrated trust management systems. (Thought I had forgotten about your original point, didn't you?) We're nearing completion of a requirements specification and hope to have an initial implementation by years end. And this is being done mostly by volunteers, as there's no money (yet).
BTW, one of the most difficult problems facing federated identity systems such as Liberty is how to get all these BigCos to work together. We're following Chaordic approach that, like the Visa payment system, melds simple but powerful global unilateral agreements (Principles) with local control of agreements that control inter-group relationships.
The antidote for misuse of freedom of speech is more freedom of speech.
-- Molly Ivins
I realized after writing that (rather hastily as I still need to finish packing for a two-week trip starting at 6 AM tomorrow morning) that I didn't offer any answer at all to the original question. So let me introduce you to another interesting rating mechanism, Affero, which enables you to rate others who help you out by essentially donating to causes they believe in. For example, if this helps you at all, you can rate me.
The antidote for misuse of freedom of speech is more freedom of speech.
-- Molly Ivins
you've certainly come to the right place! ;>
- Let each user decide who some of his friends/foes are, just like in Slashdot. Rate them accordingly, say on a scale of 0-1.
- that will filter posts by the people you've rated.
- as for the users you haven't rated:
- if there is a "path of trust" between you and that user, i.e. if there is a friend/foe of yours who has rated a friend/foe who has rated (...*x) this user, calculate a rate. You can try to multiply the rates, or use the average, whatever works best for you.
- if there isn't, or if the user is new, give him an initial rating, again, just like in slashdot.
At least that's the main idea... There are some complications when there are multiple path and so on. I have published a paper on this, so if you're interested, leave me a message and I'll send you the paper. I'm also curious about how the system can work in a non simulated setting.
"In our tactical decisions, we are operating contrary to our strategic interest."
Several people have suggested moderation or filtering schemes, in which users can say, essentially, which posts they like and which they don't like. Depending on the approach, the system could even learn, either by user (e.g. the Bozo filter) or by content (e.g. Baysian filtering), etc. It then promotes the posts people like, and hides the ones they don't like. (Sounds familiar?)
There is a fundamental problem with this though, which is particularly acute for a site such as yours that exists for the sake of political debate.
If you do this, then users will tend to be presented with opinions that are most similar to their own, and have dissenting opinion hidden from them. That's going to stifle debate and make the site much less interesting.
The only real way to do it is to find a small team of dedicated moderators who are able to objectively rate content according to its intrinsic worth rather than according to the opinion expressed. That's hard.
You say you have a problem with trolls. I suggest a "Report to moderator" link added to every post. Every user will be able to report a offending post, but it would require an action of one of the select few to actually edit or remove it. You can be the only moderator, or you can grant the power to a few other users. If the site is small, you don't need much manpower.
It would also help if you could ban IPs of the trolls. I don't suggest requiring registration with e-mail confirmation, because if your site is small, you don't want to present users with additional hurdle.
Also check this very useful site for some hints: Joel on Software - Building Communities with Software.
Future Wiki -- If you don't think about the future, you cannot have one.
If I'm interpreting you correctly, you want:
1) A global system of trust metrication, rather than one per website - so I can certify you as a non-troll once and for all, rather than once on each website.
2) A system which is not in the total command of a single website - in other words, one in which different websites can have a different "root of trust".
I'm assuming you want:
3) attack resistance as Raph Levien defines it
since the experience of sites like Kuro5hin especially demonstrate that non-attack-resistant trust metrics are pretty much a waste of time, and I can only assume that those who advocate collateral-damage string-and-sealing-wax methods like blocking IP address blocks don't realise that attack resistance in the trust metric is possible.
My TrustFlow trust metric was designed exactly to fulfill these criteria, and I believe it is the only one that does so at this time. It's pretty simple to understand, and a fast Perl/C implementation is available under the GPL (ask me if you need it relicensed).
As an experiment, I wrote a program that applied TrustFlow to the LiveJournal friends graph. Nearly 70,000 people have now tried it and users mostly report that it does a pretty good job of reflecting who is close to them in the graph.
In theory, you could set a trust root key and a trust threshold, and users could provide PK-signed documents proving that they are trusted to a particular extent by that root key by showing a subset of the trust graph. If you want to do this, though, ask for my help, there are some subtleties.
The metric, and the implementation, are discussed in the Trust Metrics LiveJournal community. I draw your particular attention to the entry describing the metric and the TrustFlow for LiveJournal FAQ.
Xenu loves you!
I just set up a wiki on trust metrics evaluation.
...)
The goal of this project is to review, understand, code and compare on same data all the trust metrics proposed so far.
I'm a PhD student and this is my phd research proposal (Trust-aware Decentralized Recommender Systems) and it is very related to all this concerns (trust, reputation, decentralization, blogs, recommender system,
Personally I think the more promising path to follow is FOAF (Friend Of A Friend) format (see the project blog). There are also 2 extensions (1) (2) to incorporate trust into FOAF files.
By the way, here is my foaf file. You can add me to your "knows" list if you like.
Ok, so now you can check the wiki on trust metrics evaluation and add anything you think should be there and there isn't.
Please, note that I have just opened the wiki and I still have some problem with the installation (there are 2 conflicting python installations and I'm not root of the machine) but the basic functionalities are working. Hope to continue the discussion on the wiki! Paolo
Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins, Grumpy Watkins,
Ruby Sleeps