A lot of work has been done in small scale academic systems. A far as I know there are none in wide use. The W3C Collaboration group is completely dormant as far as I can tell (nothing posted since 1996 or so).
It is important to know what has gone on before. And there are a lot of interesting research results. But a community the size of slashdot hasnt done "collaborative moderation/filtering" before.
Based on the reactions to this posting, I would say that people generally confuse filtering (no one can see my comment! that's censorship!) and rating (your comment sucks, put it at the bottom).
A second confusion is whether a rating says how valuable a posting is, vs. one that agrees or disagrees. Saying whether something is valuable and deserves our attention has to be done, or else we are at the mercy of the spammers.
Thanks for your introspection on how you think during moderation.
Giving the moderator a weight is a good idea. But instead of it being based on votes of agreement with their scoring, let it be based on some more implicit criterion of the moderator's contributions to the community.
I have been thinking about how to do "collaborative moderation" for a few years now. To summarize my ideas in the context of Slashdot:
1. What you call "moderator", I call "participant". Any human person can be a participant. A participant must be registered (though possibly psueodonymous), and possibly authenticated to prevent cheating. 2. Participants rate each message based on its "value" (not agreement). 3. Participants accrue "weight" based on some algorithm which is a function of the value of the messages that they have posted in the past. (weight might change for other reasons, too) 4. Messages have values based on some algorithm which is a function of rating and the weight of the rater. 5. Users can sort/filter messages based on their values.
Obviously, what this creates is a "consensus value" of a message. This would be the first thing for slashdot to do. (If I understand what you are doing now, you assign a weight of 1 to the 408 moderators, and a weight of 0 to all others).
It's also very interesting to have a second message rating, namely "agreement". 1. This helps the person doing the rating to explicitly seperate message value from agreement. 2. Its allows to compute distance between two participants based on how much they agree. 3. It allows messages to have other kinds of values besides consensus values, for example the "agreement value" of a message uses the agreement distance between participants to weight the value ratings. So you can say "sort the messages based on how much people agree with me". 4. Strong clusterings of participants might emerge that indicate different "viewpoints". A viewpoint might be thought of as a synthetic participant. Users can examine values from different viewpoints (ie weighted by viewpoint agreement), and thus escape from the "tyranny of the majority" (if they wish).
I would be very interested in comments, critisism, and collaborators. I would be interested in helping slashdot implement any such system (Im a programmer: Java and C).
Slashdot Mirror
I like it. I just posted a message with similar ideas.
I think your ideas are mostly good. But I would say that variance indicates differences of opinion. "Truth" is not the average opinion.
A lot of work has been done in small scale academic systems. A far as I know there are none in wide use. The W3C Collaboration group is completely dormant as far as I can tell (nothing posted since 1996 or so).
It is important to know what has gone on before. And there are a lot of interesting research results. But a community the size of slashdot hasnt done "collaborative moderation/filtering" before.
Based on the reactions to this posting, I would say that people generally confuse filtering (no one can see my comment! that's censorship!) and rating (your comment sucks, put it at the bottom).
A second confusion is whether a rating says how valuable a posting is, vs. one that agrees or disagrees. Saying whether something is valuable and deserves our attention has to be done, or else we are at the mercy of the spammers.
Thanks for your introspection on how you think during moderation.
Giving the moderator a weight is a good idea. But instead of it being based on votes of agreement with their scoring, let it be based on some more implicit criterion of the moderator's contributions to the community.
Besides kill files, I dont know of any standard filtering. There's no scoring at all. Isn't that true?
I have been thinking about how to do "collaborative moderation" for a few years now. To summarize my ideas in the context of Slashdot:
i _cscw.html
1. What you call "moderator", I call "participant". Any human person can be a participant. A participant must be registered (though possibly psueodonymous), and possibly authenticated to prevent cheating.
2. Participants rate each message based on its "value" (not agreement).
3. Participants accrue "weight" based on some algorithm which is a function of the value of the messages that they have posted in the past. (weight might change for other reasons, too)
4. Messages have values based on some algorithm which is a function of rating and the weight of the rater.
5. Users can sort/filter messages based on their values.
Obviously, what this creates is a "consensus value" of a message. This would be the first thing for slashdot to do. (If I understand what you are doing now, you assign a weight of 1 to the 408 moderators, and a weight of 0 to all others).
It's also very interesting to have a second message rating, namely "agreement".
1. This helps the person doing the rating to explicitly seperate message value from agreement.
2. Its allows to compute distance between two participants based on how much they agree.
3. It allows messages to have other kinds of values besides consensus values, for example the "agreement value" of a message uses the agreement distance between participants to weight the value ratings. So you can say "sort the messages based on how much people agree with me".
4. Strong clusterings of participants might emerge that indicate different "viewpoints". A viewpoint might be thought of as a synthetic participant. Users can examine values from different viewpoints (ie weighted by viewpoint agreement), and thus escape from the "tyranny of the majority" (if they wish).
These ideas are embedded in a bigger system for "wide area collaboration", focused on "a better Usenet". I wrote a paper on it at:
http://www.unidata.ucar.edu/staff/caron/collab/ch
I would be very interested in comments, critisism, and collaborators. I would be interested in helping slashdot implement any such system (Im a programmer: Java and C).