Moderation Ideas

So now with the IFRAME/ILAYER stuff mostly removed from Slashdot pages are rendering faster. I still have some SQL glitches, but I also have a few more ideas for the moderation system that I want to throw out to you guys for opinions (and my guess is you'll have a few).

First Some Comments on M2 and Karma As a whole it seems to be working pretty well. Some people are really irritated, other people like it, and some people simply are fretting over their karma way to much. Let me just remind everyone that Karma isn't that important. You should expect to see some karma come and go since your activity is subject to both M1 and M2 moderation. As an aside, M2 moderation will not push you out of the -10...10 range (in other words, being a bad moderator will never cause you to get the Comment Penalty). The only thing karma affects is eligibility for moderation participation.

I really wish I didn't feel obligated to display peoples' karma on their user info page. It tends to be a distraction. It's not supposed to be an integer determining anyone's value as a human being, it merely functions as a sort of 'Risk Factor' for moderation related activities. High Karma means you are a low risk, negative karma means you shouldn't be given tons of moderator points and turned loose on the world ;)

"The Problem" As I see it, the major problem on Slashdot is bad people gaining moderator points. The M2 system is one way to counteract that: An unfair moderator is (theoretically) weeded out. They are still free to post and participate as always, but they can no longer moderate if their karma drops below 0.

"The Assumption" I'm an optimist. I assume that there are more good people than bad people in the discussion groups. I assume that the good people will participate, and help weed out the bad ones. I also assume that eventually there will be world peace and harmony amongst all races and creeds and religions and genders. I'm young and an idealist here, gimme a break ;) As a whole, I think this is true: I've Meta Moderated many times, and I rarely find that more than 20% of moderation is bad. Many moderations are questionable, but rarely do I read one and think "What a moron". But they are there, and I want to try to make it happen less.

"The Solution" On an abstract level, the solution is to restrict the amount of power that any single user has within the system. Certain restrictions are in place already: moderators occasionally get 5 points, they and they lose them after 3 days, can't post and moderate the same discussion etc. Each of limits is designed to reduce the risk that any given person can screw with the system.

Each of these limits suck for a good moderator. This is given: the naughty people force rules to exist for the good people. If those 84 year old blue haired ladies who drive on the yellow line at 20MPH with their left turn signal on weren't around, maybe we wouldn't have speed limits either (Note: I also tend to drive 5-10 MPH under the speed limit and regularly forget to turn off my blinker, so I am part of the problem ;)

"The Idea" The idea is inspired by M2 moderations removing of the ability to choose what you M2 moderate. (This is flawed of course because you can reload and get 10 new comments but I'll fix that eventually). It also is a pain because you have no context for many of the comments. By not choosing which comments you can moderate, the risk of doing something inappropriate is greatly reduced: eg, when Mr. Bad Moderator gets 5 points he waits until GNOME article appears and then moderates all the pro-KDE comments down.

The solution is to reduce the probability that Mr. Bad Moderator can moderate a comment that pushes his agenda. This is what the point limits and the time limits are designed to accomplish today, and what this idea is designed to accomplish should we decide to implement them.

So here are the ideas:

1. Put 5x (Numbers are Variables here: Don't complain about them!) as many moderators into the system.
2. When you have moderator access, only display moderate controls on say 20% of the comments.
3. Increase the percentage of moderatable comments based on karma.

It makes sense: negative karma users would never moderate, and more karma would give you more choice in your moderation. It would cripple good moderators because they might see a bad comment but be helpless to moderate it, However since there are 5x more moderators, the odds are someone else will get it. Side benefits include more people given "a shot" at moderating and more people will be allowed to contribute. If the M2 stuff works, the bad moderators will lose karma and no longer be allowed to moderate, and the better moderators will be given a larger percentage of all comments to moderate.

Another Sort of unrelated idea:

• Integrate M2 into the comments display tree: this solves the context problem that the existing metamod page has. Simply randomly select the occasional moderated comment for M2 moderation. Plus its much more "Real Time" and becomes less of a secondary chore. Plus it allows more people to participate since M2 moderation has much less restrictions on it. The downside is that this might be a pretty serious performance hit. I haven't really thought through it yet, but its an interesting thought. Comments on it?

And finally a note to the flamers... Turn it off. Go to your user preferences and click 'Unwilling' and you won't be asked to moderate. Click 'No Scores' and you won't see scores again. Change your threshold to -1 and forget that they even exist. This system is designed to provide a solution to people who wish to use Slashdot like usenet. It's designed to provide a solution to people who want to use Slashdot more like a newspaper. It's designed for people who want something in the middle. If you don't like something, you should be able to change it. I've worked my ass off to make as much of this optional as possible. I don't care if you turn stuff off. It won't hurt my feelings at all!

Comments

Karma -> Grade Inflation

[Just Some Guy]

It seems like ratings are getting very bi-polar around here. Posts that are a little bit better-than-average are spiking out at +5, while ones that are just slightly sub-par are getting -1's. Why? Because moderators are interested in showing how much they agree with the other moderators, in hopes of increasing their karma.

I gave this some thought yesterday while I was mowing my lawn, and I came up with:

• If someone is logged in with moderator status, or they are surfing as A.C.'s, then don't display numeric ratings on the posts. This way moderators can assigned ratings based purely on a post's merits (or lack thereof) and not based on what another moderator has to say.
• Don't show the actual karma rating on the user pages; the sign of the number should be sufficient. That way, you could see whether your karma is good or bad, but not the magnitude. This isn't a race, after all..

That's just my dollar/50. BTW, I've been a moderator a few times recently, so I'm not taking potshots at a different group of people.

Re:Karma -> Grade Inflation

[torrent]

If someone is logged in with moderator status, or they are surfing as A.C.'s, then don't display numeric ratings on the posts. This way moderators can assigned ratings based purely on a post's merits (or lack thereof) and not based on what another moderator has to say.

I suspect that hiding the scores would actually increase what you want to prevent. As it is, a moderator can see whether a post has been moderated appropriately. But if you hide the scores, every moderator might increase the score on a post that is slightly above average. You would see a post that might deserve a +2 turn into a +5 because every moderator decided that it was worthy of a little extra attention, and they were unaware that the post had already been given that attention. Leaving it the way it is allows moderators to move the score towards what they think is ultimately appropriate.

Re:Moderation and karma

[luge]

Yeah. I hate to be elitist about this, but it seems pretty reasonable. I might also suggest that humor points not be counted for karma purposes- while I enjoy reading funny posts, the authors of said posts (which are much more likely than a good, serious post to get a 5) really shouldn't get so much karma credit for it.
Keep up the good work, Rob- someday, people will study your code to see how to make internet conversations that work for large numbers of people. I do believe that this is the first time anyone has seriously tried to make anything like this work- and the quasi-experimentation that you have done will be a source for the next group that tries to apply this technique to more serious fields of endeavour.

The slashdot moderation system is flawed

[Jeffrey Baker]

Rob,

Please accept my humble criticism of slashdot's comment system. I have been thinking about this system a lot lately and designing a replacement for it. I have no delusions of replacing slashdot: slashdot is a community that would be hard to duplicate. The only problem I have with slashdot is the comment mechanism, and I will present those problems and some possible solutions.

Problem: Moderation race conditions. Joe and Jane are both moderators reading a slashdot article. They both see a comment that they think is interesting, but not incredible. They both choose to spend one moderator point to raise the article to "Score: 2 (interesting)". They continue to read the comments and click the "moderate" button a few minutes later. Now the article is rated 3, but neither of them thought it was worthy of 3. Moderation has failed for both of them. One of them could choose to moderate it back down to 2, but then the article would be rated "Score: 2 (overrated)", which does not reflect their true opinion of the article.

Solution: Instead of incremental ratings (+1, -1), allow the moderator to score the article on a range, say from 0 to 5. Then if two moderators assign a score of 2, the article's score is 2, not 3.

Problem: Only a few people have a few moderation points at once. This leads to several worst-case failures. One of them is that no moderators will read a given article, and the comments in that article will be clustered around 0 and 1, making the "Show highest scores first" setting much less useful.

Solution: Allow all readers to be moderators all of the time. There will never be a deficiency in the number of moderators with this method. The number of people scoring articles will be directly proportional to the number of people reading that article. When 100% of the participants can cast votes, there is no outlying possibility if lack of moderation.

Problem: The community ranking does not allow for individual preferences. The individual may have a strong opinion for or against a certain author, but the score given by the moderators is unlikely to reflect his preference.

Solution: Use a network of preferences to tailor the rankings that a user sees and to delegate the ranking of articles.

I believe that the ideal system would allow the user to cast a vote on every comment posted. The comment display system would use the user's voting history to score and sort the comments on every new article. Take an example:

Joe, John, Jack, Jerry, Jane, and Jennifer are all users. In the past, Joe has given articles by Jack an average rating of 4.2 (out of 5), and has given John an average ranking of 3.8. Joe has consistently rated Jerry at 0. Joe, John, and Jerry all have overall ratings (the average rating given by all the system's users) of 2.

In a new article, Jane, who has never posted a comment before, posts an unusually insightful comment. John, Jack, and Jerry all post their usual stuff. All three of them rate Jane's article at 5, but the overall community ranking for Jane is only 3. When Joe reads the comments for the article, the article display system will give John and Jacks votes more weight than the community vote, because Joe has rated those two authors highly. The system will completely ignore Jerry's votes because Joe doesn't think much of him.

The result is that Jane's article (along with John's and Jack's) are listed at the top of the page, instead of lost in the middle of the pack. Other users with completely different preferences will get completely different displays. When there are no personal votes for an author or article, the community ranking can be used to score the article.

Whew, I hope that example was clear. It really isn't very complex or even original. It is just like what firefly was doing 5 years ago, but with a little less complexity and a lot more scalability. If slashdot were to implement a reputation management system like this, I think it would be a much better place.

Regards,
jwb

Meta Moderation is Redundant

[Ignatius]

It stikes me, that moderators get 10 M2 points per day but merely about 5 regular mod. points per month(!) to do their job in the first place. Using 98% of the pontential ressources for control and merely 2% for the actual work might be acceptable when handling nuclear weapons or biotoxins, but is IMHO total overkill when it comes to managing voluntary moderator work for an online forum, esp. since the exactly same information as provided by the M2 scheme could easily been extracted by better analysing comment moderations. Here's how:

Hypothesis 1: The majority of moderators are fair and take thir job seriously.

Hypothesis 2: If one moderator rates a certain comment up, and another moderator rates the same comment down, it is reasonable to assume, that they would equally inclined to consider the other's decision as unfair.

Conclusion 1: The ammount of mutually nullifying moderator decisions on a certain post provides a measure for the total 'unfairness' p of a comments moderation, where p = 2*min(#_of_ups,#_of_downs). If hypothesis 2 is true, then this p should be proportional to the karma penalty generated by the current moderation scheme.

Conclusion 2: Since we assume that there are more good moderators than bad ones, it is reasonable to assume that in a contoversial comment moderation the majority has a higher propability to be "right" than the minority (provided that there are more than 2 moderation decisins involved)

Proposed Solution: In the case of a controversial comment (i.e. a comment with up *and* down votes), generate a (very small) karma penalty proportional to p and distribute #ups/(#ups+#downs) of the penalty among the down-voters and #downs/(#ups+#downs) among the up-voters.

Example: A comment gets 4 votes up and 2 votes down. Each up-voter gets 1/12 (33% total) and each down voter gets 1/3 (67% total) of the total penalty which ammounts to 2*min(4,2)=4 penalty points (which should IMHO be somewhere around 1/5 to 1/10 of a current karma point)

In order for this system to work, it would be necessary to increase the total ammount of moderation points floating around and it might be reasonable to give some karma reward for each moderation to compensate for the total penalty generated by this system (so that in the end only the guys who regularily make minority decisions take the punishment).

Of course, the used formulas are only a proposal and fine tuning could certainly increase the overall efficiency of the system, once empirical data is available.

Karma for Funny points

[jabber]

Some of the Funny ratings are quite pertinent to the discussion at hand though. And a humorous attitude about something tends to show understanding of the subject... Well, to me at least. So a person with enough understanding to make a joke is, in effect, making a worthwhile contribution.

Maybe we should have an additional category? For relevant humor or offtopic humor? But here I diverge from my personal opinion that simpler is better...

Karmic disturbances

[jabber]

IMHO, the 'Overrated' and 'Underrated' moderation items should have the effect of MetaModeration. These are, essencially, means by which moderators can correct one another's work.

Re: Karmic disturbances

[Kyrrin]

That would not then have the function of raising/lowering the score, though, which is what Over/Underrated is useful for, no?

Moderate thoughts on Moderation

[jabber]

First off, MM is not really necessary.

A good comment is good. It is unlikely that a single, rabid individual, will sink a good comment into oblivion, without an equally determined benefactor raising it up again. By that same token, a conspiracy of evildoers to elevate each other's trash, whenever one of them has moderator access, is slim to none.

Occam's Razor applies, and a system of lesser complexity is preferable to one of greater complexity. Slashdot should not hamstring itself with rules, and rules about rules, else it will become a burocracy in which nothing ever happens.

Personally, I've never used MM, but I would rather see it go away - simply because Moderation points strike me as a rare commodity, so their negative effect seems to be outweighed by their utility to the holder. Also, with /. being a self-correcting mechanism already, they just add another layer.

MM is an interesting exercise for Rob, it nothing else, and as such it is as useful as anything that keeps a mind from being idle. :) I, for one, would rather see Rob bring his considerable talents to bear on extending the article discussion forums. On many occasions, a great off-topic discussion has been sparked by an article. This discussion either flops for being off-topic, or dies when the article expires from the main page. It would be nice to have some quantifiable worth of such a discussion, that would enable it to live on until it dies a natural death.

As for a redoux of the moderation philosophy, how about fractional moderation? Increase the number of moderators, certainly, maybe by a factor of four. But reduce the worth of each assignable point by that same amount. This way, several moderators would have to agree to elevate a post to the next higher integer score. The ability to abuse moderator privilege goes away, as does the confidence that a single vote of one individual will make a lot of difference.

As moderators, we would then add our voice to a comment, not knowing if we're the first, or the decisive ones to raise the message status. Follow?

This way, good posts will rise, bad posts will sink, and iffy posts will tent to teeter-totter on the edge of where they started.

Ah, all this opinionated rambling has tired me out. But, before my hands completely cramp... How's about making all posts require a Preview??

Restricting Moderation Too Much

[TheMeld]

I think that it is a bad idea to restrict moderation excessively. When I have moderator access, I change my prefs to browse at low scores, and look for new comments (i.e. ones that probably haven't haven't been moderated yet). That way comments that deserve to be moderated up get there. Comments that are at 3 or 4 don't need that much help.

However, once I'm done moderating, I switch the preferences back to a more filtered view, and go back and re-read the articles looking for the content, as usual.

I think it is *essential* that moderators be allowed to pick which comments they moderate during the 3 days they have their points. Giving moderator access more or less frequently based on karma is a perfectly good idea, and could achieve some of the same effect.

Finally, I would like to just throw a reality check out there. Slashdot should be good, but there is *no* way to weed out all the a-holes, flamers, trolls, etc. Slashdot already does a very good job of tagging their comments so that people like me who don't generally want to see them don't have to. I think that, at this point, the moderation system is sufficiently effective, and time would be better spent squashing some of those SQL problems, and implementing some of the great ideas for additional forms of content that could be added.

Re:Intelligent...

[maphew]

Actually, why not make karma be an average of post moderations?

And what of the lurkers? Just because a person doesn't post doesn't mean they wouldn't make a good moderator.

Actually Rob, how many users are there who haven't posted, but still read regularily?

Watch out

[grappler]

if they tie it too closely, some people will make slashdot into a big game, where the object is to get as many karma points as you can so that you can have the most influence you can, both in your moderation and high initial comment scores. I know some people already think of it this way, but it's not something to be encouraged.

Integrated M2 moderation

[lisa]

I *really* like the idea of integrated M2 moderation in the comments section.

I can't tell you how many times I've thought the moderators-in the context of a comments thread, were either clueless or not paying attention or just plain dumb. However, with the current M2 setup, I don't understand the context and don't really want to take the time to go look. I am already taking the time to do the extra moderation.

When I'm not a moderator, it would still be nice to feel I am having some sort of role in the moderation process-even if its just keeping a check on the moderators.

Problems I foresee-Someone moderates a comment, and immediately the moderation is moderated (jeez, i should choose a new word) to be Unfair. So does the moderator go back and change it? Is that unfair pressure? And if you M2 a comment in a thread, are you unable to participate in that discussion?

I like the idea of Karma. Though I think it would be cooler to see it on a scale when presented on the user info page. I pay quite a bit of attention to this site, but I still don't remember the scale offhand. A graphical bar would be nice. And instead of just numbers, relate the the numbers to real world behavior. ie. karma of -1 is called 'troll' or something like that. Be creative!

-Lisa

Moderation problems

[AeiwiMaster]

I have noticed some small problems with the moderation system.
They might be connected.

1) Good messages not posted fast after the news are not moderated up.

2) Points used on 5 points messages don't add to the story points
but decrees your moderation points.
This mean lesser points for messages in 1)

3) There is no way to view unmoderated messages
when you moderate.


I think that a solution to 2 and 3 will help on 1.

Knud

Aeiwi is a search tool with a unique interface.
Which let users add more search terms,
until they have a small number of results.

Random Mass Moderation

[the_ed]

Mostly, this sounds like a good thing. I completely agree that no one user should have too much power, but my gripe is here:

As a moderator, you should have the ability to moderate where it is needed, and not have your moderating ability limited to randomly selected comments.

If a moderator sees a particularly outstanding comment, they should be able to moderate it, and not be hindered by a random 20% factor. If you are going to increase the number of moderators, do so at the expense of moderation points, not at the expense of the ability to moderate.

And I do have to agree with Rob: if you don't like the system he's worked so hard to build, you can get around it without asking him to take it away.

Integrated and Random M2 moderation

[CocaCola]

While I agree that integrated M2 is fundamentally correct and wanted, 'random M2' has some very important properties as well: it in a sense _forces_ and ensures the moderator to take up 'independent' issues/comments and judge them. An M2 almost by definition only ever sees articles in which he is interested - this again creates inhomogenity in the distribution of M2's. Randomness distributes M2's much better between moderations, and this also makes it sure that 'bad moderators' will be caught sooner or later. Eg. if a 'malicious moderator' goes back to a KDE vs. Gnome thread, and moderates down KDE-positive comments, then this skews the _historic record_. Newcomers, analysts (and history) will be manipulated. It has a very low likelyhood that such malicious moderating will ever be detected - M2's by definition do not read old articles. (but there _are_ people who read old articles, but they are not likely to M2 moderate, or even _notice_ that the historic record has been manipulated with.)
All in one, I very much like the current 'daily random M2', because 1) it gives me a warm fuzzy feeling that even such random samples are usually 90% well-moderated 2) I can be sure conceptually that the bad guys will be caught sooner or later, and it might be me who catches one of them. M2 is really like law enforcement, you both want to patrol on the streets but you also want to do larger scale (and more random) searches.
The solution? Honestly I dont really know. Maybe this hybrid method: there would be a 'get me 1 random comment to M2-moderate' link right at the top, but also it would be possible to M2-moderate 'in the field'. Thus I could eg. random-M2 5 comments, and realtime-moderate another 5 comments. The 'split' between random and realtime M2 is thus up to the M2-moderator.

Agreed: keep "living" discussions visible!

[Dugan]

From my earliest Slashdot-lurking days, I've wished that the interesting, lively debates didn't get whisked off the front page and into oblivion so soon.

Imagine a page or pages, linked to Slashdot's front page, called:

- Current "Liveliest" Discussions: ordered by comments/minute
- Week/Month/Year's Hottest Debates: ordered by total no. of comments
- Active Debates: "old" articles with recent comments (ordered by "recency"?)

Etc... other criteria are possible.

Wouldn't this add a fascinating static element to Slashdot? Eventually certain discussions would emerge as "Slashdot's All-Time Great Debates" , with thousands of comments from a period of years. Set your threshold to +30 to savor the distilled, democratic wisdom of 30000 slashdotters ;-)

What do you think?
Dugan

More ideas...

[fremen]

The M2 thing seems to be going nicely, but I would like to propose an additional idea. When you moderate, you're limited to having very few subjects that detail how you feel about the message. Sometimes they don't do justice to your feelings about the post. That's why I think that it would be nice to have a one line comment detailing why you think the moderation you just made was fair. This is something that I have actually wanted in the past, and it might even help with the M2 moderation.

Writing a comment would be optional, as many moderations are simply self explanatory. But, it could be useful to say "I moderated this down because it was repeated above in comment #xyz," or "I moderated this down because we aren't discussing foo here, only bar." Of course, positive feedback would be good, too. Good posters could be rewarded with feedback like "Good post," or "Nice job researching this subject."

This information could then be made available on the M2 moderation page, permitting people to get an idea about why a moderation took place. A comment that might look perfectly normal could have been grossly off topic. Further, it would give the original poster an idea about why he was moderated up or down. Everybody wins.

Of course, the moderation comments would have to have limits. Perhaps the complete history of the moderation comments would only be available to the original poster, while everyone else only saw the most recent. This would prevent silliness like whole discussion threads on why a post was moderated in such and such of a way.

Anyway, it's just a thought.

possible results of the suggested moderation syst.

[dermond]

when i moderated i read through an article and if i find one (or 2 really) good posts i moderate them up.. i assume most people do it that way. the result of the new system would be that probably peopel see a good comment and can not moderate them ..

so they search for a similar one that does express a similar idea (or simply repeats most of the idea) that they can moderate up.. and the will moderat that up. so the whole moderation process would loose sharpness it would be somehow blurred. it would not help to find the perls form the posts...

mond.

Anal retentive? I don't think so

[El Volio]

In response to the people who are upset about /. getting "anal" and the users needing to "get over themselves":

Look, I usually have neither the time nor inclination to read a bunch of "m1cr0s0ft sux" posts, nor "First!", nor posts that are nothing but flamebait. Sometimes I do, and I read the comments with my threshold set at -1. It's at 4 most of the time, but I still go to -1 when I can to get the flavor.

But I'm not being anal. This isn't about what I should or shouldn't read, though I do skip posts I find useless. This is about trying to filter the massive amounts of information that come my way every day. If you have the time and desire to read it all, have at it! I'm actually jealous, since Lord knows I would like to do so.

Sadly, I can't, and neither can a lot of other /.'ers. If Rob et al. have the time and desire to improve the system for us, great. If not, I'll live. I'll still read /., post when I can, and continue with my life when I can't. But for cryin' out loud, don't act like somebody stepped on your toes when the end result of all this can be totally turned off, or even selectively turned off, by anyone who so chooses.

There, I feel so much better now. :)

OK, what's up with this??

[fable2112]

Lately it seems as though every time someone says "and of course I'm gonna get moderated down for this" or something similar, the comment gets moderated up. Either moderators are trying to prove how good they are by moderating up self-described "controversial" comments (so nobody will think they are biased), or posters are really overdoing the false-modesty thing. Or perhaps a little bit of both.

In any case, it's getting a bit tedious IMNSHO. It's not that I generally have anything against the comments contained in posts that also have this statement, but the statement would make me, personally, less-inclined to moderate the post in *either* direction.

A Most Excellent Idea!@

[fable2112]

[Speaking of moderation, someone moderate that up please??]

Of course, there is the slight potential problem of moderators who can't otherwise post to the discussion trying to get their two cents in that way, but limiting it to one line will cut down on that.

In addition, it would help pick out problem moderators. If, for instance, someone is moderating a post down and puts as a comment "You're a loser. Go away!" or something similar, we might have a clue that there's some personal bias going on. Well, that's not quite universal. I used to be a sysop on a YAWC BBS, and we used to put comments on people's Aidelines like "twitted for bad info." One such comment by another 'op was "Abusive Loser. I hate that."

Perhaps a better example would be the inevitable "Go away Katz" posts, if they were being moderated up with a "Yeah! Katz sucks, heh heh heh," or something similarly unproductive. Or moderating a Linux criticism down with "Are you nuts? Linux r00lz, dude!" You get the idea. Also, I know that a few times when I've moderated, I've wished for a "misinformation" category with which to mod something down,sort of like the opposite of "informative."

Like I said, though, the above poster has an excellent idea!

Problems with Integration

[nano-second]

Although the improved context aspect is appealing. The problem I forsee with integration is due to our threshholds.

I don't mind taking 5 minutes to meta-moderate and then be done with it. I don't mind, in that context, reading a few loser comments, which have likely been appropriately moderated.

However, if M2 was integrated, to properly participate, I would have to do it throughout the day as I read articles, with which I would probably lose patience. Or if many of the randomly chosen comments had low scores, I wouldn't even see them at all, since I like to read articles with a threshold of 2.

Furthermore, how many randomly chosen comments would be set for meta-moderation? As it is, I only can meta-moderate 10 per day, would this be kept track of if the comments were integrated?

---

Moderation and karma

[Enoch Root]

I would love to see moderation capability tied more closely to karma. In general, high karma individuals have a history of posting intelligent comments, and that would go along with being able to recognise intelligent comments. So increase the proportion of moderation power and probability for high karma! I have high karma, have been here for a while, and unless you consider reloading the Slashdot page 10 times on a workday to be obsessive behaviour, then I don't see why I haven't been given the opportunity to moderate yet.

"There is no surer way to ruin a good discussion than to contaminate it with the facts."

Re:Moderation and karma

[jflynn]

I don't agree that "Funny" should be treated differently. If anything, slashdot discussions need more and better humor. The tendency to take ourselves too seriously is nicely punctured by that occasional post poking fun at one or more sides. More Shoeboy, not less!

Disclaimer - never gotten a "Funny" karma point yet -- but appreciate those who can and do.

Having everyone always moderate is dangerous

[ucblockhead]

Solution: Allow all readers to be moderators all of the time. There will never be a deficiency in the number of moderators with this method. The number of people scoring articles will be directly proportional to the number of people reading that article. When 100% of the participants can cast votes, there is no outlying possibility if lack of moderation.


The trouble with this is that most people will soon cease bothering to moderate. When it is a rare occurance, people will take it as an honor, and work to do a good job. When it is a constant duty, people will slack off and not moderate. The only people who will moderate are those with very strong opinions on the subject, and I'd suggest that they are the least likely to be objective.

If you take KDE vs. Gnome as an example, with random moderation, you have to assume that a large percentage of the moderators will be in that largest class of people who doesn't greatly care one way or another and moderation will be fair. If everyone has the power to moderate, most people (the moderate moderators) will be sick of moderating and probably won't bother. The only ones who will are those who think it is really important to moderate posts up or down. Those are likely to be the more fanatical posters, and you won't get objective moderation.

Moderation will just turn into a new sort of internet poll, and we all know how accurate those are!

Ideally, you'd always want moderators who had no opinion on the subject. Since that isn't likely possible, choosing at random is likely the next best bet.

I do like your other ideas, though. The averaging of scores would help one problem, which is that in popular topics you seen pages and pages of fours and fives while in a lesser used topic, you might see only a couple of twos and threes. Obviously this has little to do with post quality!

More Moderation Madness

[HSinclair]

Although I do appreciate all the time you've spent coding in all this crazy moderation stuff, rob, I am somewhat concerned about some things.

A large portion of comments deserve no more or no less than the 1 (or 0 for AC's) that they get.. when you only display moderation controls on only part of the coments, almost all the comments with controls won't really be deserving of a moderation up or a moderation down. I think when this happens, some moderators, esp. the new ones will be just "itchy" to spend their points on something before they go away, and will moderate all the posts they can, regardless of whether they are deserving. (I may be wrong on this though)

What I feel might be a better solution than only having the moderation controls on a few of the comments.. Have a lot of moderators with only 1 or 2 points so your exampled Gnome bigot won't be able to moderate down all the KDE comments he sees , yet the "good" moderators will still be able to put the deserved -1 on those "Let's petrify nitrozac and hump her leg" posts.

Re:More Moderation Madness

[JordanH]

I like the idea of more Moderators with 1 or 2 points only.

Conversely, I feel that truly old hands with very high Karma should have more points. This will aid /. in maintaining it's culture and flavor.

What I'm thinking is something like:

[Moderator Points] = (10*([age of your account]/[age of slashdot])*[Karmic Factor])

[Karmic Factor] = MAX( [Karma]/[Nirvana Threshold], 1)

All of the above are self explanatory except the [Nirvana Threshold]. This could be some threshold that's quite high, possibly the 4th standard deviation positive from the distribution of all Karmas. Maybe a factor could be thrown in to lessen the influence of account age, but you get the idea. I'm not sure whether to round up or down the moderator points assigned. Of course, no fractional points should be allowed (hmmmm.... no that was dumb, no fractional points allowed).

This system has the significant advantage of being very geeky, but still understandable to someone without more than High School Algebra.

I really feel that your /. seniority should have a real weight in the matter, not just a cutoff (no newbies). I respect the old-timers who have been consistently moderated up in the past.

The problem with basing ability to moderate and moderation points on Karma alone is that newbies , and I'm probably guilty of this, might be working too hard to increase their Karma. This might tend to generate a lot of popular Comments that are really just there to get applause rather than real meat.

An advantage to allowing new moderators just a few points might encourage these new moderators to take great care in assigning them.

Another benefit to penalizing new accounts on a sliding scale is that it would tend to discourage people from creating new accounts just because they are in a Karmic hole.

One potential problem is that someone with only 1 point might be more likely to only moderate up. I know I wouldn't want to waste my point on some Troll. Possibly assigning points in pairs (+/-) would help here. I like this. Moderation points are actually quantum fluctuations.

Here's a question for Rob, What is the "average" moderation? + or -? If you assigned in pairs the "average" moderation would tend to be zero. Not exactly zero as some people might not use all of their points before expiration. Would this be a good thing or a bad thing?

In general, the idea is to increase the reliability of moderation, while allowing a broad cross-section to participate. A sliding scale, based on your Karma and seniority would help to promote this, I think.

a modest proposal

[John Regehr]

Currently, everybody sees posts that are ranked on the same scale: what the moderators think. I don't agree that this is (always) the right thing to do.

Rather, each reader should have the ability to see a customized view of /. based on their preferences.

For example, if I see a post moderated up to "5" and I think's it's a bonehead post, I would like to de-value the opinions of the moderators who liked that post. This is different from meta-moderation: I'm not saying that these moderators are bad and should be prevented from moderating, I'm saying that I don't want to see the results of their moderations.

Similarly, when I read /. (and the linux kernel dev list, and comp.lang.perl.misc) there are some people whose posts I consistently value. I would like a way to always read their posts. Conversely, I would like a /. "killfile" so that I don't have to read posts by people who have irritated me in the past. Again, I'm not talking about reducing their karma so they can't post - I want a filter so I don't see their posts.

Eventually, each user will build up a profile of posters and moderators whose opions they trust and distrust. One can picture people trading these profiles, merging profiles with other people's, etc.

As the amount of information on the net grows, developing a systematic way to figure out which of it you trust and which of it you don't is going to become a very important problem.

Didn't Brin have some sort of reputation management system in Earth? Reading up on this might be a good place to start.

Anyway, the point I want to make is the people have different tastes and want to read different subsets of all posts. There's no reason to force the same view of moderation on everyone.

Newsgroups/usenet does suck

[Ticker]

Unfortunately, alot of newsgroups do suck now, because there wasn't a decent retro-moderation system in place when the NNTP protocol was first built. It's either nothing, or all the way. When it was first designed, NNTP really wasn't intended for use by the general public at all, like alot of the Internet protocols. Now look at what a haven for spam newsgroups are. Out of 30,000 of them, only several thousand are probably useful now.

So obviously it's important for Slashdot to have a good moderation system, otherwise we might end up with a SNR equal to that of newsgroups.

The good thing about Slashdot is that we can add and take away as many features as we want, because there's no protocol involved here, only code. So we can try one thing, see how it works, and if it doesn't work out, try something else. I think though, that we have to try not to make the moderation system *too* fancy. If we make moderation too difficult and subject to review after review after review, then I think moderators will begin to ask themselves "what is the point?". I would hate to see something like this, for example, become reality.

On another topic, I think that meta-moderation in the comments list itself is a bad idea. Let's say I'm person A, or a friend of person A. Person A gets their post moderated down. I might choose to meta-moderate that post. Even if we ban the poster themselves from meta-moderating where their own posts are involved, we can't stop them from telling all of their friend to go there and do it for them.