Interesting Moderation Proposal

Kuro5hin is running a story with some interesting ideas for "the perfect moderation system". I'm not sure I care for the overall system but the idea behind it (of 'balancing' out parts of the site with strategic bonuses/penalties) is intriguing.

Comments

Resident Karma Whore, move over.

[Signal 11]

Okay, as the resident Karma Whore and de facto expert on moderation here on slashdot, I think I'm probably the one person here who can comment with authority on this article.

My gut feeling - it won't work. I think the ideal moderation system would be based on what is called the Delphi Effect, if I understand it correctly. Basically the more people you have moderating, the closer to the "true" rating it will be. ie: if 80% of people believe a post is +3, but 20% believe it is -1, it is +3. No averages. A sort of majority rules. It depends, however, on alot of people moderating.. to the point that there are more moderations than there are posts. But, my ideas aside...

The problem with this proposal is two-fold: One, it has no way to detect 1 person using 10, or a hundred, or a hundred thousand, accounts and thus biasing the voting system. It's a problem prevalent here on slashdot where the trolls have created throwaway accounts. Limiting on IP address doesn't work, because many are behind firewalls and hence multiple users can legitimately be on one IP. one account per e-mail address doesn't work either - e-mail addresses are easy to get.. often for free. The net result is a small group of determined attackers can destroy the system (sound familiar?).

The second problem is related to the first. Their idea of having the users rate themselves initially is a very good idea (rob, you paying attention?) but it suffers from the fact that someone can simply moderate their own posts, and gain a point advantage.. and we're right back where we started.

The key to moderation lies in accountability. You can create the best system in the world - but unless you can enforce some kind of "one person, one vote" standard, it will always be open to abuse.

Lastly, some advice for the kiro5hin maintainers - don't count on obscuring the statistical system to deter your attackers for long. The people who took down the slashdot moderation system did so in an organized and systematic fashion. These people are bored and have nothing else to do - but you DO and hence are at a disadvantage. Once the system is in place, PLEASE ADAPT IT - don't just deploy it and forget about it. It'll need to be tweaked, updated, maybe even entirely thrown out for a new system. Trust the wisdom of Strousoup(sp?) on this one - design the first implimentation to throw it away, you're going to do it atleast once anyway.

Cheers,

Signal 11

--

Re:Resident Karma Whore, move over.

[kuro5hin]

There's been a lot of misinterpretation of this article, due mostly to the useless writeup at the top of it. Thanks for the link, but when did Slashdot stories stop providing *any* useful information about the link?

Anyway, the mod system described is not the one on K5. It's in Glasscode, which is the system the article is about. Which does not run K5. (Note: I'm not sure if you understood this or not, Sig11, but a lot of other people got it completely wrong, so that's for them).

Lastly, some advice for the kiro5hin maintainers - don't count on obscuring the statistical system to deter your attackers for long.

Which part of the statistical system do we obscure? If you want a full and detailed explanation of how the various K5 systems work, see our relaunch article.

Otherwise, I agree-- the system always has to be evolving. Think about it-- you cannot create an automated system that isn't eventually susceptible to automated attack. It's that simple. You just have to make it hard enough to attack that it's not worth it, and use the lag time to keep ahead of the kiddys. And sooner or later, you lose the race, pick your ass up, and try again (c.f. this summer for K5).

--
There is no K5 cabal.

speaking of moderation...

[The_Messenger]

... I think Slashdot needs to remove the +50 KarmaKap. We all know it's not supposed to be about the karma, but why bother encouraging quality posts with a reward system if you can only go so high?

Some of you may feel that without such a cap, it becomes a game (as in "karma whoring" AKA "Signal 11's life"), but I think that's irrelevent. The only "downside" to this "game" are more interesting, informative, and funny posts -- not much of a downside, eh? Some call it "karma whoring", but I think that it's simply the knowledge of how to be a good Slashdot citizen.

Yes, I check my karma totals, but not because I feel it's a game. I use it to gauge the overall effect my contributions to the site by observing the rate of karma growth. And if my karma drops suddenly one week, I know it's time to cut back on the flames and trolls. ;-)

And yes, it's also fun. I'll admit that.

I understand that one of the major reasons for having such a cap is to avoid abuses like this: let's say Signal 11 post interesting, insightful, and funny things for a year. From my experience, an active, consistently good poster can, on average, easily get 100-200 in a year. This means that Signal 11 could start posting Goatse.cx links, and it'd be weeks before he loses his +1 bonus.

I have a proposal to solve this: if a user is moderated down more than 20 times in one week, he loses his +1 bonus for a week. If he is modded down more than 30 times a week, he posts at 0 for week, then at 1 for a week. If he's modded down more than 40 times a month, he posts at -1 for a week, 0 for a week, and 1 for two weeks. I think this not unreasonable, considering that most users with enough karma to get the +1 bonus won't suddenly become PBG or *syringe. :-)

Who else agrees that the +50 KarmaKap has to go?

---------///----------
All generalizations are false.

Moderating articles for /.

[Frac]

I wish I can moderate articles on slashdot. In fact, I wish there are little radio buttons next to each article that says:

I don't want to see this article again and again

or

Please post this article again and again and again

so CmdrTaco and his very diligent team will know which articles we want to see redundantly.

Slashdot - News for Attention Deficit Disorder. Stuff you saw yesterday.

On a slightly more serious note - surely Slashdot must be getting very uninteresting for the Slashdot admins if they arent' reading their own site. What does that say about quality control?

For starters, just /document/ what slashdot does!

[devphil]

Very little of how /.'s moderation and meta-moderation works is documented. How come my karma never goes above 64 even though I get moderated up? Why is it that occasionally it just drops a few points even though I haven't been moderated down? (Does karma age?)

Why can't we talk about moderation somewhere on Slashdot? If it gets brought up in a normal discussion, it's -1, Offtopic. I've never tried to submit a Slashdot article that concerns Slashdot itself, but the people who have say those are rejected.

How about a new category: -1, Herdthink, for those posters who just spew the party line about "and this is why Linux is so much better." At the very least they shouldn't be getting Insightful points for copy'n'pasting stuff from the FSF or OSI's webpage verbatim.

If we had a better FAQ, it would at least contribute to more "Insightful, Interesting, and Informative" discussion about moderation.

Enh, just my two timeslices.

Re:Spendable Karma

[DeadSea]

I most often see things I would like to moderate like this in older articles. It seems that moderators never hang out in anything that is no longer near the top of the front page. Here is my analysis of why it is happening and what can be done about it:

The first problem I see is that the first posts to an article are the most likely to be modded up. Moderators tend to hang around the couple most recently posted stories.

Not everybody refreshes Slashdot every two hours. The people that do, are the ones that agree most with the stereotypical slashdot agenda.

Insightful posts take time. It could easily take an hour to *read the article*, do some other research, and post some meaningful commentary. Those who post fast seem more likely to spout out their gut feelings.

To sum up: The people who post first are likely to be avid slashdot readers and more zealotous. Posts that are made soon after the article goes up are not as likely to be based on facts.

On hot trigger issues such as this one, I have read comments soon, then comments later and been pleasantly suprised by a couple better posts that get moderated later. Often on looking further, I notice that there are several more that I would have modded higher than the ones that are modded higher.

Let me try to illustrate this with a graph:

PostQualityv sTime:
|high
|
| +---+
| +++--+
|+-+ +---+
|++ +---+
|++ +---------------
|++
|+-+
|++
|++
|
|low
+-----------------------------------------
time-- ->

Sumofmoderationdone
|more +-------------------------
|&nbsp ;+-----+
|+--+
|++
|+
|++
|+
|+
|+
|+
||
|
|less
+-----------------------------------------
time--->

As you can see from the graphs, I think there are a lot of good comments posted later that don't get moderated, while a lot of earlier comments that might not be quite so good, do.

I suggest the golden moderation system.

You get 5 moderator points.
2 of the are gold.
2 of them are silver.
1 of them is bronze.

gold points can be used on any post at any time. Silver points can be used on posts attached to articles that are more than 2 hours old. Bronze points can be used on posts attached to stories that are more than 1 day old.

I think this would really do wonders for Slashdo

More ideas for slashot moderation

[aardvarkjoe]

All too often, there are posts that really need a +1 or -1, but the choices just don't cover it. Therefore, I say we need the following choices added:

+1, Troll
/* Really, we need more than just 'Funny' to reward the good ones. */
+1, Whore
+5, Signal 11
/* So we don't have to waste multiple moderators' time. */
+1, Slashdot Already Posted This
+1, Claimed They Were Expecting -1
+1, Redundant
/* Perhaps split into several categories, such as 'about [copyrights | patents | free speech | privacy | gun control | jon katz]' */
+1, Only Intelligent Post in the Entire Discussion
-1, Stupid
-1, Clueless
/* I expect these to be used rather heavily */
-1, Opposing view
-1, goatse.cx
+1, goatse.cx

There could be many more, of course ... Perhaps, instead of the list, we could have a textbox where moderators type the reason for their moderation.

Spendable Karma

[\\x/hite \\/ampire]

How many times have you seen a post that really really deserved moderation but couldn't do a thing because you weren't privileged enough to be a moderator at the time? Many times I'm sure. How about this... in addition to the usual "randomly chosen" moderator stuff, for each story a user would be allowed up to three moderations. The catch, each of these would be at the cost of one karma point each. Why would this work?

You need to be logged in to moderate

You must have karma to give karma

It would actually give some type of value to karma

Honestly, the only way I can see to abuse such a system would be transferring karma across account, but why would anyone really need to do that? ;-)