More Moderation Madness

The quest for a better moderation system continues onward. In response to the huge amounts of feedback I've received in the last 24 hours, I've implemented a few changes to the system that you should note. Click the link below to read about some new features, include the upcoming system for "Meta Moderation".

Karma
By far the most popular topic is karma. When it was just called "Points" nobody really cared, but now that I changed the name to "Karma" everyone has input on it. Must have a few MUDers out there. Since I intend to start using karma in a few other places, I added a field to display it on your user preferences page. I might add it to the comments display, but I'm holding on that for now since it will just clutter things up.

Default Comment Scores
For many moons now, users with high or low karma were given either a +1 (k:25) or -1 (k:-10) on their comments. Yesterday I added an additional -1 (k:-20) that I have since removed. Many people argued reasonably against it, so its gone.

In addition, I added a much requested feature to allow posters to optionally pass on the +1 bonus when they post. Many people who have earned the bonus point occasionally wanted to say something that they didn't feel deserved the bonus. I guess thats fair.

Anonymous Posting w/o Logging Out
Due to popular demand, I've added an option to allow logged in users to post anonymously. If you use this option, you are every bit as anonymous as you would be if you had logged out, except if you have a +1 bonus, your comment will still get it.

At some point I may eliminate the old AC posting in favor of this one, if only to eliminate a certain amount of knee-jerk posting, but I'm not convinced on this one. I really believe that people should be able to post anonymously, and this system while it still allows that, it would require a login. Its just a hoop- the comment is every bit as anonymous, but I suspect I'll take some flame for making people jump through the hoop. Then again, the flamers are probably a large part of the problem ;) Anyway, I'm not sold on the idea, so I'll probably leave it as is.

Meta Moderation
So who moderates the moderators? Every day comments are mailed to me with a note saying 'this comment was unfairly moderated'. Sometimes they're right. Sometimes they're wrong. But regardless, it seems like the community should be able to regulate this itself. So I've implemented MetaModeration.

I'm debugging it now, and it should be online within the next few days, but I want to post the concept for evaluation:

Basically, anyone who is eligible to moderate is eligible to MetaModerate [M2]. (Registered users with non-negative karma who have had accounts for "awhile" (the definition of which is likely to change but right now is probably a month or so)).

An M2 gets 10 comments, and the moderation done to them. They are then asked to decide if the moderation was fair or unfair. The opinion of the M2 affects the original moderators karma. In otherwords, if you moderate good, you get better karma, you moderate bad, you lose the ability to moderate in the future. As a side bonus, users will get some karma (on a sliding scale so it won't be much) for being an M2.

Its just a thin layer of accountability, but if everyone plays fair, it'll work. (Just please don't start asking for M3 or M4 moderation or I'll start crying).

Some other stuff
So I labored on my labor day (and Andover even has it officially listed as a holiday!) Thanks for all the feedback (good and bad!) in the last few days, please keep it up. I'm sorry we can't implement all the suggestions, but of course some of them aren't feasible, and some are just silly. But as a whole, I think we're getting better. (Or at least my TODO list is getting shorter ;)

Comments

Moderation of AC's

[Anonymous Coward]

Somewhat related: I've noticed that when an AC's posting gets moderated up once, it gets a score of 1 (e.g. Score:1, Insightful), whereas a non-AC's posting gets moderated up once to a score of 2 (e.g. Score:2 Insightful).

Now, it seems to me that an "Insightful" post should have a score of at least 2 no matter what the source. So if a moderator marks an AC's post as Insightful or Informative, it should jump immediately from 0 to 2. I realize this can get complicated really quickly, for instance, if an AC posting with score 2 gets marked DOWN, does it go back to 0 or 1?

Just something I noticed...

[Skyshadow]

Ever noticed how nothing gets moderated about 3 anymore? I mean, I've seen some really great stuff lately that gets stuck at 3 and stays there. I wonder if there could be some incentive to moderate up...

----

Moderator Moderation is Excellent, in Moderation

[Seumas]

I thought of skipping this article all together and not posting to it. Sometimes one more opinion is one opinion too many. But I love Slashdot. It's my morning coffee. It's something I look forward to throughout the work day and it is a wonderful source of current events and professional insight. Even at its worst, it deserves more respect than 95% of your typical media-slop and geek-pages.

Unfortunately, Slashdot regresses at times. It grows a square jaw and furry, burrowed eye-sockets. It loses stature and walks in a hunched over lurch. In these times, it can not articulate its thoughts and resorts to grunting and pointing. And each time, even though it manages to return to its typically evolved state, a lot of us worry just a little that it may get stuck in that regressed period one of these days.

Many of us saw Slashdot regressing again this weekend. The entire Stevens article was appalling. What should have been a brief chance for people to thank the man and talk about his achievements and contributions became a free-for-all where civility evaporated from most of the posting souls and floated into the ether.

I would never have imagined that the people who frequent Slashdot would ever have conducted themselves so astonishingly and with such a lapse of sympathy. I thought that most of us were professionals. Professionals would not walk into the office and, upon the news of a co-worker's death, start bad-mouthing them and standing on their operating system / programming language / philosophical soap-box.

That article seemed to be the Columbine of Slashdot. The call to action, if you will. Now everyone at Slashdot seems to be frantically looking for a solution to allow us to moderate ourselves when we can't conduct ourselves in a tolerable manner on an individual basis. (I can only imagine what the people at Andover.net must have been thinking of this whole drama.)

But moderation alone is not the answer. We're familiar with the cliché that "absolute power corrupts, absolutely", but whatever degree of power is given to a person can be equally corrupted to that same degree. There are always going to be people who will use their small chance to have power to get a laugh or wreak a little havoc. Even something as simple as ticking the score of a post up and down on Slashdot can't be trusted to some people.

After reading the Moderator FAQ the last time I wielded moderator privileges, I was of the understanding that the pool of posters was plucked from the group of Slashdoters who were in the median range. That is, they made the average number of posts, visited Slashdot the average number of times and did not have a history of heavily negatively-moderated posts of their own.

Which means that the people who abused that chance were the same people who had been treated fairly in the past by having their posts moderated appropriately.

So the answer Rob has devised is to moderate the moderators. This strikes me as a parallel to fixing a government problem with another government program (making government larger).

I actually agree with this idea, though. The average Slashdot reader is probably a rather agreeable sort who isn't going to misuse his or her points. So when the occasional misuse occurs, the chances that one of the other moderators will correct the misuse is pretty high.

A typical scenario that I've seen is the following, which occurred to me (I'm using myself as an example, but I have seen others have the same experience):

Out of four or five posts I made in the last 24 hours, two of them were marked as flamebait -- without cause, I feel. Yet, two of the other posts were moderated to a +2 and a +3. None of the posts were written with the intent to disrupt, inflame, anger, disturb, insult or offend anyone. All were intended to share one person's view-point and eventually, disappear into the Chasm of Old Slashdot Posts.

One of my posts seemed to be moderated strictly out of bias or maliciousness. While the post had been ticked to flamebait, it simultaneously drew a half-dozen emails from fellow Slashdotter's who agreed with what I said and thanked me for voicing it. A couple even went so far as to say that they intended to make a donation to W. Stevens' favorite charity (which was discussed in an earlier post) after reading my comments.

My comments were not particularly insightful. In fact, most of my comments are not particularly insightful or funny, even when moderated with such notation. But being moderated down as a result of a moderator's misuse of the system is aggravating and frustrating. Especially when it reduces a post that, at least, had some heart and thought put into it to the same section of Slashdot as the "I'm jizzing in my pants" and "first post" comments.

I hope that when people receive their moderator-moderating points, they do not just skip them to move on to moderating un-moderated articles, but take the time to browse the previously moderated ones, too.

I also hope that there is a way to allot karma to moderators. Those who use their moderation points wisely could, perhaps, get an extra point or two. Those who squander them and are constantly over-moderated by other moderators who come along and clean their mess should lose a couple karma points (in the very obvious and malicious instances).

Moderation seems to be a very fickle thing around here. Sometimes it lifts a hum-drum comment far above the level most would agree it belongs at and other times it kicks a well-deserving comment under the rug to be ignored.

Nobody knows the absolute and correct answer to the whole problem, but unless moderation is done wisely, fairly and reliably, a lot of people are going to be discouraged from participating.
---
icq:2057699
seumas.com

Nameless can be good...

[Fong Sai Yuk]

I have one reason I want to keep anonymous posting around. It's the fact that we can get insiders from wherever (Microsoft, Red Hat, Apple, ect.) to post without implicating themselves. While the usual grain of salt applies with most AC's, I have seen plenty of anonymous posts that had to have been from people inside the industry. That kind of information can't be had any other way. Allowing AC's can keep us in the know.

Makes you wonder how many high and mighty people in the tech world are AC's here....

My two cents...

[n8_f]

Here's my two cents.

1. Karma
   The way I understand your implementation of karma, every post I make influences my karma and goes into the number to be averaged. This means that every post I make has to be awesome. A large number of responses to other people's posts w/ questions or clarifications aren't likely to be moderated up, so when I do post my "awesome" comment, the effect of the moderation applied to it is watered down by all of the unremarkable comments I made. I think that only moderated comments should count towards karma.
   
   
2. Default Comment Scores
   I think that the opposite approach should be taken. The default post wouldn't have the bonus, with a check box to enable it. I see a lot of lame comments at a 2 or 3.
   
   
3. Waiting Period
   You mentioned in the moderation FAQ that you were thinking of implementing a waiting period inbetween posts. I think this is an excellent idea. Imposing a half-hour or hour waiting period between posts would be fine. It would force /.'ers to think about their posts (something too few posters seem to do). (Maybe also remove submit and go instead to a preview screen? Posters could then just hit submit, so in wouldn't be too intrusive, but a lot more people would hopefully proofread, since it would be there. But I digress....) Of course, maybe have some mechanism for posting small questions or clarifications. How about combined with my suggestions above? The default for posts is that they don't get your bonus and that they aren't considered for moderation? Small questions, requests for information, and clarifications are released under this, with perhaps a five minute waiting period. Larger comments that you do want moderated (interpreted as meaning you have something new and valuable to say) are posted with the bonus/moderate box checked, with a subsequently longer waiting period between these types of posts. Most posters seem to read all of the comments responding to their post, so they'll see the comments anyway (perhaps instead of a 0 score they are just given an "unmoderated" status, kind of like NULL). Impose a size restriction on these so that anyone giving a large opinion can't try to dodge being moderate or has to post as an AC (which has no or fewer restrictions).
   
   
4. Moderation
   This is where I want the biggest change. The current moderation system penalizes well thought out comments. The sooner you post, the more likely your chance of being moderated up. Later posts have little chance of being moderated, because most moderators have moved on to newer articles. For example, I posted to the Berkeley removes advertising clause article. I researched copyright law and posted a article related to the many posts on GPLing BSDL code. However, because I posted on Saturday, two days later, there is no chance of my comment being moderated.
   And what about the commments that are a few layers into a thread? What about the comments tacked onto the end of a thread? The point is, there are a lot of comments slipping through the moderation cracks. I think that you need a radical change to the moderation system.
   Here's my proposal. I realize that it is probably extremely difficult to integrate any of this into your system, but maybe some of the ideas can be adapted to fit in.
   Instead of giving moderators points, give them articles. You use the current system for selecting moderators, except you do a check everytime someone accesses an article. So when Joe Schmoe tryies accessing the "Microsoft's new evil plan..." article, his chances of being a moderator are checked. If he isn't selected, he goes onto view the page normally. If he is, however, he is taken to the moderator version of the page and given, say, 5 comments to moderate. The page looks like the standard version. The article at the top, comments below, but inbetween is the listing of his articles to moderate, listed similiar to the way your articles are listed in your user page. As I mentioned, the rest of the comments are below it, but the catch is nothing (including the 5 to moderate) has a score displayed. Now, Joe's job is to read those 5 (or 3 or whatever) comments (and any of the other comments) and give those five a score. Once he does that and submits it (or he clicks the "Decline" button), he is taken to the standard version of the page and he is free to post comments.
   Couple more things.
   
   • Comments with the least number of moderations are given preference.
     This way, one comment doesn't get moderated 50 times, and hopefully all comments get moderated at least once. If the threshold for moderator is set high enough, every comment should get moderated multiple times, with the final score being the average of those moderations, possibly weighted by the moderator's karma.
   • Moderators only overlap on 1 (if they only moderate 3 articles) or 2 (if they moderate 5 articles) articles, maximum.
     This is done in the scripting, so you don't have two moderators moderating all of the same articles.
   • Because this doesn't really work with a small number of articles, perhaps you have a threshold of like 20 articles, after which moderation is turned on.
     In fact, you could not display any comments until you get 20 (or 10 or whatever), which would cut down on those annoying posters who try to get first post (like it's a race or something). Or, perhaps you have something like a moderation acceleration curve, where the chance to moderate is modified by the number of posts.
   • People can moderate multiple times (perhaps only do the moderation check once per hour or day).
     So long as comments from any thread they have responded to are banned, there is no reason they shouldn't be able to. It isn't very likely that they would remember what score articles have gotten if they had been there before, and they would most likely get newer articles that they hadn't seen yet. This way, the more traffic a discussion gets, the more moderation.
   • People set their maximum number of comments to moderate.
     Everybody starts at 3, and the better moderators increase in the number of articles they moderate, up to their maximum. Also, perhaps people with a default bonus can only be moderated by people with an equal or greater default bonus. This prevents trolls from getting a large number of articles to mark down, or getting a chance to mark down people who may be over their head. : )
   • Expand the moderation system to include multiple categories.
     Many /. posts have horrible grammar and bad spelling. I've seen many posts over this issue, back and forth, and although I do feel for those people who don't speak English natively, a lot of it seems to just be linguistic laziness. I'm sorry, but how you communicate your point is almost as important as what your point is. You may be the most brilliant person since Stephen Hawking, but if you can't communicate your ideas in a way that other people can understand, it doesn't do anybody any good, does it? I'm not a grammar Nazi, I don't mind capitalization or punctation mistakes (afterall, e.e. cummings made it into an art form), but when I have difficulty understanding a poster's point or even determining what their point is, it's a problem. So writing should definitely be a category. Relevancy should be another. Perhaps something like idea or originality (redundancy). And then an overall or "must read" category. There are some posts that are hard to read and might be irrelevant to the current topic, but that should be read by everybody. Then users could set thresholds on the individual categories.

These are just some ideas. I have tons more, if you're interested. I realize implementing any of these may be impractical, but maybe they'll trigger something that is. I think that encouraging more moderation is only a good thing, and by increasing the moderation threshold so that every comment gets moderated multiple times, you'll increase the usefulness of the moderation system.

Thanks,
n8

Is "logged-in anonymity" really anonymous?

[Luis Casillas]

I think this is a very serious question. If I log in to slashdot and check the "Post Anonymously" box in a post, clearly my handle won't be revealed to the readers. But does the system log that it was me who posted anonymously? And what guarantees can I possibly have to that effect?

If in the future, people are required to give a handle or an email address to post "anonymously", their identity could be compromised, since /. would have an email address that might be possible to use to track down the person.

Not that I don't trust Rob and Co. with my privacy-- they have proved time and time again to be reliable. But such a record of the source of "anonymous" postings might even be exploited against their will.

Case in point: some guy X posts something "anonymously" in slashdot that offends some powerful company. Company considers the post to be difamatory, and demands slashdot give the email address of the poster. Slashdot refuses; the company sues slashdot for their posting records.

Even if the suit is unsuccessful, it wouldn't definitely be nice for /. to get harrassed.

---

Anonymous Cowards Are Necessary

[Effugas]

I mentioned this in another thread, but bottom-line "Anonymous Posting Is Golden" evidence can be found at the following URLs, posted during the NSA Backdoor discussions:

http://www.slashdot.org/comments.pl?sid=99/09/03 /0940241&cid=13
http://www.slashdot.org/comments.pl?sid=99/09/03 /0940241&cid=79

Read these to understand why AC posts are excellent to have around.

Yours Truly,

Dan Kaminsky
DoxPara Research
http://www.doxpara.com


Once you pull the pin, Mr. Grenade is no longer your friend.

moderation of extremely nested comments

[mcc]

i've noticed that sometimes you see real gems maybe three or four levels of replies deep in a conversation.. i mean, stuff that is within the context of that particular thread, but still brings up a point that everyone ought to see.
The thing where it always displays the full text of posts score:4 or higher is nice, but the problem is the moderators usually don't bother reading messages nested that deep. so they just scroll by without even knowing that message is there, and it stays at score:1.

maybe you ought to set up some kind of flagging system, something where everyone, even non-moderators can say "this derserves moderating up".. and even though it wouldn't count in points, a little red icon invisible to non-moderators would appear next to the posting's listing in the parent threads. so that the moderators would know it's worth looking at.

I dunno how well this would work-- seems pretty easy to abuse. You'd have to set it up so people couldn't flag their own posts.. might be worth thinking about though.

Fixing Slashdot Moderation

[Artie FM]

*I also posted this in the slashdot poll*

The Steven's piece has pointed out several weaknesses in Slashdot moderation. If Slashdot moderation were ideal all you would have to do is set your threshold to 1 and you wouldn't have to worry about this. This is not an ideal system ... so we should think of ways to make it better.

Here are some of the previous proposals:
----------------------------------------
1) Make it easier to score thing negative.
The current moderation system can be overloaded by spam like attacks. This is useful because you want moderation pts spent on finding good posts, not weeding out bad ones. But how do you make sure this is not abused? or that valid but possibly inflammatory comments are not marked down? Both things will happen.

2) Eliminate Anonymous Cowardice...
This sounds extreme.. even if we do something simple like requiring a mail address to post I think many good comments from knowledgeable
sources will not be made.

3) Combat moderator overload by making more moderators
The problem here is that average posts may get marked higher than they should.

Here goes what I propose as a solution.
--------------------------------------
1) Make the score system secondary to classification system. Let moderators classify posts as "Funny", "Informative", or "Flamebait" without spending pts. This way a post might show up as 30%flamebait but 70% funny. This make fair moderation a non-issue because every moderator gets a say about every post. Add a second category to measure the quality of the post as "Must Read", "Good Read", or "Average" this might cost points to moderate or simply be an average of what all the moderators think.

2) Instead of having moderators make only 5 moderations take a look at post volume on that subject. Discussion threads with a very high number of posts need more moderation than normal. Detect this and let moderators make multiple changes cheaply in these threads.

2) Personalize the scoring system for each reader. Already in preferences there are ways to add pts to long posts or subtract from short. This would be an extension to let the reader decide what kind of posts show up. Here goes a list of possible features.
1) each feature listed here may be turned on or off in preferences
2) add/subtract pts to posts from certain authors.
3) subtract pts from all AC's
4) add/subtract from certain types of posts -i.e. humor +1 flamebait -1.