Keeping Web Discussions Open, Yet Civilized?

gsnedders asks: "With the rise of 'Web 2.0' and user created content, often in the form of comments, how do you keep the discussion open, yet civilized? I've found Slashdot's moderation to be very good — the good stuff gets moderated up, and the bad stuff down. On Digg, correct and valid information often gets dugg down, and offensive comments up, showing that having an open moderation system doesn't always work. However, moderation like on Slashdot, requires a large numbers of users to have enough moderators without giving everyone moderator access, therefore making it impossible to use on smaller sites. How can you keep the discussion civilized, while keeping commenting open, and not requiring large numbers of users for the moderation to work?"

Comments

Slashdot selects moderators automatically

[Animats]

Note that on Slashdot, not everyone has moderation privileges. Moderators aren't selected entirely randomly, either. Only users somewhere near the median posting rate are selected. This filters out both new users and overly active users. It works surprisingly well.

Now if only we could use it on stories, too...

Re:Slashdot selects moderators automatically

[Knightking]

You have to post occasionally, but posting too much pushes you over median very quickly. This will be my 25th post in the last 18 months, which is apparently fairly close to median, as I generally get mod points multiple times a week.

All relative

[Keebler71]

I've found Slashdot's moderation to be very good -- the good stuff gets moderated up, and the bad stuff down.

You've obviously never posted anything representing a remotely conservative viewpoint.

Re:All relative

[CHESTER COPPERPOT]

There does seem to be a problem. They really should be modded -1 Stupid, but the mods currently have to choose between Flamebait and Troll.

I disagree with using the word stupid but the essence of your post is a good point. The way /. moderates is confined by the words in which we moderate. Good posts are moderated insightful or interesting. Yet how many are genuinely insightful or interesting? In strunk and whites 'elements of style' there is a passage on the word 'Insightful'. It states the following:

The word is a suspicious overstatement for "perceptive." If it is to be used at all, it should be used for instance of remarkably penetrating vision. Usually, it crops up merely to inflate the common place.

How many times on /. have you seen an insightful comment that was of "penetrating vision" on a topic?

Perhaps to fix the language, that /. moderation is confined by, there should be some sort of comment field as to why you gave the moderation. But I guess that may have limitations as well.

Moderation is the key (most times)

[thedogcow]

Moderation is the key. Take a look a three sites.

Slashdot.
IMOH, I think Slashdot has a "pretty good" moderation system and meta-moderating is making it better. Most of the comments are are insightful or interesting is moderated as such. People come to Slashdot really for the comments because I think most of the readers are articulate and can provide something interesting/insightful to a story. It is really an intelligence thing. Generally, nerds are smart.

Digg.
I think Digg has a "fair" moderation system. One can see that it is fair to you if you think like most of the Digg users. Now, some can say that about Slashdot but stories are not deleted on /. just because they express something that a Digger doesn't want to hear like on Digg.com. I personally don't like Digg that much... or at least the comments and they don't offer or contribute that much to the parent story. I think Digg is mostly a trendy thing.

Fark.
I think that Fark has a "poor" moderation system. They let any yahoo express his or hers opinion. I think that the majority of Farkers are jobless alcoholics anyway... but that is besides the point... Most Fark comments are just random knee-jerk reactions. Moderators of Fark don't care... all they do is focus beer and naked people anyway... nothing insightful or interesting.

Re:Moderation is the key (most times)

[SuperKendall]

Slashdot...People come to Slashdot really for the comments because I think most of the readers are articulate and can provide something interesting/insightful to a story

Totally true. People complain about the moderation but really it generally works out pretty well. Someone else posted about an anti-conservative slant here and that is true; but moderators do mod up some conservative views and so even that is not as bad as it could be, you can get both sides of things at least some of the time.

Digg.
I think Digg has a "fair" moderation system. One can see that it is fair to you if you think like most of the Digg users.

The second part is sure true. Digg is like a flock of starlings, all moving alike... Digg I think is being utterly overrun but anti-conservitive thought with the most crackpot people getting stories onto the front page with regularity. I'm sure they'll do just fine with only half the people in the US reading it but it's really too bad.

It's not just liberal/consertive either, it's large blocks of momentium on all kinds of issues like macs/PC's, Microsoft in general, so on and so forth. It makes the front page seem kind of crazy for a variety of reasons and makes all sorts of comments take giant hits even if they are somewhat insightful but generally don't agree with popular opinion. The Slashdot ceiling/floor on moderation is a great thing in this regard as no comment can every be so buried it cannot be dug up; or so elevated that later evidence cannot bury something moderated high that turns out to simply be wrong,

Imagine wanting to go back to read a story in three days - would you choose Slashdot or Digg? I feel like after three days the Slashdot commentary would be kind of polished, whereas the Digg comments would look like a funhouse - all distorted. I pretty much never read Digg stories older than a day but afer a long trip I'll scan back through a week or two of Slashdot.

Fark.
I think that Fark has a "poor" moderation system. They let any yahoo express his or hers opinion. I think that the majority of Farkers are jobless alcoholics anyway...

I suppose the real question is; would any better moderation of Fark improve the aulity of people commenting there? Not sure it would.

Things that have been sucessful

[also-rr]

Group moderation like Slashdot -Pro, very hands off (once past a critical mass of users). -Con, promotes group think.

Wikipedia style moderation -Pro, very hands off (once past a larger critical mass of users). -Con, promotes group think.

Direct moderation (approval of everything) -Pro, very accurate. -Con, very time consuming.

Retroactive moderation (normal form style - post first delete spam later) -Pro, very accurate. -Con, very time consuming and crap still shows up until it's dealt with.

I have never seen a working system that was not based on one of these principles. Things that have failed:

Anything with no moderation at all. Look at usenet. These systems are only sucessful if combined with user filtering - one prospective area might be a system with very good user filtering, but then you shift the burden from the admin to the users and why should they bother when there are people willing to do the work for them?

To give you an idea here is a small graph of spam activity. It took 5 days for comment spammers to find an open site and start abusing it, and once they find something that has worked once they just dont stop. And that's even before you consider the malicious idiots who aren't exactly spammers but just twist and distort and abuse other posters - how do you deal with them exactly?

Re:Things that have been sucessful

[rolfwind]

While there is groupthink anywhere, having participated in both, I think something like Digg is more prone to groupthink than /. is.

I have seen /. consistently promote comments that went against standard group mentality, while at Digg comments I deem intelligent are shot down and many comments that I think are inane or at least plainly obvious get promoted to the heavens.

There, it doesn't even seem worth it to come up with anything more than a one sentence post (the other problem being the threads).

Part of it is that Digg doesn't have comments like "+1 Funny", "+1 Insightful", or "-1 Redundant" or any of that. Just points up and points down. Also, points up to infinity and down to negative infinity (or whatever the range is) gets ridiculous and really promotes group think I would say.

Don't get me wrong, there are things I like about Digg, hence I spend some time there. Things like group participation on articles bought to the front minus recent scandals and that there is not so much slowdown at night like on /. (get more international editors please, the world does not sleep when you do...)

I would think a blend of the two systems would be ideal, but /. has digg beat on moderation as far as I am concerned.

Re:Things that have been sucessful

[JaredOfEuropa]

It took 5 days for comment spammers to find an open site and start abusing it, and once they find something that has worked once they just dont stop. And that's even before you consider the malicious idiots who aren't exactly spammers but just twist and distort and abuse other posters - how do you deal with them exactly?

I propose door-to-door moderation of these characters...

More complex that just modding up or down

[Jack Pallance]

One of the biggest challenges that such a system would face is the fact the most sites will order the oldest posts first. Thus, these get read more times, and get proportionately more points up.

This would seem to make sense because other posts may need to be read in context with the previouse messages.

However, this same principle negates the effect that the later posts are often times more valuable that the first posts, because they incorporate thoughts from the earlier posts (usually more efficiently). That is to say, when a new topic is opened up, the earlier posts will make the most basic statements. The later posts will combine these into more complex, but relevant conclusions. But these later posts are the same ones that would not get modded up because the simple posts have "gotten in the way," and the readers never follow along long enough to get to them.

See also: SlashDot.Org

Don't be surprised

[Lazerf4rt]

Conservative viewpoints are discouraged on the Internet because the Internet has no central governing authority, thereby giving it a liberal bias.

Plus, all forms of ideology are gradually becoming unfashionable due to open communication on the Internet. Conservatism is more recognizable as an ideology and that's why it's targetted first.

And, there's the whole Bush thing too.

Re:Don't be surprised

[Aceticon]

Conservative viewpoints are discouraged on the Internet because:

• Theory A:
  

  ... the Internet has no central governing authority, thereby giving it a liberal bias.

  
  
• Theory B: Like minded people tend to group together (even on the Net) and you're not frequenting the right sites (ie, those with a conservative bias) and thus in your perception the Internet is a liberal place

I do agree that central governing authorities tend to promote the status quo - they obviously want things to remain as they are (since after all they have power and they would very much like to continue having it) - which by definition makes them conservative (as in, they don't want change).

However i don't think a lack of a central governing authority would switch the political gears on a whole information-exchange media all the way from conservative to liberal - to me, it seems more logical that the Net is more or less politically neutral and whichever slight political bias it might have comes from the demographics of it's users - if most of it's users are young people instead of old people, expect to find more opinions from apolitical, progressive and/or liberal people than from conservative people (in average young people tend to be in the "disapointed with politics"/"wanting change" field while old people tend to be in the "keep things as they are" field).

As more people join the discussions going on the Net (discussion groups, blogs, etc), expect that the range and intensity of the opinions being voiced on the Net more closelly match the "outside" world.

Note however that there are two factors which might skew what you see on Net vs what you see outside:

1. It's much more easy to express one's opinions on the Net than it is outside. The (semi-)anonymity of posting on the Net allows one to express opinions which are currently non-mainstream without the social risks of publicly going against the majority (like the risk of losing one's job for being a "radical").
2. In sites such as Slashdot you're in contact with a lot of non-Americans. This means that in here you're getting a much broader, world representative range of viewpoints which you won't get from mainstream American media (which in my good days i call the "navel gazers" and in my bad days i call the "circle jerk"). For example mainstream European political beliefs could easilly be percieved by Americans as having a strong "liberal" bias - allowing gay marriage, abortion, consumption of soft drugs and looking at the world as a complicated place in shades of grey - while in Europe we percieve American political beliefs as having a strong conservative-religious-moralistic bias - forbiding consenting adults to engage in non-mainstream behaviours, seing the world as "us the good ones"/"them the bad ones")

Quite possibly, the Net is much more representative of full range of opinions (political or otherwise) throughout the world than any local media would be (which tend to focus on the "accepted" mainstream opinions on a specific country). For an American the contrast might be even more glaring since American mainstream media seems to be even more guilty of navel gazing and always painting everything with the same two political colors ("Democrat" or "Republican") than most mainstream media i've been exposed to (the mainstream media of several countries, which mainstream media do have a tendency for navel gazing and for often using a restricted palette of political colors, though rarelly quite as extreme as the American one).

Too much discounting of late comments

[Anonymous Cowdog]

Early comments get most of the mod points.

Late comments, even very good ones, get almost no mod points.

Fixing this would help encourage civilized dialog by keeping some of the good quality commenters interested.

It's fine to reward early posters. But the magnitude of the effect is way out of proportion to what it needs to be, and it means that many excellent comments go unmoderated, just because they came an hour after the story instead of ten minutes.

What? Me, bitter? Heh.

Caste And Slums

[SuperKendall]

I know you mean well, but what I think ends up happening with a tiered system is that the lower tier always ends up being a ghetto, that upper tier people do not want to wade through - and newcomers cannot get through (or are not willng to) because of lack of upper tier particiapation and response to comments.

Lots of people have said on Slashdot they value replies more highly than high moderation, and I think that's true for a lot of people.

I just don't think any forum that doesn't let a user spontaneously join and start commenting is going to have an easy time attracting new members, the lifeblood of online forums.

Re:heres how!

[dshaw858]

However, moderation like on Slashdot, requires a large numbers of users to have enough moderators without giving everyone moderator access, therefore making it impossible to use on smaller sites

I don't think that this is necessarily true. For sites as large as slashdot, you need a large pool of moderators. For smaller sites, you'd need a much smaller pool of moderators. You could have median posting group moderation points given, plus dedicated moderators to overrule, or no mod points given at all- just the mod staff could handle a smaller site. For large sites like Slashdot, you'd need a large group of users to have moderators... which you would, since it's a large site. I'm quite tired so I hope this post was coherent, and hope that it helps :)

- dshaw

Group think is a con?

[Lazerf4rt]

What's wrong with group think? If you had an original idea, and nobody took it seriously, you might claim that it's due to group think. But if your idea got adopted, and became part of social consciousness, that would be due to group think too. So you can't really call it a con. It's just always present in discussions. If you really wanted to avoid it, you probably wouldn't discuss at all.

As one who leans to the right on slashdot...

[ChePibe]

I've found this to be somewhat true, but not always. If a post contains direct criticism of the DNC or praise/direct agreement for Bush, you can count on a few "flamebait" or "overrated" moderations.

For some posts, such as this one I receive all kinds of complaints about how "right wing" I must be to dare complain that the DNC still hasn't come up with anything resembling a platform. I receive angry ALL CAPS COMMENTS - DON'T YOU KNOW WHAT'S HAPPENING, IDIOT!?! responses. Still, it at least retained a respectable 4, insightful.

Then there's posts like this one which held onto its 5, insightful, but received responses like this where I'm reminded from someone about the left that I shouldn't "THINK", just do whatever the left says because what is happening is wrong, WRONG!

And all along I thought the right was supposed to be anti-intellectual...

I don't really care about the biases among editors, moderators, or whatever. I post what I think, and receive moderations accordingly.

I do, however, remember this when it comes to meta moderation time and, while acting within the rules, I act accordingly when I see posts modded inappropriately.

The moderation system, however, consists really of choir preachers - people mod up what they want to hear and mod down what they don't. That's all it comes down to.

Change the format of the replies

[Jack Pallance]

On Slashdot, comments are either hidden (as a subject line, or "Below Threshhold"), or displayed in their entirety.

Perhaps instead of showing comments in their entirety, you could show previews. The length of the preview could correspond to the mod points. For instance:

-1 = Below Threshold

0 = user name only

1 = user name + Subject Line Only

2 = user name + Subject Line and first line of comment (Or x number of characters)

3 = user name + Subject Line and first two lines of comment (Or x times 2 number of characters)

5 = Comment posted in its entirety

The usernames or subjects could be links to the entire comments for when the reader is interested.

The benefit would be an overall improvement in quality of post per inch of screen space.

Re:Build a Community

[stastuffis]

Civilized discussions are created by not only attracting people capable of such dialogue, but they also occur within an atmosphere which promotes it. Effective moderation with a clear set of concise rules is the quickest route to maximizing the "clarity" of discussions online. And while it is important to push a certain standard within any community, I feel it is also important to allow a certain freedom for error. I know that I've been on several forums that feel too restrictive in the ways I communicate. I felt like I was filling out a template, and it quickly killed my experience. However, there are always other factors to having "civilized" conversation.

For example, when a site, forum, chat room, or IM has established an "identity," then it will automatically promote such content to be pushed forward. Seriously, who goes to a hate group discussion to have an open talk? Anyone go to Myspace to have serious and informed debates about current events?

But for the most part, real world rules apply as well. Many people you encounter won't be capable of carrying on open AND civilized discussions, so why should the internet be any different? In fact, it seems more to be prone to acting out as escaping is easy as closing the browser.

This is a timely thread

[Dr_Ish]

This post will almost certainly get modded into the basement, but what the hell.

I have been watching the blogs and noticed an interesting phenomenon that arises there, as a function of the lack of moderation, quality control, or anything like that. The fact that bloggers can post what they wish and also delete comments that might challenge their positions can lead to some pretty unhealthy outcomes.

Although I hate it when folks post commercials for their blogs as much as the next person, I am going to suggest that interested folks might want to look at a couple of recent posts that deal with these issues. My post Poison Girls describes in detail the kinds of things that can go wrong when there is no quality control. Another post, Blogs and 'Community Solipsism' also deals with this issue. Both posts offer concrete examples.

I am an academic and thus have some investment in the blind refereeing process. It is far from perfect, but it keeps some of the worst excesses of 'anything goes' at bay. Hopefully, something like the slashdot system will get implemented for blogs too.

What really concerns me is seeing people who lack competence in a field, still pontificating at length on topics. People who do not know better tend to get sucked in by the more manipulative types. They end up listening to worthless advice, yet taking it as gospel. For instance, I have a colleague who is pretty much a failed academic. They have had nothing appear in print for over six years (however, they are tenured). Yet, they are currently offering apparently sagely advice on being a scholar. The putative advice is bad and misleading. Yet, there is nothing that can be done about it.

My best response is to remind folks that, as the Bard said in the Merchant of Venice, "All that glisters is not gold" and that is especially true in the world of blogs, yet it seems that the problems appear to continue. Any comments or suggestions on this matter would be very welcome. I live in fear of the day that a high school kid starts claiming to be a cancer physician and offering bad advice to people with serious health issues. In the blog arena, it appears that it is likely to be believed by some. This is a very scarey thought.

Re:This is a timely thread

[hankwang]

This post will almost certainly get modded into the basement, but what the hell.

I wish people didn't use this type of karma-whoring phrases. If the post is not utter flamebait or troll, it has a fair chance of being modded up by a moderator who wants to prove to himself that he can value a comment that doesn't agree with his own opinion. (I think I've done that myself one of the first times I had mod points. Now I never mod such posts, even if I think they deserve it based on the rest of the content.)

Re:What about a Spam Filter

[hankwang]

One thing I've done on my site is bayasian filtering of new comments.

Interesting idea. How well does it work as expressed in false/true positive/negative rates?

I can use those two corpuses (corpii?) to filter against.

Corpora. In Latin there is no such thing as an -ii ending, which is only used in l33tsp34k virus/virii. Corpus is a 3rd declension word with neuter gender.

Groupthink solution: ("Left-handed whuffie"?)

[s-gen]

If a post is simultaneously receiving loads of troll and insightful mods, the community is probably split along some axis. (left/right, mac/linux, sick-sense-of-humor/think-of-the-children, etc, etc). Groupthink is enforced when, for example, a well thought out conservative post is labelled "troll" because 6 liberal moderators have outvoted 4 conservative ones.

If however you could identify the prejudices of the moderators, you could build a system which didn't enforce groupthink like this. Instead, a well thought out conservative post could be labelled (say) +5 insightful/+5 conservative. A good, balanced comment would get votes from both sides (+5 insightful/neutral), out-and-out troll still gets canned, etc.

It could let you see *good* posts by people you disagree with. Or, if you're sure your mind is closed on some matter, it could let you NOT see them.

Lots of problems, of course. eg... identifying the axes, identifying the prejudices of the moderators (does the system let them tick a "conservative" box, or does the system work out for itself where the moderators are, or even what the axes are?)

Solution: Users who are willing to do some work...

[singularity]

I agree. In general, I tend to read at 2, Threaded, Highest Scores First, which generally shows me older, higher moderated comments (and any decently moderated comment made in response to it).

I find that most of the top four or five parent posts on an average Slashdot story have 2-3 moderated replies on them, at least.

When I have moderator points, I read at my normal 2/thread/highest, but never moderate anything I see. I then go back, change the story to 0/flat/Newest (ignore threads). Reading like this is fairly difficult, but I can generally keep track of what is being talked about from my original read. I then tend to moderate only comments scored as 0 to me (10% of new users, any AC comments), trying especially hard to get Anonymous Coward comments.

When I have moderator points, I generally moderate about one out of every three stories I read, and only about one or two messages per story.

My point? It is not the easiest thing in the world to do. I know that when I have moderator points I am going to to spend significantly more time on each story I read. In the end, though, I do not mind - I know that I am making Slashdot a better read for the people reading the story after me.

One additional comment: The same "early bird gets the worm" works in reverse on actual stories. A story at the very top of the homepage is probably going to get a lot more moderations than an older story sitting at the very bottom of the homepage, even though some very insightful comments might still be being posted.

I suppose on a small scale you could do moderations like Slashdot does metamoderation: Just choose some posts at random and have users moderate them up or down on a regular basis.

Conservatism is the opposite of ideology

[donscarletti]

Conservatism in its purist form is the opposite of ideology, it is a type of cynicism. It is the belief that the average decision made by a human is more likely to be bad than good. Ideology is a strong belief in a cause and the desire to base ones choices around this belief.

People like Dubya get mislabeled as conservatives as a bit of a euphemism, but conservative governments are careful, slow to act and even indecisive. George Bush however is a cowboy and could never be any such things (with the exception of Hurricane Katrina I guess), he likes to start silly wars with dubious causes, he tries to make the rich richer at the expense of the poor, this isn't conservative at all, this is changing the status quo. A conservative government would have stayed out of Iraq and just bombed Afghanistan until they were sorry for helping Osama.

If you want to see conservative, you need to look at Japan, in Japan the bureaucracy has the power so nothing ever happens, laws take decades to pass (child porn was only banned in 1998 IIRC) and havn't invaded a country for 65 years. The American government knew that if Japan was to loose its bad habits its government would need to be slowed to the point of total stagnation and now they are a nice country that everyone but China likes.

And by the way, how can something be biased because it is totally open? A person can be biased for sure, but society as a whole tends to be right about most things political, because politics are dictated by society. Which leads me to my next point, society has thought some really crazy things over the years, but its always been right about it. These days ephebophilia is out but homosexuality is in, who could have seen that coming a century ago? Back in the 70s western women were fighting for equal rights at the same time Muslim women were putting their headscarves on for the first time in almost a millennium. Portugal deposed its hardline Catholic government at almost the same time as Iran had its Islamic revolution and brought about a theocracy. Being conservative is to acknowledge society's indecisiveness and leave it alone. This is cynicism to the highest degree.

later posts good for "late to the party" mods

[quist]

For occasional/irregular readers blessed w/ mod points, the late comments are a good area to sprinkle moderation joy. The early comments are often already moderated into position before "Capt'n Occasional" arrives.

After a quick scan of the early posts, Capt'n should heigh on down toward the end to trawl for gems.