Slashdot Mirror
Interesting Moderation Proposal
Kuro5hin is running a story with some interesting ideas for "the perfect moderation system". I'm not sure I care for the overall system but the idea behind it (of 'balancing' out parts of the site with strategic bonuses/penalties) is intriguing.
Different people have different tastes in article zposts. Some people like funny posts others like informative. I believe collaborative filtering would work better than a set points system. Everyone votes on the quality of posts and your sorting depends on how closely you mimic others who vote like you.
CF can be integrated in various ways, but the method I'm thinking of would work similar to this:
1. Everyone votes on various posts at will. There is no limit to the number of posts you can vote on.
2. Your votes are compared against other users votes to find the difference in "taste" between you and every other user.
3. Posts are weighted based on the distance between you and other users. Users who vote like you and vote on posts are weighted higher than users who don't vote like you and vote on posts.
The underlying math to implement this is rather simple (with the exception of some of the baysian network models) and widely known, but as far as I know, there are only a few very limited open source implementations in existance.
The world is neither black nor white nor good nor evil, only many shades of CowboyNeal.
(FWIW I'm currently moving away from collectives such as /. and towards weblog-type things. My favourite source for the unusual and nerd-esq is Penny Arcade. Sure, it's mostly centered around a comic strip, but the journal entry / news that is attached to each strip is a very good read (even though it's a bit computer game centric). Otherwise I'm returning to traditional news vendors -- I have BBC, Fox, ABC (Oz), Salon & Wired all syncing to my TRGpro.)
The people who took down the slashdot moderation system did so in an organized and systematic fashion.
And if you want to see an example of Slashdot just before wide scale moderation and karma went in, check out: Slashdot Moderation Phase 1.1 . Even the Anonymous Cowards generally had good comments.
S/N ratio was pretty high in those days, much like kuro5hin now, but from the look of it, community moderation and a 'reputation system' has failed in it's goal to keep ratio up. Instead we've gotten a Karma Whore 'n' Troll circus with lots of anonymous flamebait on the side, which is entertaining and might generate more pageviews, but is probably not good keeping readership up in the long run. Which is what you'd expect when you let a site 'run itself'.
Unfortunately, it's probably only a matter of time before kuro5hin is 'hacked' and it's system has been rendered as useless as Slashdot's for encouraging valuable posts. You are damned if you do (let the readership moderate itself) and damned if you don't (admins end up censoring content, people rebel against 'elite' moderators.)
--
Business. Numbers. Money. People. Computer World.
You might want to start looking up some stuff about Autonomy. This is possibly the fastest growing tech company in the UK. They have a product that uses NLP to help with search solutions, agents, and knowledge management (to use the new-media-hype terms).
I have seen their products in action and they are *extremely* impressive. What might be of most interest to you is the way their algorithms work. Now I'm sure they won't release that info, but you may be able to glean enough info from their material to get on the right track. (Start here perhaps). Basically their technology is based on Bayesian algorithms (Bayes was an 18th century English cleric who came up with some cool ideas about NLP that couldn't be proven until recently when the computing power and information volume to make them work became available - how's *that* for far-sighted!) combined with Shannon's Information Theory. It is WAY powerful in practise!
I agree that it would be really cool to see some kind of automated moderation system based on this type of principle. I'd also *love* to see some Open Bayesian NLP work.
Good luck with the experiment.
"Give the anarchist a cigarette"
A little planning goes a long way...
How can there be a perfect system for moderation? Moderation is just that - a moderation of thoughts and desires, really nothing more than a system to balance the will of the masses against the will of the few. That is, letting people read what they want and ignore what they don't vs. being forced to read what they don't want and listen to what they don't like. Too little moderation and the little voices are drowned out by the big guys, too much and the same thing happens. So in that sense, it's really just a great big continuum.
I think that in the end, moderation is really just a personal preference issue and not worth that much effort. None is boring, some helps, but too much is worse. Though I must applaud the efforts going into this, and I do enjoy the debates the issue provokes, maybe instead of looking at it like and trying to build the "perfect" system, we might be content to just call it exploring options or do something more interested. But trying to "perfect" moderation seems silly.
It would be a good idea to allow someone to buy moderation, where:
1. the donor has more than 20 karma (e.g. is a good player);
2. the donor can push up a post only to a total max per post that levels it out at 2 points (e.g. nasty people mod down a good post to -5, nice donors can only spend it up to +2, cost of +7);
3. extra karma spent to buy moderation costs 10 karma for every karma given (e.g. +7 would cost +70 karma);
4. each donor could only spend 10 karma per story (e.g. you'd need to find 7 people to back you up in my example, and they would not be able to donate to any other post in that story);
5. a donor can only donate up to 10 karma once per day (to keep karma whores from abusing it).
Concept?
Will in Seattle
How About this:
...Then again, that'd probably be illegal in most areas of the world.
1) If you think you were unfairly moderated, challenge your moderator to a duel (Say... High noon in front of the Geek compound)
2) You beat each other until somepody passes out.
3) The loser has his picture taken in all his bloodied glory, and posted in a new 'moderation' photo gallery for all to see.
Maybe that will get the moderators to be a little less biased, having to worry about their own butt.
jdb
In all of the moderation systems I have seen proposed, there's something missing that keeps all moderation systems ridiculously stupid. It's called negative feedback.
/.'s "napster, mp3.com, linux" current portfolio. This kind of site would be a great place to get a fresh look on things.
All of the systems I have seen either totally lack feedback at all (for instance, Slashdot's miserable attempt called MM) or have some type of positive feedback - this proposal included. I may be wrong, but it seems that a "poster" with an "idea" can grow to wield power. Someone accumulates enough points, he or she can consistently post to the most active categories and be seen. What would this lead to? A few people who are very active and dominate the board.
I don't know what to propose as a negative feedback system for this type of situation. All I know is that I see a desperate need for someone to design a system in which all users share power equally, however cheesy that may sound. I would love to see a proposal that intends to keep power out of a select few individuals so we rarely see the same name twice. Everyone could have the power to moderate every story they see. The bad stories will quickly drop off the front page, and the good ones will stay. The site won't have the highest-quality news, but you simply can't have both.
What you will get from a site like this is a myriad of opinions and views from all sorts of places. It won't be CNN, and it won't be nearly as homogenized as
This is just an idea, but such a site would cater to the crowd that kiro5hin has attracted. Hence, I think it would provide a nice counterpoint to Slashdot, where one can get a more mainstream, less off-the-wall and imaginitive, quality news source.
Again, admittedly, I didn't inspect the original story too closely..I read the first half and skimmed the second. Just felt like giving something to think about though.
Every time this subject comes up, I see a lot of people arguing about variations on a global moderation system - I keep wondering why nobody brings up the possibility of collaborative filtering.
Basic idea: allow everyone to mark whether or not they like particular articles and/or authors, then use that profile for each user to find the articles and/or authors that user is probably more interested in.
This is even more useful when you compare user's profiles against each other so that if two users have been marking messages/authors in such a way that their profiles are very similar, then when one of the users marks a message/author favorably, it automatically gets a higher rating for the other user (or down, if marked unfavorably).
In effect, each user will statistically get messages rated through a moderation scheme based on their own past history, and the history of other users who happen to agree with them.
So, how come this kind of filtering/moderation system is never discussed? Is it too computationally difficult to implement on a large scale?
Cheers to this idea!
Even if it is just an added option to the current system. To make this transparent the spender/s should be identified!?!!!! That would be really interesting and would prevent siggy from dominating moderation for the first couple of weeks.
Just think...do I spend on this comment and lose my +1 for the good of humanity? If it was done transparently this would be even more interesting as your moderations would tell as much about a person as their posts.
I say lets do it.
no sig.
The problem is not moderation in any shape or form but instead human nature. It doesn't matter if there are a few moderators a la slashdot or a lot a la kuro5hin, people will generally appreciate an opinion that reflects theirs and be hostile to an opinion that is in conflict with theirs. If there was an open way of holding people accountable for their moderations, just as we can read vote histories in article submissions on kuro5hin, then maybe people would be more careful with their moderations (or it could become like slashdot where the many tyrannize the few via meta-moderation).
In my opinion all moderation is flawed because it relies on human nature which is inherrently flawed. I personally suggest reading without scores, after all USENET has no scores and this did not alter the quality of the discussions in several groups.
Second Law of Blissful Ignorance
Why else would they be home all day long to claim "f1rst p0st fuxxxxerz!!!JH(*)&# eye 0wn jew" and talk about naked and petrified actresses?
I always thought it was some form of performance art.
--
If you don't like a post, just click "-1, don't like it" and be done with it. Don't like a story, moderate the story as "-1, post-IPO-esque" and you're through.
Strong opinion about a particular user? Moderate every he has posted and ever will post. Just think, with out combined might we can hand out a bitchslap even the Taco would envy.
Heck, you even moderate moderation. If you disagree, there's "-1, wrong". If you like it, it's "+0, right" (because if its right, it doesn't need to be moderated). Hell, you can moderate the whole moderation system; if it gets low enough, it will be defuncted and Anarchy shall reign.
Moderate Slashdot as a whole...negative moderation will bring more stories confiscated from Kuro5hin, positive yields more Jon Katz stuff.
It's extends to the micro-level as well. Moderate the topic list, color scheme, each others' passwords...
Moderation at the bit-level may be difficult, until we get quantum computers and get moderate an individual bit as "+5, very" rather that just "0, false" or "+1, true".
My mom is not a Karma whore!
Something needs to be done.
The moderation on a lot of articles is just plain wrong. Or opinionated. Some moderators rate articles not on the merits of the content, but on how much it angers them or tickles their fancy. Articles that follow the groupthink of the cult get good moderation.
I'll see your senator, and I'll raise you two judges.
On problem, as with any universal rating scheme is that it would be easy to, say, create 20 accounts and consistantly mark-down a certain author, something you cant do on /. because you would have work the accounts up to moderator status first.
/., and the system allows for a finer grading of articles, but i cant really see it being better in the end.
/.s moderation scheme my two-pennies worth is that you could combine this scheme with /.s in a 3 tier manner. Any reader can vote a story up or down, however no effect registers until a moderator (generted in the /. manner) comes online. They are then presented with the top x movers, and check to see if the public vote is accetable. They can then ratify or veto the decision. It could even take 2, or 3 agreeing modrators before the story was moved up or down. Metamoderation would then take place in a similiar way to at present.
On my reading of the summary the system doesnt seem to have a way to deal with this. Indeed it doesnt really seem to be adaptable to deal with this.
On a positive note offensive comments should go down quicker than they do on
Since this is going to lead to an inevitable discusion about
However i dont really think that the system would be an improvement on the present one, which, given the circumstances, works rather well (though i might only be saying that because i got +4 karma today ☺).
My gut feeling - it won't work. I think the ideal moderation system would be based on what is called the Delphi Effect, if I understand it correctly. Basically the more people you have moderating, the closer to the "true" rating it will be. ie: if 80% of people believe a post is +3, but 20% believe it is -1, it is +3. No averages. A sort of majority rules. It depends, however, on alot of people moderating.. to the point that there are more moderations than there are posts. But, my ideas aside...
The problem with this proposal is two-fold: One, it has no way to detect 1 person using 10, or a hundred, or a hundred thousand, accounts and thus biasing the voting system. It's a problem prevalent here on slashdot where the trolls have created throwaway accounts. Limiting on IP address doesn't work, because many are behind firewalls and hence multiple users can legitimately be on one IP. one account per e-mail address doesn't work either - e-mail addresses are easy to get.. often for free. The net result is a small group of determined attackers can destroy the system (sound familiar?).
The second problem is related to the first. Their idea of having the users rate themselves initially is a very good idea (rob, you paying attention?) but it suffers from the fact that someone can simply moderate their own posts, and gain a point advantage.. and we're right back where we started.
The key to moderation lies in accountability. You can create the best system in the world - but unless you can enforce some kind of "one person, one vote" standard, it will always be open to abuse.
Lastly, some advice for the kiro5hin maintainers - don't count on obscuring the statistical system to deter your attackers for long. The people who took down the slashdot moderation system did so in an organized and systematic fashion. These people are bored and have nothing else to do - but you DO and hence are at a disadvantage. Once the system is in place, PLEASE ADAPT IT - don't just deploy it and forget about it. It'll need to be tweaked, updated, maybe even entirely thrown out for a new system. Trust the wisdom of Strousoup(sp?) on this one - design the first implimentation to throw it away, you're going to do it atleast once anyway.
Cheers,
Signal 11
--
Some of you may feel that without such a cap, it becomes a game (as in "karma whoring" AKA "Signal 11's life"), but I think that's irrelevent. The only "downside" to this "game" are more interesting, informative, and funny posts -- not much of a downside, eh? Some call it "karma whoring", but I think that it's simply the knowledge of how to be a good Slashdot citizen.
Yes, I check my karma totals, but not because I feel it's a game. I use it to gauge the overall effect my contributions to the site by observing the rate of karma growth. And if my karma drops suddenly one week, I know it's time to cut back on the flames and trolls. ;-)
And yes, it's also fun. I'll admit that.
I understand that one of the major reasons for having such a cap is to avoid abuses like this: let's say Signal 11 post interesting, insightful, and funny things for a year. From my experience, an active, consistently good poster can, on average, easily get 100-200 in a year. This means that Signal 11 could start posting Goatse.cx links, and it'd be weeks before he loses his +1 bonus.
I have a proposal to solve this: if a user is moderated down more than 20 times in one week, he loses his +1 bonus for a week. If he is modded down more than 30 times a week, he posts at 0 for week, then at 1 for a week. If he's modded down more than 40 times a month, he posts at -1 for a week, 0 for a week, and 1 for two weeks. I think this not unreasonable, considering that most users with enough karma to get the +1 bonus won't suddenly become PBG or *syringe. :-)
Who else agrees that the +50 KarmaKap has to go?
---------///----------
All generalizations are false.
--
I like to watch.
I don't want to see this article again and again
or
Please post this article again and again and again
so CmdrTaco and his very diligent team will know which articles we want to see redundantly.
Slashdot - News for Attention Deficit Disorder. Stuff you saw yesterday.
On a slightly more serious note - surely Slashdot must be getting very uninteresting for the Slashdot admins if they arent' reading their own site. What does that say about quality control?
Very little of how /.'s moderation and meta-moderation works is documented. How come my karma never goes above 64 even though I get moderated up? Why is it that occasionally it just drops a few points even though I haven't been moderated down? (Does karma age?)
Why can't we talk about moderation somewhere on Slashdot? If it gets brought up in a normal discussion, it's -1, Offtopic. I've never tried to submit a Slashdot article that concerns Slashdot itself, but the people who have say those are rejected.
How about a new category: -1, Herdthink, for those posters who just spew the party line about "and this is why Linux is so much better." At the very least they shouldn't be getting Insightful points for copy'n'pasting stuff from the FSF or OSI's webpage verbatim.
If we had a better FAQ, it would at least contribute to more "Insightful, Interesting, and Informative" discussion about moderation.
Enh, just my two timeslices.
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
The first problem I see is that the first posts to an article are the most likely to be modded up. Moderators tend to hang around the couple most recently posted stories.
Not everybody refreshes Slashdot every two hours. The people that do, are the ones that agree most with the stereotypical slashdot agenda.
Insightful posts take time. It could easily take an hour to *read the article*, do some other research, and post some meaningful commentary. Those who post fast seem more likely to spout out their gut feelings.
To sum up: The people who post first are likely to be avid slashdot readers and more zealotous. Posts that are made soon after the article goes up are not as likely to be based on facts.
On hot trigger issues such as this one, I have read comments soon, then comments later and been pleasantly suprised by a couple better posts that get moderated later. Often on looking further, I notice that there are several more that I would have modded higher than the ones that are modded higher.
Let me try to illustrate this with a graph:
PostQualityv sTime:
|high
|
| +---+
| +++--+
|+-+ +---+
|++ +---+
|++ +---------------
|++
|+-+
|++
|++
|
|low
+-----------------------------------------
time-- ->
Sumofmoderationdone ;+-----+
|more +-------------------------
| 
|+--+
|++
|+
|++
|+
|+
|+
|+
||
|
|less
+-----------------------------------------
time--->
As you can see from the graphs, I think there are a lot of good comments posted later that don't get moderated, while a lot of earlier comments that might not be quite so good, do.
I suggest the golden moderation system.
You get 5 moderator points.
2 of the are gold.
2 of them are silver.
1 of them is bronze.
gold points can be used on any post at any time. Silver points can be used on posts attached to articles that are more than 2 hours old. Bronze points can be used on posts attached to stories that are more than 1 day old.
I think this would really do wonders for Slashdo
All too often, there are posts that really need a +1 or -1, but the choices just don't cover it. Therefore, I say we need the following choices added:
/* Really, we need more than just 'Funny' to reward the good ones. */
/* So we don't have to waste multiple moderators' time. */
/* Perhaps split into several categories, such as 'about [copyrights | patents | free speech | privacy | gun control | jon katz]' */
/* I expect these to be used rather heavily */
... Perhaps, instead of the list, we could have a textbox where moderators type the reason for their moderation.
+1, Troll
+1, Whore
+5, Signal 11
+1, Slashdot Already Posted This
+1, Claimed They Were Expecting -1
+1, Redundant
+1, Only Intelligent Post in the Entire Discussion
-1, Stupid
-1, Clueless
-1, Opposing view
-1, goatse.cx
+1, goatse.cx
There could be many more, of course
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
You need to be logged in to moderate
You must have karma to give karma
It would actually give some type of value to karma
;-)
Honestly, the only way I can see to abuse such a system would be transferring karma across account, but why would anyone really need to do that?
``We are the people our parents warned us about.''