Slashdot Mirror
Ask Slashdot: Best Practices For Using a Reputation Engine To Rate Information?
GrantRobertson writes: For my graduate project, I am considering developing a web engine designed around sharing and organizing actual information in a way that people would actually like to and easily be able to use it. Unlike a wiki, the information will be much more granular with lots more metadata and organization. Unlike a web forum, the information will be be organized rather than dispersed throughout thousands of random posts, with little room for dominant personalities to take over. While I like Stack Overflow, I am planning far more structure. While I enjoy the entertaining tangents on Slashdot, I don't want those to take over sites created using my engine. Naturally, there must be some way to prevent armies of bots or just legions of jerks from derailing web sites created using this engine. Given that, what would you say are some good rules to include in the reputation engine for such a site. What kinds of algorithms have you found to be most beneficial to the propagation and spread of actual knowledge. What would you like to see and what have you found to be dismal failures?
you are counting on Slashdot to do your graduate project for you? That is a horrible idea in so many ways...
Pretty much any set of algos is going to be easily defeated by humans trolling and no system is going to be anything near perfect. My thoughts;
1) Create a small set of simple, concise rules that are inviolate
2) Have a system so people can mark submissions as good (no rules broken/useful) or bad(rules broken)
3) Have your referees do nothing but determine if that submission is breaking one of your rules
4) Based your user trust as a derivative as how the user voted compared to what the referee votes
The theory is any controversial submission is going to get flagged & referees attention. Their job is limited in scope to just determining if the post breaks the site rules or not, nothing to do with quality / content / opinion. If users are trying to game the system their votes are going to conflict with the referees so their user trust is going to go down, whereas if people agree their trust is going to go up.
Eventually you'll have a group of users that you can generally trust to do the right thing so you can weight their actions accordingly.
Obviously there are some weaknesses;
- Referees are pretty much god (that's why the scope of their power is extremely narrow and simple)
- You can end up with hive mind (though you can combat that if enough trusted users conflict with other trusted users). I'd argue it's a way better protection than pure crowdsourcing ala reddit where the demographics crush submissions into hivemind
Just tossing that out there off the top of my head. It's not something to replace automated reputation management, just something augment it and limit some of the abuse.
Isn't the whole point of thesis work that you find some novel solution to a problem through your own research not enlisting others to do it for you?
I have a brilliant answer to your question. But it seems like you want it answered for a big shiny price of "free". I'll keep it to myself. Oh, and if you are thinking of having a contest and hope to get my idea without actually paying for it (and no, having a contest is not it), you can forget about it. I won't submit to any such contest. If you want data analytics ideas start paying people who spend time of their lives learning how to do data analytics.
Any guest worker system is indistinguishable from indentured servitude.