Slashdot Mirror
Developing a Vandalism Detector For Wikipedia
marpot writes "In an effort to assist Wikipedia's editors in their struggle to keep articles clean, we are conducting a public lab on vandalism detection. The goal is the development of a practical vandalism detector that is capable of telling apart ill-intentioned edits from well-intentioned edits. Such a tool, which will work somewhat like a spam detector, will release the crowd's workforce currently occupied with manual and semi-automatic edit filtering. The performance of submitted detectors will be evaluated based on a large collection of human-annotated edits, which has been crowdsourced using Amazon's Mechanical Turk. Everyone is welcome to participate."
Apparently, how their vandalism detector works right now is by automatically reverting any edits done by anonymous editors.
(And yeah, that's a bit sarcastic, but somewhat true.)
Welcome to Slashdot. Although everyone is welcome to contribute to Slashdot, at least one of your recent posts did not appear to be constructive and has been modded down. Please use TrollTalk for any test edits you would like to make, and read the welcome page to learn more about contributing constructively to this web site. Thank you.
Whoever posted this clearly isn't aware of the actual work being done in the field. For instance, I was running an anti-vandalism bot in 2006, and it wasn't new at the time. They've gotten gotten much more sophisticated since then.
Why are they so intent on reinventing the wheel? Do they not even realize that the wheel exists already? Why not just improve on it instead?
Cyde Weys Musings - Scrutinizing the inscrutable
I've had many more problems with admin abuse than vandalism. Vandalism is quick and easy to deal with. Admins are the biggest problem in Wikipedia editing; they have no accountability and abuse their power.
How about a log of each admin's activities, including reversions, bans, etc, and a way for non-admins to challenge actions (without spending countless hours in an appeal process worthy of a federal court).
In response to whether those two examples are vandalism, the answer is no, they are not.
You'd need a strong AI to be able to make those determinations, and if such a thing existed, it'd make more sense just to have the strong AI write the encyclopedia.
What we're talking about here is obvious vandalism (blanking, insertion of curse words, etc.) of the type that can be detected by an algorithmic/heuristic program.
Cyde Weys Musings - Scrutinizing the inscrutable
Before any more detectors are rolled out, how about they come up with a workable definition of vandalism? And actually use it fairly, ethically and logically.
There's a great deal of evidence to suggest the current definition of "vandalism," is something a wikiadmin decides he just doesn't like, or disagrees with, or in some way interferes with his power-trip.
Right now, you can think of wikipedia as having two columns per article - first is the working article column, with the second being the discussion column.
What we really need is a third column, one for the currently published version of the article.
While this may not be popular, it would go a long way to getting rid of the spam, and might even solve some of the other issues facing wikipedia.
With such a system, you could even assign articles to a subject matter expert as the editor, who could approve changes, or just incorporate the best changes in.
Not every article would need to have this, but as articles mature, they could move to this over time.
On the other hand, do gibberish pages like this need much more editing, or is Harry Potter's Wikipedia entry basically finished as far as anyone cares?
Since the problem is tantalizingly easy to frame as a standard data-mining or machine-learning problem, albeit with some quirks, there's quite a lot of work from a lot of research groups that seems to be looking at it. Some examples: one, two, three, four, five, six, seven.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
There is an art to Wikipedia abuse. If someone cites a Wikipedia article in some argument they're making, you can always just go to Wikipedia and edit the page so that they're wrong. But that's what a novice Wikipedia vandal does.
A pro knows to edit the article in a very subtle way, so that it looks like the person has poor reading comprehension. Let's say the person cites a Wikipedia article with a sentence like this, in order to support the argument that Colbert is a Democrat.
Although by his own account he was not particularly political before joining the cast of The Daily Show, Colbert is a self-described Democrat.[12][13]
This bears the mark of authority, because of the footnote subscripts that are already on it. (We can skip the step where we maliciously relocate them here.)
A novice might change it to this (correctly preserving the authoritative footnote superscripts):
Although by his own account he was not particularly political before joining the cast of The Daily Show, Colbert is a self-described Republican.[12][13]
It makes the person appear to be wrong- and the vandalism is obvious- like swapping Eurasia for Eastasia. There's no way he could have misread that.
But change it to this
Although by his own account he was not particularly political before joining the cast of The Daily Show, Colbert has even been described as a Democrat.[12][13]
and the person looks not only wrong, but plausibly wrong because it looks like he can't read. That's what makes successful Wikipedia vandalism an art.
Whoever posted this clearly isn't aware of the actual work being done in the field. For instance, I was running a ___[thing]___ in _[year]_, and it wasn't new at the time. They've gotten much more sophisticated since then. Why are they so intent on reinventing the wheel? Do they not even realize that the wheel exists already? Why not just improve on it instead?
* * *
This looks like a useful template for the standard "why reinvent the wheel" Slashdot post; I hope you don't mind if I reuse it.
I believe that vandalism on Wikipedia can be limited. But would it really be possible to detect all kinds of vandalism?
FTA:
"Yahoo! Research will award a cash prize of 500 Euros to the winner of the plagiarism detection task. "
500 Euro's doesn't sound much for detecting plagiarism on a site like Wikipedia...
Officially, vandalism is defined as edits made in bad faith. If you are trying to improve the article but are an idiot (which includes people that don't realise their own bias), that isn't vandalism, it's just idiocy. It is only if you are editing with the intention of making the article worse that you are vandalising.
... but the truth?
http://en.wikipedia.org/w/index.php?title=Nick_Xenophon&oldid=326486984#As_federal_senator
If I had mod points, I'd mod the parent up and the grandparent down. Seriously, almost everything in Wikipedia is transparent. Search the revision history and logs and look for the information you need. RTFM.
A lot of people on /. seem to derive very general opinions about admins from a personal disappointing encounter. They do not include diffs of their edits or their username. From my experience in most cases the guy who got reverted by an admin broke some kind of rule (and often enough they just got reverted by a regular non-admin, but they assume it was an admin). Instead of RTFM those people post as AC complaining generally about admins without providing any traceable cases of admin abuse. I know my opinion isn't very popular, but unless you give concrete examples your allegations are just FUD.
You're looking for a DWIM (Do What I Meant) interpreter with PDCH (Predictive Digital Concierge Heuristics). While the technology is available it's currently quite costly. Bugs, errata, and maintenance can deliver less than an optimal experience. Might I instead offer you this mail order bride? We have imported personal assistants in stock from less privileged nations - and if you have the means we can outsource minute-to-minute management of them to our Bangalore VPDT (Virtual Presence Discipline Team). Please consult your accountant and tax lawyer concerning withholding for personal staff, particularly if you intend to pursue public service.
/At your service!
Help stamp out iliturcy.
Case in point --- There is an article in Wikipedia about a certain country.
In that article, they blame their previous British colonial master for everything.
I tried to make some corrections to that article to make it more "neutral", and they changed it back within 10 minutes.
I tried again, and again they changed it back.
For the third time, I was warned by someone from Wikipedia (dunno if it's a volunteer or something) that I have no right to make any correction to that particular article anymore.
The "THEY" in question is the government of that country. They have a "cyber-patrol" group in charge of "online propaganda" and that Wikipedia article is one of their many lies, aka propaganda, they have put online.
Now, how do you define vandalism in this case?
Muchas Gracias, Señor Edward Snowden !