Slashdot Mirror


Wikipedia Creates AI System To Filter Out Bad Edits (thestack.com)

An anonymous reader writes: Wikipedia has developed a new artificial intelligence system aimed at improving the quality of its entries and detecting both mistakes and damaging edits made to its articles. The technology is named the Objective Revision Evaluation Service. The Wikimedia blog explains that the system is able to highlight incorrect edits, allowing editors to filter them out from the "torrent" of new amends and scrutinize their credibility. The entire service and process is open – with Wikipedia making revision scoring transparent and audit-able by publishing the source code, performance statistics and project documentation publicly under open licenses.

51 of 83 comments (clear)

  1. Is this some kind of joke article? by Anonymous Coward · · Score: 1

    The acronym for wikipedia etc is W.O.R.S.E?

    1. Re:Is this some kind of joke article? by alvinrod · · Score: 4, Funny

      Actually it would be W.O.R.E.S. considering its name is the Wikipedia Objective Revision Evaluation Service.

      But now that you mention it they missed a golden opportunity to call it the Holistic Objective Revision Evaluation Service or something along those lines.

    2. Re:Is this some kind of joke article? by Anonymous Coward · · Score: 1

      I think you mean O.R.E.S (or (W.O.R.E.S.)

    3. Re:Is this some kind of joke article? by xxxJonBoyxxx · · Score: 1

      >> I think you mean O.R.E.S

      Sounds like they've been playing Minecraft.

    4. Re:Is this some kind of joke article? by Dutch+Gun · · Score: 4, Insightful

      Why a joke? Consider it a spam filter for Wikipedia entries. The more popular entries have unfortunately had to get locked down due to rampant vandalism. Wikipedia, for all it's faults, is one of the wonders of the modern age. Between it and Google, it's like a global repository of human knowledge... or at least, a summary of human knowledge, with links to deeper knowledge.

      But while its power is derived from the masses of humans that create and edit that content, the masses are also its weakness. Humans are endlessly creative, and this tool will probably just teach trolls how to vandalize more creatively in order to fool the algorithm. Still, if it cuts down the noise for editors, it may end up being a good thing. It's hard to say until it's actually been deployed in the real world for a while.

      --
      Irony: Agile development has too much intertia to be abandoned now.
    5. Re:Is this some kind of joke article? by ItsJustAPseudonym · · Score: 2

      Almost there.

      Wikipedia
      Holistic
      Objective
      Revision
      Evaluation
      Service

    6. Re:Is this some kind of joke article? by BradMajors · · Score: 1

      a summary of human knowledge

      No. It mostly a collection of stuff that was found with Google search.

    7. Re:Is this some kind of joke article? by Shadow+of+Eternity · · Score: 1

      Wikipedia is a giant litmus strip. Whoever can pump out the most "reliable sources" and game the bureaucracy best gets to redefine truth through the woozle. It's the ultimate test of how good you are at PR or propaganda.

      --
      A bullet may have your name on it but splash damage is addressed "To whom it may concern."
    8. Re:Is this some kind of joke article? by reboot246 · · Score: 2

      They needed another word.

      Objective Revision Evaluation Service = ORES
      Objective Revision Evaluation Online Service = OREOS!

    9. Re:Is this some kind of joke article? by KGIII · · Score: 1

      What I find interesting is that Bill Gates was publicly pontificating about such back in the very early 1990s. Their partial implementation was known as Encarta and had many of the same goals but slightly less altruism. The Wikipedia (ah how deliciously ironic) article indicates that it was first released in 1993 but there's at least one video segment and one talk given where Gates discusses it a bit more as well as the objectives of the project.

      I'm not positive but I think the talk was given at ComDex and the video segment was a part of a slightly later (1994, I think?) video that was released and had to do with Microsoft Plus! being made available soon.

      I don't think that editing was on his radar at that time but the concept of allowing people to upload and share their created media (music, video, and images) was quite specifically mentioned. One could assume that editing was a logical and natural progression had Wikipedia not come along. I don't know how libre it would have been but I'd also guess that, by now, it would be free (as in beer) if nothing else.

      I've no idea where I'm going with this but I figured that the recollection of history might be of interest to someone. The times, they were interesting. Things were changing rapidly. People were, back then, talking about ownership of private information and how your personal information might not only be tracked and shared but it might be in as many as five or six data bases at some point and how scary that would be. Even Cringely was talking about it and how they'd be building profiles and using that to target ads and that they'd be able to discern a great deal about you by the shows you watched, the products you bought, and the data you gave away in order to access a service.

      And, if I must mention some of the "elders" or "historically significant" then I'll add that, on the other hand, RMS has been a lunatic since the beginning. One of his earliest acts was to protest the use of passwords in the MIT computer labs. He's hated DRM for a very long time. Oddly, the people who copy him and espouse his beliefs, parrot them - at least, seem disinclined to stop using passwords and, presumably, still chmod as needed. But, I digress... Stallman's foray was to send out emails and verbally convey the idea to the users that they should simply leave the password field blank and not configure it as a way to prevent others from imposing their rights on you. He may have had a point, even a valid point, but he's still a lunatic.

      Err... I should also point out that no, I'm typing this from a Linux-based (I dunno how much GNU is actually in Lubuntu) OS. I also would add that I'm pretty fond of Wikipedia and accept it for what it is. I think it's a wonderful resource that has a variety of limits but those limits are far outweighed by its value. Ah well, there went the digression into some history of encyclopedic content on the computer. Encarta was also not the first to do so. I seem to recall IMDb being something maintained by movie buffs back on Usenet? My memory may be a bit faulty there but that's one other example that I can think of. I'm sure that there are plenty more.

      --
      "So long and thanks for all the fish."
    10. Re:Is this some kind of joke article? by KGIII · · Score: 1

      Well, at least the shell script will be consistent. So, there's that. I'm not sure why you're modded troll. It appears pretty accurate to me. I still find it useful but that's because I'm willing to check the sources and use it as a jumping off point to do further research. I have noticed a trend where omission, I'll not go so far as to say it is deliberate, is used to ensure the tone of the article is both factual and biased. It does, indeed, contain the truth but it's a very limited set of truths and implications are there but the articles are, indeed, factual.

      I've not dug into edit histories, looked to see what has been deleted, nor do I bother to check who did what. I'm not usually concerned with this. I am a bit of a history buff and I've seen a variety of omissions that result in slanted views being listed as factual - specifically WWII era stuff but I'm more likely to notice it there. It hasn't been important enough to warrant any great effort on my part, I simply don't care enough and am not going to even begin to put the effort in to make changes.

      I would say that the slant is not always in one direction and sometimes rather petty. Like all things, it is best consumed in moderation and with consideration given to the source. Would I use it in a quick internet article? Yes, and I'd accept the findings from others there unless greater effort was needed. I'd just not put a lot of stock in it and I'd actually read the sources if it were a scholarly effort. Selective quoting can (and does) happen and very few things on this planet are without bias.

      --
      "So long and thanks for all the fish."
    11. Re:Is this some kind of joke article? by Aristos+Mazer · · Score: 1

      Nonsense. The editorial staff is larger than it has ever been, and there's a record number of writers contributing. Check the change logs. They're public. The sheer number of junk articles deleted is higher than ever but as a percentage it does not appear to be -- as the size of Wikipedia has grown, so has the junk people have tried to add into it.

      The shell script should help make sure that the necessary task of pruning the junk is applied objectively.

    12. Re:Is this some kind of joke article? by ale2011 · · Score: 1

      Yeah, at least that shreds light on the true meaning of holistic!

      Now, let me try and explain these comments from a non-chauvinist point of view. Etymologically, the term whore is related to desire, wish, aspiration —our noblest qualities. Nowadays it is used to indicate females who vilify those qualities by turning them into a source of profit. By extension, human males use the term to refer to just any woman, thereby unwittingly recovering the original, etymological meaning. That attitude is considered bad, for obvious reasons. However, we must admit that it emphasizes how we routinely ruin our best resources in an attempt to make more money. Like most occurrences of indecent language, it reminds us of what we really are, our success in building a logic that obscures our true essence notwithstanding.

      Wikimedia is not using its resources to get rich. It complies with the spirit (and the letter) of free software. So, why should we use AI to help us editing our knowledge? Possibly, our rational knowledge is artificial already, in a sense. We wouldn't need to take recourse to circumlocutions meant to exclude rationality from our emotive talking otherwise. Free sex was already attempted in the 60s. I hope free software is blessed by a better fate.

  2. Detects mistakes? by Anonymous Coward · · Score: 5, Funny

    Great! We can hook it to a random number generator and have it create all knowledge!

    1. Re:Detects mistakes? by dsmatthews9379 · · Score: 1

      Great idea, almost as good as pointing Google Goggles at a detuned TV.

  3. If everyone reading this right now gave $3... by TraumaFox · · Score: 2

    ...we'd reach the singularity in an hour. That's right, the price of a cup of coffee is all our future robot overlords need.

  4. Well it's a start by axl917 · · Score: 4, Insightful

    Now if the AI can be programmed to weed out the;

    *Man-Child admins who lack authority in real life and become a Wikipedia admin to (over)compensate
    *Agenda Warriors who go there to further their ideological battles
    *Bullies
    *Harassers
    *Stalkers
    *Jimbo Wales

    we might see some improvement.

    1. Re:Well it's a start by MrLogic17 · · Score: 4, Insightful

      The first 2 are the reasons I stopped contributing to Wikipedia. Today, article contents reflect the most stubborn editors with the most free time on their hands - not people who actually know what they're talking about.

    2. Re:Well it's a start by ShanghaiBill · · Score: 1

      Now if the AI can be programmed to weed out the ...

      That should be possible. If the AI can effectively discriminate between "good" edits and "bad" edits, then it should also be able to identify admins who revert a lot of "good" edits.

    3. Re:Well it's a start by Pharmboy · · Score: 2

      As an admin there who fights to prevent this, I have to agree. It isn't an easy battle when the warriors have more time than the skilled editors and admin combined. Some things we do right, controversial or complicated topics, we do poorly.

      --
      Tequila: It's not just for breakfast anymore!
  5. Nah doesn't work by goombah99 · · Score: 3, Funny

    I'm the inventor of the scroll lock key but everytime I ad this to my Wikipedia profile it gets deleted.

    --
    Some drink at the fountain of knowledge. Others just gargle.
    1. Re:Nah doesn't work by tepples · · Score: 1

      Probably because Wikipedia aims not to include crap. I'd be interested to see the third-party sources that name you as the inventor of Scroll Lock.

    2. Re:Nah doesn't work by orgelspieler · · Score: 2

      [citation needed]

    3. Re:Nah doesn't work by ShanghaiBill · · Score: 2

      I'd be interested to see the third-party sources that name you as the inventor of Scroll Lock.

      Here is the source.

  6. It will be an improvement by Opportunist · · Score: 1

    At the very least it cannot get any worse.

    My last attempt to contribute something sensible to Wikipedia was removing what I deemed a vandalism attempt in an article about Greek columns where a certain person was named along with his sexual preferences that are allegedly inspired by Greek society. Which was evidently relevant to the understanding of Dorian column styles, it seems, for my removal of said person's name along with the pertinent information about his sexual preferences was dutifully restored mere minutes later.

    --
    We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    1. Re:It will be an improvement by BrookHarty · · Score: 4, Interesting

      The list of wikipedia astroturfing and politics turfing is beyond annoying. One of my favorite wiki feebles is when they told Philip Roth he’s not “credible source” on book he wrote

      I wish I would of bookmarked the editors page, but there was this historian who specialized in mis understood beliefs in wikpedia, but he would have his updates removed because the standard "group think" belief was he was incorrect. He even posted the references required but was still reverted. He finally just edited his personal page, so it would be searchable, no idea if it is. His popular one was that no Canadian military was in vietnam, but Canada did have military medical there with hospitals and mash type units. He even linked to the CA GOV MIL page that had the medals given out during the vietnam war.

      Don't get me started on the whole left slanted views given. Even the gamergate page is dripping in political correctness and not very subtle slanted views. Read the comments section on there, you can see how Jimbo had to ban anti-gg editors for awhile due to the bias.

      I remember when MGTOW was coming about, some editors on wikipedia wouldn't allow a mens right page or mgtow to be referenced or mentioned. Finally they they came up with a plan, create a MGTOW page and redirect it to MTOW (Maximum Take Off Weight) to make sure MGTOW couldn't get a page.

      Also the page for "Cultural Marxism" was deleted, and then a redirect to the "Frankfort School". The irony is dripping. Now according to wikipedia is a "Conspiracy Theory"...

      Check out the differences on some differences in definitions for Cultural Marxism.

      Urban Dictionary
      The gradual process of destroying all traditions, languages, religions, individuality, government, family, law and order in order to re-assemble society in the future as a communist utopia. This utopia will have no notion of gender, traditions, morality, god or even family or the state. The Philosophy was proven not to Work already by Vladimir Lenin as he tried in vein to control and subjugate the people. He admitted before he died that capitalism was the only true system in which people understand how to live with each other.... Lenin knew that there were a few western Idiots who kept spreading the communist ideas long after Lenin gave up.... he called these people useful idiots as they had more emotion than brains and could be used to subvert the western states for a military takeover in the future as the citizens would already be perverted and sick and weak from poisonous ideas, decadent lusts and mindless entertainment.

      Metapedia
      Cultural Marxism is an ideology which emphasizes culture as a main cause of inequalities. Critics have seen cultural Marxism and its influence as an important cause of political correctness and as an important cause of a perceived decline of humanities, social sciences, culture, and civilization in the Western world.

      Rational Wiki
      The term "cultural Marxism" is most commonly encountered as a snarl word decrying everything right-wingers don't like, alluding to a conspiracy theory involving sinister left-wingers in the cultural and artistic spheres, including the media and academia, supposedly being engaged in a decades-long plot to undermine Western culture. With bonus anti-Semitism.

      Wikipedia
      DELETED.

    2. Re:It will be an improvement by bunratty · · Score: 1

      ...and then you just gave up. I totally see how it was that other guy's problem.

      I've had similar experiences, but I don't give up. If you let asshats walk all over you without putting up a fight, expect asshats to walk all over you. Wikipedia has all sorts of policies for how to deal with this sort of conflict. And I'm sure that in most conflicts both parties think they are in the right. So whomever loses comes here and complains about how bad Wikipedia is because their "good" edit was reverted, so now they refuse to try to edit articles any more.

      --
      What a fool believes, he sees, no wise man has the power to reason away.
    3. Re:It will be an improvement by bunratty · · Score: 2

      One of my favorite wiki feebles is when they told Philip Roth he’s not “credible source” on book he wrote

      Absolutely correct! People cannot be sources for Wikipedia. Previously published material is the only allowed source for Wikipedia. And you can't just write a web page yourself and use that for your source. Now if Philip Roth wrote an article about his life and published it in a reliable periodical, that article could be used as a source, just as if I wrote an article about Philip Roth and had it published that article could be used as a source, too.

      --
      What a fool believes, he sees, no wise man has the power to reason away.
    4. Re:It will be an improvement by Opportunist · · Score: 1

      Fight? Over what? An edit I don't care about? If they insist that their page about Greed columns should inform people about the butt fucking preferences of certain people, I don't deem myself to be in any position to tell them they cannot do so.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    5. Re:It will be an improvement by bunratty · · Score: 1

      Ah, you don't care about it so much that you're bitching about it on Slashdot. I see. Besides, why would you think you're telling someone they cannot make an edit? You're just stating your opinion and he's stating his. If you bowed out of the conflict, it's your own damn fault you lost. Whinging about it here and then turning around and claiming you don't care is just icing on the cake. I bet you're a real go-getter!

      --
      What a fool believes, he sees, no wise man has the power to reason away.
    6. Re:It will be an improvement by axl917 · · Score: 2

      That's not the point.

      The problem illustrated is that recognized experts in a field are routinely shouted down by pseudonymous man-children. This is exemplified, ironically enough in a Wikipedia project-space titled "Randy in Boise" - https://en.wikipedia.org/wiki/...

      If Randy and a bunch of Randy's friends find a blog that says there were skeleton warriors in the Peloponnesian War, they can crowd-surf their opinion in and keep the experts' out. The Wikipedia is an exercise in gang warfare, not "crowd-sourced writing" as Jimmy-Bob likes to espouse.

    7. Re:It will be an improvement by Actually,+I+do+RTFA · · Score: 1

      I see aqll the time that math articles have errors. Not "oh, this obscure theorum is wrong" but "the math they used in their example doesn't check"wrrong. You don't need a source for "your example problem doesn't do sums right". But they insist it does...

      --
      Your ad here. Ask me how!
    8. Re:It will be an improvement by gustygolf · · Score: 1

      Wikipedia
      DELETED.

      https://en.wikipedia.org/wiki/...

      HTH, HAND.

      --
      "Slow Down Cowboy! It's been 58 minutes since you last successfully posted a comment" -- slashdot, driving users away.
    9. Re:It will be an improvement by Opportunist · · Score: 1

      I just thought it's funny, that's all. Seriously, if I have to fight for and defend an anti-troll edit, I don't even want to be bothered to do a content edit that could actually be questioned for its relevance or sensibility.

      We are literally talking about the undoing of "XXX is a fag". There is no universe in which this bit of information is relevant or pertinent in an article about a style of Greek columns. Or any article that isn't directly dealing with Mr. XXX.

      Sorry if I don't feel that fighting through the defense of this isn't worth my time.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    10. Re:It will be an improvement by gustygolf · · Score: 1

      Anyway, I now realise that you already mentioned that it was a redirect.

      The redirect was created with the note:

      redirect, per result of many, many, many, many discussions

      And that brings to a complaint of mine of Wikipedia: If you aren't "in the know", there's no way you can find these "many" discussions. They're hidden somewhere in the Wikipedia: or Talk: namespaces, but I wouldn't even know where to start searching for them.

      --
      "Slow Down Cowboy! It's been 58 minutes since you last successfully posted a comment" -- slashdot, driving users away.
  7. Re:What a time to be alive! by Opportunist · · Score: 1

    I'm honestly waiting for the first helicopter editor being reverted by the AI due to him being identified as a troll and vandal. It would probably serve as the proof that the AI is working correctly...

    --
    We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  8. Re:Russian trolls soon out of jobs? by Opportunist · · Score: 1

    What do the specs have to do with whether or not the plane was used to shoot down another one?

    --
    We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  9. Citation timing by tepples · · Score: 1

    To break such a citation feedback loop, prefer a pre-2001 citation for claims about applicable topics. By the time Wikipedia came around, the Scroll Lock key had been present for well over a decade, so if there were a citation, it's more likely than not that there would also be a pre-2001 citation.

    1. Re:Citation timing by Shadow+of+Eternity · · Score: 1

      At which point you get your revisions undone and buried in warnings for edit warring, verifiability and not truth, placing undue importance on a marginal opinion, and the moment you try to get help from the admins to deal with all the flaming and stalking it's boomeranged back on you.

      --
      A bullet may have your name on it but splash damage is addressed "To whom it may concern."
    2. Re:Citation timing by tepples · · Score: 1

      edit warring

      It's only edit warring if you try to reapply your edit. If you instead discuss the revert in a civil manner, there is no war. If discussing the revert has not worked for you, I'd like to see diffs so I can help you figure out what went wrong and why.

      verifiability and not truth

      If you have pre-2001 citations, you have verifiability.

      placing undue importance on a marginal opinion

      If you have pre-2001 citations, plural, the opinion is not marginal.

    3. Re:Citation timing by Shadow+of+Eternity · · Score: 1

      I've been editing wikipedia pretty much as long as it's been wikipedia. If you are on the wrong side of the groupthink or disagree with one of the wikipedia untouchables you WILL be screwed. Every single source you try to bring will be accused of being an unreliable source supporting a marginal opinion, you will be told that wikipedia is about verifiability and not truth, and you'll be viciously flamed the entire time and threatened with all kinds of disciplinary procedures. The moment you attempt to complain about your treatment you'll get boomeranged.

      --
      A bullet may have your name on it but splash damage is addressed "To whom it may concern."
    4. Re:Citation timing by tepples · · Score: 1

      If discussing the revert has not worked for you, I'd like to see diffs so I can help you figure out what went wrong and why.

      If you are on the wrong side of the groupthink or disagree with one of the wikipedia untouchables you WILL be screwed.

      I hate to have to say it, but [citation needed]. I'd understand better if I could verify what you're talking about through diffs.

    5. Re:Citation timing by Shadow+of+Eternity · · Score: 1

      Seriously tepples? Enough with the denialism. You're either being mendacious or are so out of touch with the current state of wikipedia you should stop trying to advocate for it. It's ruled by powerful cliques like wikiproject feminism and other unblockables.

      --
      A bullet may have your name on it but splash damage is addressed "To whom it may concern."
  10. Secret Code snipit by laurencetux · · Score: 3, Funny

    ... IF POSTER = DONOR MORE THAN US$15E5 THEN GOTO POST EDIT ...
    rest of filter code goes here

  11. Re:If only it filtered out bad admins by Coren22 · · Score: 1

    That was my thought as well, when will Wikipedia come up with something that identifies bad editors and blocks their reverts.

    --
    APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
  12. Re:Russian trolls soon out of jobs? by Coren22 · · Score: 1

    The plane is a air to ground attack aircraft, kind of like the A-10, it can't shoot down a plane flying at 30kft, so if you try and make it out to be an F-35 instead of an A-10, maybe people will believe that it wasn't a AAA battery that Russia lent to the Rebels and is instead a fighter jet that Ukraine is using.

    --
    APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
  13. Re:Russian trolls soon out of jobs? by Opportunist · · Score: 2

    The problem is that the only people who care about whether or not they were shot down by this or that plane also don't care about whether they can or cannot do it.

    --
    We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  14. Re:Russian trolls soon out of jobs? by Opportunist · · Score: 1

    Well, it's technically possible for an A10 to down a civilian plane. It's not like you have to engage in high-g dogfights to win that battle...

    --
    We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  15. Re:If only it filtered out bad admins by Coren22 · · Score: 1

    Unfortunately, due to going camping this weekend, my next two days suck, but thanks for the offer. Maybe if you end up in this area again.

    --
    APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
  16. Re:Russian trolls soon out of jobs? by Coren22 · · Score: 1

    My assumption was that the A-10 couldn't reach those altitudes, but it looks like I was wrong, as service ceiling on the A-10 is 45k feet, on the SU-24 it is 35k feet. It is possible for these aircraft to shoot down a civilian aircraft at cruise altitude, it is just highly unlikely. The SU-24 apparently carries 2 R-60 A-A missiles, so it is theoretically possible, however I don't believe that the SU-24 the Russians are pointing to ever flew above 10k feet. It is a scapegoat to try and draw attention from the AA battery that was smuggled back into Russia with two missiles fired.

    --
    APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
  17. BRD: Do not rush the discuss phase by tepples · · Score: 1

    Enough with the denialism [User:Beeblebrox/The unblockables].

    The only "denialism" here is learning how to "deny" the unblockables an opportunity to accuse you of edit warring. It starts by not edit warring.

    It's ruled by powerful cliques like wikiproject feminism

    You are correct that I haven't been active in controversial topics, especially those related to civil rights of a particular group. But the examples in Beeblebrox's essay appear to skip at least one step of the BOLD, revert, discuss cycle.

    Seeing no reason for this, the newbie makes the same or very similar changes again.

    Wikipedia doesn't make it obvious, but a reverted user should refrain from immediately making the same changes again. Instead, the user should take it to Talk:.

    The user posts their reasons to the talk page and adds the content back in again. The admin reverts them and says there is no consensus for adding the content.

    I've tended to have more success by not rushing through the discuss phase of BRD. Waiting 7 days between the post on Talk: and redoing the changes lets me either elicit more detailed feedback on the problems with a change or, barring that, use silence on the talk page as evidence of change in consensus.

    If you don't want to be accused of edit warring, don't edit war.