Slashdot Mirror


Wikipedia Edits Forecast Vice Presidential Picks

JimLane writes "The Washington Post reports on the findings of Cyveillance, a company that 'normally trawls the Internet for data on behalf of clients seeking open source information in advance of a corporate acquisition, an important executive hire, or brand awareness.' Cyveillance decided 'on a lark' to test its methods by monitoring the Wikipedia biographies of Vice-Presidential prospects. The conclusion? If you'd been watching Wikipedia you might have gotten an advance tipoff of Friday's announcement that McCain was selecting Sarah Palin. 'At approximately 5 p.m. ET (Thursday), the company's analysts noticed a spike in the editing traffic to Palin's Wiki page, and that some of the same Wiki users appeared to be making changes to McCain's page.'" The article goes on to say that watching Wikipedia pages for the Democratic VP hopefuls would have tipped Obama's choice of Biden, as well. NPR also has coverage (audio).

42 of 152 comments (clear)

  1. What's This? by iamwhoiamtoday · · Score: 5, Interesting

    Politicians (or their group) editing wiki pages in order to appear better to the public? (the same people who have the power to put them in office) Gasp. Shocked I am. I honestly am starting to expect this kind of thing. PS: I do think that it's rather interesting, looking for spikes in Wiki traffic to predict assorted events, perhaps we should start monitoring the "US invades the entire middle east" page

    1. Re:What's This? by Anonymous Coward · · Score: 4, Interesting

      It is called traffic analysis. An old trick of what used to be called trade craft and probably is by the spooks

    2. Re:What's This? by Z00L00K · · Score: 5, Interesting

      So if an event is expected it may pay off to monitor the Wikipedia traffic to the related pages and by that forgo the official announcement.

      This poses some interesting prospects. Like if it was possible for party A to beforehand predict that a certain alternative was going to be selected by party B and therefore making that selection problematic.

      Only way around this is of course to make sure that the inner circle doesn't use the web for a while before official announcements are done.

      And this does of not only apply to politics but also to a lot of other events. Like potential inside affairs when it comes to buying/selling on the stock market. Pattern analysis evolves, and it may not even be necessary to actually listen in to a certain message, just measure the amount of traffic to a certain node to make a statistically based deduction. So even if you encrypt your information it may be traced and therefore provide valuable information.

      At least we do live in interesting times!

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    3. Re:What's This? by tubapro12 · · Score: 2, Insightful

      Wikipedia's edits forecast the future? Don't they say the same thing about Nostradamus' Les Propheties ?

      What's that? It's easy to see trends from nothing leading to something after the fact..?

    4. Re:What's This? by iamwhoiamtoday · · Score: 5, Insightful

      I do want to point out that because this article is being read by thousands and thousands of people, the assorted political groups are likely to not make the same mistake again. They will most likely compensate for this in the future.

    5. Re:What's This? by OpenSourced · · Score: 4, Insightful

      Only way around this is of course to make sure that the inner circle doesn't use the web for a while before official announcements are done.

      The problem is of course that they want the biographies "updated" for all the press and other interested parties that are going to hit Google in the first hour after the announcement.

      So much more likely will be that before such announcements, they will update like ten or twenty biographies, to mask which is the real one.

      That of course if they care enough.

      --
      Rome taught me patience and assiduous application to detail. Virtues which temper the boldness of great, general views.
    6. Re:What's This? by smittyoneeach · · Score: 4, Funny

      Indeed. They'll just have the staff whip up edits to several other distractor pages.
      Think of the cable news effects.
      Olberman: This just in: Oh My God! Traffic analysis on Wikipedia seems to indicate that Michael Moore might pick me to be his Vice President! I'm going to need a private moment, folks. Excuse me.

      --
      Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
    7. Re:What's This? by lazy_playboy · · Score: 2, Informative

      So much more likely will be that before such announcements, they will update like ten or twenty biographies, to mask which is the real one.

      Perhaps, although personally I would prepare any edits in advance and make them at exactly the same time as any announcement (/leak or whatever)

    8. Re:What's This? by Sj0 · · Score: 2, Insightful

      This story is completely meaningless.

      Anyone can stand up after the fact and say "Hey! I could've predicted this!"

      --
      It's been a long time.
    9. Re:What's This? by Fael · · Score: 2, Funny

      I knew someone would make that point sooner or later.

    10. Re:What's This? by Sj0 · · Score: 2, Funny

      I had a feeling someone would say that.

      --
      It's been a long time.
    11. Re:What's This? by OeLeWaPpErKe · · Score: 4, Funny

      To commit suicide ?

    12. Re:What's This? by OeLeWaPpErKe · · Score: 3, Informative

      Just one more example of wikipedia's "neutrality" NPOV policy being used to promote exactly 1 point of view, silencing all others.

      As has been the point of half the comments on this story ... I don't think anyone's surprised at all.

    13. Re:What's This? by chunk08 · · Score: 3, Funny

      We can only hope...

      --
      Do away with our corrupt tax code. Support the Fair Tax
    14. Re:What's This? by Jah-Wren+Ryel · · Score: 2, Funny

      It is called traffic analysis. An old trick of what used to be called trade craft and probably is by the spooks

      They could have figured out the same thing if they had paid attention to the increase in pizza-deliveries to the alaska governor's mansion for the two days beforehand too.

      --
      When information is power, privacy is freedom.
    15. Re:What's This? by Z34107 · · Score: 2, Interesting

      It is called traffic analysis. An old trick of what used to be called trade craft and probably is by the spooks

      Except that they used to literally analyze traffic - if you see a lot of cars in a parking lot overnight, it means people are working late hours and that, presumably, something is happening. If you see triple the usual amount of cars parked outside the Department of Defense, it may be something to phone home about.

      --
      DATABASE WOW WOW
  2. Leaks to Wikipedia by Apple+Acolyte · · Score: 5, Interesting

    It's pretty cool that Wikipedia has become a de-facto official source of leaks for such information. Fox News was reporting that Palin had moved to the top of the list but had no confirmation of her selection about an hour before officials confirmed it, and at that time they reported that Wikipedia listed her as the pick. Someone within the campaign evidently leaked it to Wikipedia before leaking it to offline media.

    --
    Part of the hardcore faithful who believed in Apple long before it was cool again to do so
    1. Re:Leaks to Wikipedia by WhatAmIDoingHere · · Score: 3, Insightful

      But the problem with that is some random jackass could see "Oh, so-and-so is PROBABLY going to be picked, so I'll edit it to say they were picked, since it's going to happen anyway."

      And that edit could get picked up by tons of people and spread around, even if it's not accurate.

      --
      Not a Twitter sockpuppet... but I wish I was.
    2. Re:Leaks to Wikipedia by djcapelis · · Score: 4, Funny

      And that edit could get picked up by tons of people and spread around, even if it's not accurate.[citation needed]

      --
      I touch computers in naughty places
    3. Re:Leaks to Wikipedia by fyoder · · Score: 2, Insightful

      But the problem with that is some random jackass could see "Oh, so-and-so is PROBABLY going to be picked, so I'll edit it to say they were picked, since it's going to happen anyway."

      Aye. Had wikipedia existed back in 1948 someone might have written "Dewey and Warren won a sweeping victory in the presidential election yesterday. The early returns showed the Republican ticket leading Truman and Barkley pretty consistently in the western and southern states."

      --
      Loose lips lose spit.
    4. Re:Leaks to Wikipedia by ericspinder · · Score: 3, Informative

      I was one of the people who viewed (didn't edit) her page that morning, I did so, because I had heard that there was a private jet that had just landed in Dayton, OH, apparently under a great deal of secrecy, which had a fight plan from Alaska. That fact was replicated at the bottom of her wikipedia page. Otherwise the page looked like a fair, short, biography of the Governor. It even included information about her Troopergate scandal, however, it was just a short blurb. I didn't check the history page, one should always check the history page for a fast moving story.

      --
      The grass is only greener, if you don't take care of your own lawn.
  3. Re:Pre hoc, ergo propter hoc by ptbarnett · · Score: 4, Informative

    So basically, TFS says that wikipedia edits are made to a relevant article prior to an event, and therefore, these wikipedia articles were caused by the event.

    The tip-off seems to be that the same people were editing both the Presidental and (eventual) Vice-Presidential candidate pages. The same pattern was observed with Obama/Biden.

  4. Subject intentionally left blank by jadin · · Score: 4, Insightful

    Hindsight is 20/20. Now try using this to _predict_ something correctly.

    1. Re:Subject intentionally left blank by RealGrouchy · · Score: 5, Funny

      I predict that people will interpret the findings of this article as meaning more than they do.

      - RG>

      --
      Hey pal, this isn't a pleasantforest, so don't waste my time with pleasantries!
  5. why I don't believe in conspiracy by fermion · · Score: 5, Interesting
    When working at various companies, I always monitored the stock price. Invariably, the few days prior to major announcement the stock volumes would go crazy.

    Invariably someone will slip up and do something to give the game away and such traffic analysis will give the game away. All that is required is that someone look.

    This is especially true for government conspiracy. For the most part, too many people have to be involved, and too many people are looking.

    --
    "She's a scientist and a lesbian. She's not going to let it slide." Orphan Black
  6. Re:Pre hoc, ergo propter hoc by MBCook · · Score: 2, Insightful

    So... people interested and informed in politics?

    --
    Comment forecast: Bits of genius surrounded by a sea of mediocrity.
  7. It just goes to show... by Jane+Q.+Public · · Score: 4, Funny

    campaign organizations, as a whole, are still idiots.

  8. Too late by Darkness404 · · Score: 5, Funny

    Too late, the elections are already decided http://www.theonion.com/content/video/diebold_accidentally_leaks

    --
    Taxation is legalized theft, no more, no less.
  9. Re:So sick of politics by pcolaman · · Score: 2, Funny

    So what are they missing now? What's the opportunity cost of all this insufferable coverage of minor insects like Joe Biden and this Alaskan twit? What's the big story of the decade that we're not hearing about?

    Your mom revealing that she really didn't mean to bring such an angry child into the world.

  10. Re:Another indicator by smittyoneeach · · Score: 3, Informative

    Actually, it was the Gulf War at the Pentagon with the 'za:
    http://tafkac.org/politics/pentagon_pizza.html

    --
    Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
  11. Re:It's interesting, but not predictive. by pcolaman · · Score: 3, Insightful

    Even as a registered Republican, I think the world (mostly) of Lieberman (the only thing I dislike about him is his stance on censoring games, but then again most senators and representatives are for this) but think that his choice would've sealed the deal for Obama. Many of McCain's own constituents don't want to see a Pro-Choice ticket, and with Lieberman on the ticket they would be more likely to just stay at home on Nov. 4. It was a very smart strategic play by McCain to pick Palin for several reasons. She's not establishment, which is a stigma that I'm surprised the Obama camp hasn't tried to label McCain with more. She's a mother of 5, including a special needs child, so if Biden hammers her too hard in the VP debates it could appear to some that he's picking on a woman and therefore create an image of someone who's cold and hard. This is definitely not the image I'd want to paint if I was a Democratic candidate, since they are supposed to be the party of the common man (bullcrap IMO, I actually think the party system should be abolished, but that's just my view). She also gives McCain someone who is strong on reform issues and is a whistle blower, something that you can hardly say about Romney or Pawlenty. Personally I think it was a good choice, as all anyone was talking about yesterday was her, not Obama's speech. Stole some of his thunder. Whether it works for McCain in the end has yet to be seen, but it will be certainly interesting to watch the Biden Palin debate, whereas I think I would have just watched something else rather than Biden v. Romney or Biden v. Pawlenty. They both would've been boring choices indeed. Whatever happens, it's going to be a fairly close election, although not as close as 2000.

  12. Cyveillance are slimy by Bert64 · · Score: 3, Interesting

    I get lots of hits from cyveillance addresses to my web servers, and the hits from the cyveilance robot are masquerading as IE users, and they don't even bother to try and retrieve robots.txt...

    If you contact them about it they will offer to remove your address range from the spider, but this is also a lie, after contacting them and supplying address ranges for them to stop spidering they simply started spidering from a different source address, this time the whois record for the ipblock shows nothing unless you directly query cogent's whois server which again reveals the ranges are registered to cyveillance. This looks like a very poor attempt to hide their actions. Their spider also has a very recognizable pattern, so it would be easy to pick up anyway.

    When i attempted to contact them again, they simply ignored all of my mails.
    Incidentally, after being explicitly told their company has no permission to access my web servers, their continued attempts amount to unauthorized access.

    --
    http://spamdecoy.net - free throwaway anonymous email - avoid spam!
    1. Re:Cyveillance are slimy by Jah-Wren+Ryel · · Score: 2, Interesting

      Incidentally, after being explicitly told their company has no permission to access my web servers, their continued attempts amount to unauthorized access.

      Bullshit. If the web were to work that way, it would kill it.
      You don't want them spidering your public website, then don't make it public.

      If I were you, I would fuck with them. Pollute their data. You've obviously been able to figure out which accesses are there's - use that knowledge to feed them disinformation. If you are lucky, you might even able to manipulate their clients in a way that can end indirectly making you money.

      --
      When information is power, privacy is freedom.
  13. Reverse Troll? by spineboy · · Score: 4, Insightful

    This may be an example of a reverse troll. By taking an extreme opposite position, it makes your position look more reasonable.

    Republicans did this about 10 years ago, by pretending to be really annoying Democrats, calling people at inopportune hours, etc.

    --
    ..........FULL STOP.
    1. Re:Reverse Troll? by Alsee · · Score: 5, Interesting

      >Republicans did this about 10 years ago, by pretending to be really annoying Democrats, calling people at inopportune hours, etc.

      [CITATION NEEDED]

      Searching republican "false flag" robocalls brings up hundreds of good hits on it.
      Here's the first hit describing a series of MORE THAN 20 harrassing calls, pretending to be from the Democratic candidate. The Republicans act like jackasses making harrassing robocalls, trying to trick people into thinking the Democrat is the evil jackass, so that people will get annoyed and vote Republican.

      Republicans have done it countless times across the country. Here's the Slashot story on it. It cites it happening in 53 Congressional districts in 2006. So these false flag tactics are a common Republican ploy. The only problem with the original post is that it said "Republicans did this about 10 years ago". Republicans still do it. I hardly expect them to stop just for the 2008 election.

      If you, or anyone you know, gets annoying robocalls "from Democrats", they are likely from Republicans. They also like to run bogus phone "polls". They will ask wildly biased questions like "Candidate X voted against a law to protect children from pedophiles, does this make you more or less likely to vote for candidate X?" Where of course candidate "X" is the democratic candidate. By inserting "facts" about their opponent into "questions", they make it sound like innocent neutral information from an innocent neutral source, to hide the fact that they are actually wildly biased and distorted accusations being flung by a Republican smear campaign.

      -

      --
      - - You can't take something off the Internet! That's like trying to take pee out of a swimming pool.
  14. prediction markets; race and polls by bcrowell · · Score: 4, Interesting

    They say prediction is difficult, especially about the future. Yahoo has a "political dashboard" (flash app) that tries various things to predict the outcome of the presidential race. One technique they use is prediction markets, which are sort of similar to this thing about the wikipedia edits: instead of asking people their opinions, you watch their actions. In the yahoo dashboard app, you can click to switch between a map based on opinion polls and one based on prediction markets. One interesting thing is that the polls show Ohio leaning to McCain, but the prediction markets show it going to Obama. One thing that's really tough about predicting this election is that historically, racist white people have often lied to pollsters about their race-related opinions. Even though Obama is ahead in the polls, I'm kind of expecting that McCain will win, simply because the polls are likely to have this systematic error in them. OTOH, some people say that this racism-hiding effect in polls is no longer as strong as it used to be. The February Scientific American had an article that treated prediction markets with skepticism. Some of the evidence that people have been quoting in favor of prediction markets is apparently bogus, and nobody has the faintest clue how they really work.

    1. Re:prediction markets; race and polls by 4D6963 · · Score: 2, Interesting

      The February Scientific American had an article that treated prediction markets with skepticism. Some of the evidence that people have been quoting in favor of prediction markets is apparently bogus, and nobody has the faintest clue how they really work.

      Well the basic idea behind the Iowa Electronic Markets is that people, anyone, can bet money (a limited amount) on who they think will win an election. Basically, polls ask people who they want to vote for, but arguably you'd have a better idea of the outcome of an election if you ask people not who they want to vote for but who they think will win. It's called the wisdom of crowds. Show a certain amount of people a jar full of pickles and they'll tell you about how many pickles are in, the more people you ask the more precise the results get (if I'm not mistaken under ideal conditions with a lack of a bias in their judgment 100 times more people should get it 10 times more precisely, that's like coherent averaging).

      That's the idea behind the IEM. With a twist, instead of just asking people who they think is gonna win, they make them bet on it, as becoming more interested in it makes them be more serious about it. And in case you're wondering, Obama is so winning!

      --
      You just got troll'd!
  15. Brilliant Pick Indeed by ricegf · · Score: 2, Funny

    Looks like McCain just wrapped up the election this year. I mean, he has all of Alaska's electors in the bag!

  16. Re:It's interesting, but not predictive. by flyingsquid · · Score: 3, Insightful
    Ah, but it's not about the base. It's about the swing voters. In this case, stealing dissatisfied Clinton voters.

    If that is the strategy, I don't think that it is going to work particularly well. Sure, Sarah Palin is a woman, but that's where the resemblance to Hillary Clinton starts and ends. She's an evangelical Christian who thinks that creationism should be taught alongside evolution in the classroom. She says she's not convinced that global warming is the result of human activity. She opposes abortion even in the case of incest or rape. When the environment and industry are at odds, she's squarely on the side of industry. She does have good qualities, but she actually pushes the ticket to the right in terms of values and issues. As a centrist Democrat, the chances of me voting for McCain have just gone from slim to none.

    Of course, that may be intentional: McCain may be trying to shore up his support on the right. If so, then that's a bad sign. The Democrats are enthusiastic and Obama has built a powerful political machine; that McCain is still trying to figure out how to generate enthusiasm this late in the game is not a good sign.

  17. Re:It's interesting, but not predictive. by ricegf · · Score: 3, Interesting

    that McCain is still trying to figure out how to generate enthusiasm this late in the game is not a good sign.

    Perhaps, although his campaign raised $4 million over the Internet in the 24 hours after the announcement. Their previous single-day fund-raising record was under a million. So at least he seems to have figured it out. :-)

  18. Re:Pre hoc, ergo propter hoc by Anonymous Coward · · Score: 2, Informative

    This whole thing is nonsensical. I did some research on this myself earlier in the day wading through hundreds of diffs.

    On the democrat side:
    We had a very good idea days before the official announcement it was Biden. Obamas people said their pick would be no surprise and it was common knowledge most of the other runner ups were not chosen essentially leading to Biden.

    On the republican side:
    There was a small edit war starting on the 28th for Palin someone kept writing it was her and other people changed it back to a less assertive statement. Other edits to Palins page said it was Tom Ridge and there were similiar edits to Tom Ridges page proclaiming he was the VP pick.

    There were no similiar edits asserting VP nomination (although there were "rumor" sections) on the Liberman or Romney pages.

    People noticing frequency of edits then using that as an indicator of prior knowledge is not a very convincing argument as increased interest would naturally follow increased edits as the expected announcement neared. Comments regarding Palin as a possible VP pick have been included on her page for many months.

    The only *intersting* thing I took away from this is how in the last two days the conterversy sections of Palins article which survived several months suddenly disappeared in a puff of smoke over the last two days?

  19. Re:Palin still a ReThuglican Jew Puppet c*nt by Teancum · · Score: 2, Informative

    Anyway, the article doesn't really explain the mechanics of how this analysis works. Do they just run a program to fetch the page every n seconds, use a reg exp to find the area where the number of edits are, get the counter and repeat for some number of hours?

    I guess that this is possible but it seems a bit crude. Anyone know a more sophisticated method? err ... does anyone know a more sophisticated legal method?

    The method of analysis is quite a bit more mundane than you seem to be implying here. Every Wikipedia page has a "history" log that shows every contributor, when the edit happen, and even what words were changed on each edit. All they did is take this page history and perform a modest analysis between each one of the VP candidates... and that was done more as a forensics review than anything when it was happening. The page history is public data, and you simply have to go to the Wikipedia article and click on the "history" tab to see the information.

    As far as monitoring changes in real-time, you can do that via RSS-feeds which you can get for each page individually or for Wikipedia as a whole (although the whole Wikipedia RSS-feed is a firehose of data). Basically, you can get notified when each page gets edited or modified. Usually this is used to catch trolls, but it could be used for this sort of analysis a well.