Slashdot Mirror


Researchers Mine Old News To Predict Future Events

hypnosec writes "Microsoft Research has teamed up with the Technion-Israel Institute of Technology to develop software that can predict events like outbreaks of disease or violence by mining data from old news and the web. The project, if successful, will result into a tool that would provide information that is more than just educated guesses or intuition. The team consisting of Eric Horvitz from Microsoft Research and Kira Radinsky from Technion-Israel Institute tested the program with articles from New York Times spanning over 20 years from 1986-2007."

19 of 99 comments (clear)

  1. Past performance... by Overzeetop · · Score: 4, Funny

    ... is Not Necessarily Indicative of Future Results.

    Do these guys not even read their own prospectus?

    --
    Is it just my observation, or are there way too many stupid people in the world?
    1. Re:Past performance... by Trepidity · · Score: 3, Interesting

      True, though it can to the extent that there are recurring patterns and you find the right ones.

      On the other hand, there's also a circularity problem. Say you find, from analyzing 20 years of correlations, that certain events tend to happen some period after certain news reports. This might impact whether that relationship continues to hold in the future. That's already quite common for financial events: if you can reliably predict that when News Report Type X happens (for a possibly complex "X"), then Stock Move Y will happen, you can profit from it, but only until it becomes known by enough people, after which the arbitrage opportunity will close.

    2. Re:Past performance... by mug+funky · · Score: 3, Informative

      jokes typically place consistency with reality a little further down in the list of priorities than whatever you were expecting as a first post.

  2. For example, did you know... by Anonymous Coward · · Score: 5, Insightful

    There will be a Presidential election in 2016 and 2020

    An NFL team will win the Super Bowl.

    Many new TV shows will be cancelled each fall.

    Old people will die.

  3. Asimov was here by Minion+of+Eris · · Score: 5, Insightful

    The shade of Asimov raises his head..... Does this seem a little like Psychohistory to anyone else? Where's the Mule?

    --
    Please don't dominate the rap, Jack, if you got nothin' new to say.
    1. Re:Asimov was here by maxwell+demon · · Score: 2

      The shade of Asimov raises his head..... Does this seem a little like Psychohistory to anyone else? Where's the Mule?

      More importantly, which is the empire that will break down?

      --
      The Tao of math: The numbers you can count are not the real numbers.
    2. Re:Asimov was here by gmuslera · · Score: 3, Interesting

      More important than the mythical Mule (unless telepathy really exists and to that extent), the important message in Asimov's Psychohistory is that predicted people shouldnt be aware of the predictions on them, and that includes the government. Is a good way to invalidate predictions, acting with the knowledge of the prediction instead of acting "naturally", whatever be it.

  4. Re:Consider.. by maxwell+demon · · Score: 5, Insightful

    Actually, the weather report for the next day is pretty accurate most of the time.

    --
    The Tao of math: The numbers you can count are not the real numbers.
  5. Science! by Intrepid+imaginaut · · Score: 2

    Discovers experience.

  6. Here, Let Me Save Them Some Time by Nova+Express · · Score: 4, Informative

    Unrest in Middle East
    African Regime Unstable, May Collapse
    Congressman Indicted
    Experts Say Hurricane "Extremely Dangerous"
    Audit Finds Serious Misuse of Funds
    Green Energy Firm Declares Bankruptcy
    Apple's Latest Product Selling Like Hotcakes
    Patch Released for Serious Windows Vulnerability
    Unemployment Rises, Unexpectedly

    There's your headlines for the year.

    --
    Lawrence Person (lawrencepersonh@gmailh.com (remove all "h"s to mail)

    http://www.lawrenceperson.com/

    1. Re:Here, Let Me Save Them Some Time by Robert+Zenz · · Score: 2

      You forgot: "It will snow in winter." At least here in Europe it is a serious, unpredictable problem which surprises the people every year.

  7. Re:Queue the ... by phrostie · · Score: 2

    would you consider a Douglas Adams?

    “Anything that happens, happens.
    Anything that, in happening, causes something else to happen, causes something else to happen.
    Anything that, in happening, causes itself to happen again, happens again.
    It doesn’t necessarily do it in chronological order, though.”

  8. Re:This has all happened before... by oodaloop · · Score: 2

    Almost makes you think that was the joke. But only almost, apparently.

    --
    Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
  9. Disease outbreaks? by Hartree · · Score: 2

    You mean by looking at twenty years of newspaper articles they were able to predict that there will be a large increase in the number of cases of influenza during November 2013 to March 2014 compared to the preceding 5 months?

    Or, when disaster causes infrastructure to break down and crowds refugees into unsanitary temporary housing there's a high likelyhood of more cholera breaking out than at other times?

    Gee. Color me impressed.

    How is this greatly different than many different types of analysts have been doing for decades via headline counts in world newspapers and the like? (See John Naisbitt of Megatrends fame, for example. And he certainly wasn't the first.)

    The math used to find the probabilities may be a bit better, and it may be more automated, but it's not particularly new.

  10. Will it predict the future, or media cycles? by dorpus · · Score: 2

    I have two family members in the journalism business. The media business has a cycle of covering particular topics and moving on when the public gets bored of it. Plenty of news does not get coverage at all, at least by English language media, if the country is too remote or the topic is too cliche.

  11. Only as good as the source material by RogueWarrior65 · · Score: 2

    The NYT? For only the last 20 years? That's worse that basing global warming predictions on just the last 20 years.

  12. Re:That's why we study history (or used to) by Anonymous Coward · · Score: 2

    But tell me please - this baby boomer asks why do so few gen X and Ys show any interest in history?

    Because the Founding Fathers were Christian through and through; the Civil War was about slavery and nothing else; and we're the chosen people, spreaders of truth and justice and democracy, having never committed wholesale genocide or locked up our own citizens in concentration camps.

    Also, they hate us for our freedumbs.

    Seriously, though, most history teachers bore kids to death by doing little more than demanding they memorize places, names and dates. While some of them are shitheads who get off on this sort of trivial idiocy, a great many probably suffer from having their hands tied by our lackluster schools.

    God forbid a history teacher informs his class about how many bitches Ben Franklin got.

    Dude got mad bitches, by the way.

    tl;dr: When you hear factoids instead of interesting stories about historical figures, said historic figures seem fake; unreal; and people end up with a sense that we're completely different than the humans who lived even a hundred years ago, despite the fact that we're no different than the humans alive at the time of the Roman Empire and earlier.

  13. Re:The end of Windows by Alex+Belits · · Score: 2

    I believe, the result was "Five months before Palestinian State will be established with stable borders and elected government".

    --
    Contrary to the popular belief, there indeed is no God.
  14. Re:Queue the ... by maxwell+demon · · Score: 2

    (For AC's)

    And for future readers finding your post long after you've changed your signature to something completely different.

    Nobody should ever refer to sigs (not their own, and certainly not other people's) without quoting them.

    --
    The Tao of math: The numbers you can count are not the real numbers.