Google Launches a Data Prediction API

databuff writes "Google has released a data prediction API. The service helps users leverage historical data to make predictions that can guide real-time decisions. According to Google, the API can be used for prediction tasks ranging from product recommendations to churn analysis (predicting which customers are likely to switch to another provider). The API involves three simple steps: upload the data, train the model, then generate predictions. The API is currently available on an invitation-only basis." Google also recently announced several other API additions, including Buzz, Fonts, and Storage.

Comments

Psychohistory ?

[Vapula]

What about feeding it with historical events, train with the outcome from these events and try to get a glimpse at which way the future will evolve ?

Re:Psychohistory ?

[0100010001010011]

Or use the last half of your data set as blind data. Train the model on 1900-1990 and see if it can predict 1990-2000.

How far can you predict? 1%, 10%, 50%?

If you want to really see how good it is feed it stock market data and see how well it predicts that.

Re:Psychohistory ?

[Kilrah_il]

The nice thing about the stock market is that when everything is fine the analysts say that their models are great, but when something unexpected happens they go all "but we couldn't have foreseen that. Except for this unexpected incident, our models are great!". The problem is that these "unforeseen incidents" are what drives most of the extreme changes in the stock market, and more generally, in our entire society.
Just look at 9/11 (to use your example): It not only affected the economy, it affected (and still affects) our entire lives - from airport searches, to US PATRIOT acts to wars in Iraq and Afghanistan.
These extreme events are called Black Swans ( http://en.wikipedia.org/wiki/Black_swan_theory ) and I do recommend the book by the same name by Nassim Nicholas Taleb. Fascinating reading (if a bit repetitive sometimes :) ).
The bottom line: Trying to predict the future from past events is fine, until it breaks up, and it does so more than we care to imagine.

Re:Psychohistory ?

[fusiongyro]

+1! "Past performance is not a predictor of future success." Taleb is my hero. Everyone should read Fooled by Randomness, which I didn't find repetitive at all.

Re:Data mining

[LizardKing]

It's worth remembering the saying with data: "if you look hard enough, you can find anything you want to".

A friend of mine works as a quant at one of the big investment banks. He admitted that the models his team creates are useless at predicting the unexpected (as you'd probably expect). Adding in a degree of randomness rarely produces better models, as there are too many possible sources of such unpredictability and the reactions to them depend on many unquantifiable forces. This results in models that are OK at telling traders what they want to know - that they're doing the right thing by all doing the same thing. As soon as something undesirable or unexpected happens, then all hell breaks loose and the traders panic. Having mulled this over for a bit, I suggested his job was pointless, to which he agreed, but pointed out that the pay's great. So much wasted mathematical genius.

Re:Data mining

[E IS mC(Square)]

Often misused, definitely. But that does not invalidate the importance of any tool, including data mining.

One good example is Netflix recommendation engine. I know it's far from perfect (as there is nothing perfect about prediction), but is it useful? Hell yeah. It's the best recommendation engine I have used and have benefited greatly from.

Problem is when it's applied to areas where stacks are higher - like risk analysis by the investment banks.

And that brings me to mention an interesting (old) and related read - "Fooled by randomness" by Naseem Taleb.