Slashdot Mirror
Google Snaps Up Stats Tool from Swedish Charity
paulraps writes "A stats program that began as a teaching aid for a university lecture has just been bought by Google for an undisclosed sum. The statistics tool, Trendalyzer, was developed by a professor and his son at Stockholm's Karolinska Institute. Unfortunately for the developers, the project has been run under the auspices of a charity, Gapminder, and financed over the last seven years by public money. Maybe that seemed smart at the time, but the professor, admitting that he won't see a dime of Google's cash, now seems regretful. As for what Google has purchased: 'Public organizations around the world invest 20 billion dollars a year producing different kinds of statistics. Until now, nobody has thought of collecting all the information in the same place. That should be possible with Trendalyzer, which will be able to present that quantity of data in a clear way as well as giving the user the ability to compare many different kinds of information.'"
Google, I dig you for now, but I'm not really sure that I care for the idea of having google own nearly all of the search data for every search done by every individual around the planet in the history of google and beyond combined with all of the world-wide traffic analysis data.
And as someone who would be targeted for this service -- why would I bother? There are plenty of free open source utilities out there that provide every ounce of data you could ever want and they're incredibly easy to configure and deploy.
No, the benefit here seems to be less for the end-users deploying the service and more for whoever google then turns around and sells the massive amounts of correlated information to. For instance, let's see every bit of data about a specific user so we can see everything from each search he does to his entire browsing trail. Bet we could sell that for a lot of money!
Hopefully you will still have a simple way as a user to prevent google from collecting this information just like you can do with their stupid Urchin service (by blocking it). And, sadly, people will still continue to use this new service because they'll sell out their mother's medical history and offer up a sample of their own blood and cholesterol ratings if it means getting something "for free".
Neither article nor summary explain what Trendalyzer actually does. The animated mapping of stats at http://tools.google.com/gapminder is a little more illustrative.
...why isn't it already in the public domain?
Lot's of people have great ideas that never reach fruition for reasons that have nothing to do with them. And sometimes, those ideas can take off and be promoted for reasons that have nothing to do with them. Often these things offend our sense of fairness.
Yet life is not fair and often people have regrets and indulge in "what if" fantasies.
For something like this, even if the fellow gets no money, he can get publicity and recognition and might be able to leverage that into something to get him more money if that's what he wants.
The past is past and the price for obtaining "justice" and "fairness" can be quite high and more than one should have to pay; you can lose your future doing it.
Learn from the past and develop a plan to move forward and leverage on the lessons learned; the best revenge is always living well.
"I believe in Karma. That means I can do bad things to people all day long and I assume they deserve it." : Dogbert
Shortly thereafter, a site called Nation Master cropped up, with a bit flashier and simpler user interface, but focused on CIA World Fact Book data, rather than the States of the US. (The same folks later did State Master using similar UI technology.)
Finally, Google tested Gapminder with an even spiffier and simpler UI -- again focusing on by Nation correlations.
Aside from the usual complaints about "The Ecological Fallacy" (a fallacy that cuts both ways BTW) there are two big pitfalls for this stuff:
What I did about missing data was simply eliminate any data points where data was missing from one or both of the variables being correlated. This reduces the sample size, hence statistical significance, but it bypasses arguments over what sort of missing data should be used. The Netflix Prize is coming up with really good algorithms to compute missing data efficiently and accurately so maybe there is hope for something more effective here.
Statistical significance is more difficult to deal with. Usually one must look at tables for statistical significance of correlations under the assumption that the variables each follow a normal distribution. Unfortunately, many variables follow polynomial (like squared) or exponential distributions, so you have to do things like take the sqrt or log of one or both of the variables to try to normalize them. However, when you are looking for correlations, sometimes it its the relationship that is polynomial or exponential -- in which case you can apply sqrt or log to get the maximum correlation coefficient at the sacrifice of normality of one or both of the variables. Unfortunately, there is no simple arithmetic formula for calculating the significance level of a correlation given a non-normal distribution -- you can't just plug in the skewness, kurtosis, etc. as well as sample size and correlation coefficient, and get out a valid statistical significance. Therefore it is hard to make good statements about many very important correlations without watering them down to meaninglessness.
Also, a complaint about the "simple" user interfaces:
Some of the worst reporting from news media comes when they refuse to report statistics in terms remotely related to anything meaningful -- for example you will frequently hear statements to the effect that "California has the most orange trees in the nation." or some such. Such statistics are nonsense for the purposes of correlation studies since the size of the ecology (California state) is all you are really measuring with such statements. You have to divide by the population or divide by the total GDP or something to rationalize the ecology against other ecologies.
In Laboratory of the States, I did this with all my variables but I also left the raw variables around and allowed people to do arithmetic on them -- like dividing them -- to get their own rational comparisons if for some reason my choices were not adequate. This problem isn't as bad with Gapminder as it is with Nation Master and State Master -- but Gapm
Seastead this.