Slashdot Mirror
Economic Predictions Using Web Usage Data
Makarand writes "The Chicago Tribune has an article on the claims of ComScore Networks Inc., that it can predict major economic
trends by tracking the online activity of 1.5 Million people.
The company gains access to people's Internet travelogues by giving
them free security software and programs that speed up their connections. Economists say that
the company's models need to be tested over several years
before they can be considered accurate."
ComScore also looks for trends in the credit card statements that about 30,000 of its panelists view online
Is it just me - or does that sound slightly worrying?
They claim to look at a cross section of society, but I'm willing to bet only the criminally insane would sign up knowing that they are perusing your credit card statements...
Sig (appended to the end of comments you post, 120 chars)
As a second gauge of spending, ComScore also looks for trends in the credit card statements that about 30,000 of its panelists view online.
That's right: If you have their spyware installed on your computer, they are going to be looking through your credit card statements.
Why isn't this illegal yet?
Tarsnap: Online backups for the truly paranoid
Notwithstanding the privacy issues involved, which have been discussed by other people -
:)
I'd have thought that if you could get a representative group of people of sufficient size, and allow for intrinsic skew in the data, then watching what they do online - what their ecommerce browse to conversion rates are, whether they're shopping at all, whether they're looking at holidays, cars, that kind of thing - could well provide a very good short-term predictor of where the economy is going next.
You could find out, for instance, that people were planning to buy new cars or go on a long-haul holiday weeks or months before that was converted to Real Money in the retailers' pockets, and upwards of three months before the quarterly reports from the companies themselves start to reflect the changes in the economic climate.
Sounds to me like this could be a really interesting toy to use as an adjunct to playing the markets
Danny.
I have written over 900 book reviews
I recently learned about Zipf's Law, which uses a very simple formula to predict quantities of all sorts of things.
It's truly amazing. For example, it accurately predicts the populations of the 10 most populous cities, the number of appearances of the 10 most oft-used words on the entire Web, etc.
From a quick Google query: "Zipf's law, named after the Harvard linguistic professor George Kingsley Zipf (1902-1950), is the observation that frequency of occurrence of some event ( P ), as a function of the rank ( i) when the rank is determined by the above frequency of occurrence, is a power-law function Pi ~ 1/ia with the exponent a close to unity."
Here is some more information: http://linkage.rockefeller.edu/wli/zipf/
If you celebrate Xmas, befriend me (538
What bothers me here is that the programs that are being used to bring spyware to the common user are programs that do things for which either there already exists an spyware-free solution, or is a program us /.ers could write in minutes.
Speeding up an Internet connection is more-or-less a myth in the first place, you can't make software to cause a modem to go any faster than it goes physically. The only thing that really can be done is to make sure there's nothing stupid in the Windows registry slowing down the connection... and guess what, in older versions of Windows there is! Microsoft initially set the Maximum Transfer Unit (MTU) to something that made sense on a LAN connection, but caused an annoying number of retransmitted frames on a modem connection. Lower that number to something sane, and web pages will appear to the user to be faster. However, that didn't really speed up the modem, it's now just not wasting as many cycles on bad data. Changing the MTU number is a registry hack, the program needs to only be run once... no need for it to be there on every boot.
Another such program syncs your computer's clock with to official U.S. Government time. That's a cool and useful function, but it's really just using the Network Time Protocol (NTP) standard to contact government servers. Anybody who bothers to read the docs can write their own program to do that. Microsoft has even built NTP into Windows XP, although once-a-week updates isn't exactly enough for most users who care about their clock accuracy.
Another program hitches its ride offering the local thermometer reading from your local TV station's WeatherNet system in an icon in your system tray. Cool feature... but wait a second here. What if you don't live near a WeatherNet site? Oh, that's simple, it taps into the National Weather Service data to get you a report. But NWS's data is public, paid for by your tax dollars. The info is available on both FTP and HTTP servers that are absolutely free to access.
Open source projects could knock these "Download me!" programs out of existance. Why don't we?
Except that in ComScore's case, #2 is "Provide that something for free if people agree to be tracked by spyware.". If this is the ComScore I'm familiar with (and it looks like it is), they're up-front about what they're doing. You get their package knowing that you're becoming the equivalent of a Nielsen family with every single page you view being tracked and recorded. I personally wouldn't agree to that, but I imagine there's a lot of people who wouldn't care (or who have another computer for the browsing they don't want recorded).