Slashdot Mirror
Why Good Data Can Be Hard to Find Online
WSJdpatton writes to mention that Carl Bialik has an interesting look at why good data can be hard to find, much less understand, online. He cites a couple of examples, both Google's first-quarter performance numbers and Alexa's revamp of their number-tracking process. "Now Alexa is incorporating other sources of data -- though it says the prior ranking 'wasn't wrong before, but it was different.' Some sites saw big changes in their rankings following Alexa's move: The tech blog TechCrunch said it fell far from its prior position in Drudge Report territory (rarefied air in Web-traffic terms). On Friday afternoon, Drudge Report ranked 545th, compared with TechCrunch's ranking of 1,784th, according to Alexa's new math."
This isn't exactly on topic, but I think you should give it a read before you make a final opinion on what the article is trying to stay.
I read online somewhere that 70% of statistics online are made up. This article seems to prove the point. 4 out of 5 slashdotters agree! ;)
I don't know what they're complaining about, mine went down over a million positions!
http://crummysocks.com
Another example besides Alexa of "readjustment" is Hitslink. Last November, they revised their figures for OS share for March through October 2007. Linux went from a reported .81% share in October, to .50%. They made only a brief allusion on their site to filtering out "unrepresentative" hits from their data. Recently, they again revised their Linux share for January 2008, from the original .67% to .64%. Even though Hitslink seems to have trouble deciding how many Linux users there are, that doesn't keep people (like Westlake, who keeps posting Hitslink numbers on Slashdot) from citing them.
http://www.gooddata.com/
Well, maybe not good data.
Just observing the Internet and then reading this ... just wow.
Good data is HARD to find ANY FUCKING WHERE, never mind limiting your search to just online. Seriously!
News online? read the same story from 8 sources, form your own opinion. MSM sucks worse.
Scientific data? Well, unless it's peer reviewed, you know it's probably suspect and need to verify it with other data. Damn, even peer reviewed scientific data should be compared to other data these days.
How about Encyclopedic data.. There is wikipedia, but make sure to corroborate the data, right?
Read it in a blog? Check the data before you make up your mind.
Hmmmm this sounds a lot like trying to find good data before the Internets were active. Damn, all that data is proffered up by humans... Humans are not infallible so I'm guessing that data provided by humans is going to be a bit 'not infallible' also.
Where does the assumption that data online should be good data come from? wtf?
Support NYCountryLawyer RIAA vs People
Only Bad Lore can.
"good data can be hard to find online..."
Yes, especially if you're reading Drudge.
it is just your google skills that's sucks... that's what my boss keeps telling me!
"Steve Jobs invented the world" -- Bill W. GATES
Alexa confirms it.
With Comcast's monitoring of user traffic, they could provide reliable stats for their customer base. We ought to get something back from all this Big Brother stuff.
I initially read this as being, "Why a Good Date Can be Hard to Find Online". Hell, I could have told you that! But alas...
... is good to find.
A public health expert from Sweden - Hans Rosling, who teaches at Karolinska Institutet - has (some time ago, already) announced that he was able to persuade holders of UN-collected population data to publish their data on-line for anyone wanting to analyze it (eg, using his innovative tools for displaying it: GapMinder).
;-)
I would say that the data which he managed to get put on-line for anyone's use might be a counterexample to the poster's claim.
Of course, you can decide for yourself...
See his 2nd talk at TED.com for URL's and other details regarding access.
The reason good data is hard to find online is chiefly a problem with perspective and the models we are using to differentiate good data from bad data. That model primarily relies on the idea that it's all about numbers, or simply that more data is the same as better data. Whenever we come up with bad data, the "quantity model" dictates we just need a larger sample.
This model is directly related to how companies measure TV show quality. The theory is, the more people who watch a show, the better that show must be. This model is so obviously faulty; almost everyone can agree that American Idol isn't even in the same qualitative ballpark as The X-Files, Arrested Development, or Star Trek. The reason the model is faulty is because of the hugely limited scope of the examination. There are a number of variable factors that aren't being considered, such as people own more TVs than when Star Trek was on, and they're mistaking curios interest with enjoyment. Average person will stop and watch a car wreck for roughly the same amount of time they'll play with a yo-yo, that doesn't mean the entertainment value of each is directly comparable, there's a whole different brain process going on in the observers of each, but the model of measuring quantities assumes that two activities which consume the same amount of time are equivalent in all ways.
Back to internet statistics. All this data mining and gathering is designed to ignore the differences in activities, it's only cataloging information for the purpose of what's the same. As the article states, Alexa is always checking for biases. Well the biggest bias in this model is the assumption "in sufficient quantity, all things are interchangeable." It's the assumption that telemarketers and scammers work on, which is why so many people go broke buying into those schemes, because they buy into an assumption which is absolutely wrong.
Many internet business models, specifically data miners, are designed on, assume that 1 million hits is the same regardless of where it comes from. When you consider real factors, having 1 million people see your hand-made chain pouches at a shopping mall is not going to generate the same level of interest as having 1 million people see them at a renaissance fair.
Of course that introduces a whole different problem with assumptions about targeting (I'm not going to get into that, only state that targeted marketing makes the assumption that timing doesn't matter).
In conclusion, you can't play people as a numbers games, People's behaviors (including their online behaviors) are complex and any model which treats people's differences as a child might divide up a bag of skittles by color is going to have a very high error rate.
This is a problem I've noted before (for example, here). I have an equivalent Google page rank with sites with hundreds of times more traffic. In short, I've yet to see the metrics or analytics tool that is truly reliable.
Development is programmable; Discovery is not programmable. (Fuller)