Slashdot Mirror
iTunes Sales Not 'Collapsing' After All
john82 writes "Earlier this month we had a report from Forrester, based on a random sampling of 2,000 credit card accounts, that purported to show that iTunes sales were crashing. Now comes another survey from Reston, VA-based ComScore which indicates the exact opposite. ComScore's report which is based on actual iTunes sales shows a 84% increase during the first nine months of this year compared to the same period last year. Meanwhile the author of the Forrester report, Josh Bernoff, noted in his blog yesterday that they shouldn't be pummeled just because everyone took what he wrote and ran with it."
Meanwhile the author of the Forrester report, Josh Bernoff, noted in his blog yesterday that they shouldn't be pummeled just because everyone took what he wrote and ran with it."
Well, that is why people should be responsible for their reporting. In my business, when you report something, you stand by it. If you present data or a theory with the suspicion that it is incorrect, that is fraud in my line of work. Seriously though, did you *really* think that a sample size of just over 1000 purchases on credit cards obtained through a back channel source is a reliable sample size for the number of iTunes purchases? If I correctly recall, Apple announced back in February that they were selling about 3 million songs/day and if the current estimates of increases on the order of 84% are correct, your sample size is woefully under-representative. Thats just high school statistics by the way...
I am not saying that you should lose your job over this one, but this should be a tacit reminder of how important good reporting is and if you are beyond your means or competence on a particular story or analysis, go find some help before you publish it, do some fact checking and be more careful with stories that can have a significant impact on companies and individuals.
Visit Jonesblog and say hello.
I stand by this man as long as he isn't proven wrong.
ComScore. With a reputation like theirs, it must be true!
"When life gives you lemons, don't make lemonade. Make life take the lemons back!" -- Cave Johnson
Technology sector analysts, the likes of Forrester and Gartner, are essentially paid mouthpieces for their biggest clients. Whether pumping your own products or badmounthing the competition, you can count on these guys to earn their money with totally bogus conclusions.
Find a big analyst company that will admit that Itanium is a colossal disaster, that businesses don't want and don't need Vista, that HP's supply line trouble and incompetent management are sinking the company (particularly during the Carly years), that Oracle is terribly insecure. You won't, because they all have contracts with Intel, Microsoft, HP, Oracle, etc. But they won't hesitate to beat up on Sun (how many times have they called for McNealy's resignation), AMD, Apple, and predict their doom*, and others that don't spend the kind of money on various analysis contracts.
So sure, iTunes sales are collapsing (according to Forrester), but nobody will call Zune a turd. It's all in a day's work.
*disclaimer: I might be considered a fanboy of one of these companies, and it's not Apple
Ah yes, truthiness. How I've missed thee...
For that matter, the Forrester data was based on credit card payments on the iTunes Store.
It totally ignored the little lime-green $15 gift cards that litter the checkout stands of every Target, Best Buy, CVS Pharmacy, and Kroger in the US. Each one of those is 15 songs, and fifteen purchases that don't register as credit card transactions.
The US free market: two halves of a government-granted duopoly are free to set the market price.
He concludes with this statement in his blog:
To a degree, he has a point. With Apple's secrecy, articles like these are run without having all the facts. Sensationalism becomes rampant. Then he has to go and say "In the research business we like facts." All too often we read more about speculation rather than facts from these research companies. They complain secretive companies like Apple or Google don't give them enough information, but I wonder where the actual "research" in research business has gone.
hackers of the world unite!
A few readers commented when the story was posted yesterday that they were wondering "How" the credit card data was obtained. It seemed from yesterday's story and the posts that Forrester Research had obtained credit card detailed transaction lists (w/o the credit card numbers, etc, I hope!).
So, I would like to ask, how was the data obtained and is this level of detailed information avilable for legal purchase? I'm just curious as to how much information is available about credit card puchases.
He should take a hit on credibility. He maybe should be fired. But I agree that physically injuring him is probably more than he deserves.
Avoid Missing Ball for High Score
I've been reading Forrester, Jupiter, IDG and other pundit research papers for over a decade. They're almost always just rationalizations of some preconceived notion, some foregone conclusion that their methodology reinforces. I don't know if they plan it, or if marketing people just can't tell science from "Tang". But I don't know why anyone reads these reports expecting anything but a blast of conventional wisdom.
Which is, of course, why everyone just takes what they write and run with it. That's the measure of success at marketing research peddlers. It's the CIO self-perpetuation. One reason why so little ever gets done right, but so much does get done without being called wrong. To blame their own market for taking them seriously when they ought not be is finally a whisper of honesty from these chattering weasels. I expect them to fix that in the next release.
--
make install -not war
"Look at the trends here"
Yes, there was a massive spike last Xmas that hasn't been exceeded during the 11.5 months that followed. Indeed, if you draw a line from that peak to the present, iTunes queries are down from a year ago. It's proof positive - especially if you don't know a fucking thing about statistics!!
I can't find my ass with both hands around statistics and even I can see what's wrong with Forrester's report. So, Forrester my ass.
Most of the stuff on
Actually, the sample size of 1,000 was probably fine, or at least it would have been if they had used a truly random sample of credit cards. However, it is evident from their results, that they didn't. The failure was in trying to extrapolate results from data that wasn't statistically valid.
Where your information belongs to everyone but you.
putting the 'B' in LGBTQ+
From the testimony of Mr. Marc E. Kasowitz before the US Senate Committee on the Judiciary:
& wit_id=5486
One particularly effective illegal strategy involves the
following scenario: the short-selling hedge fund selects a
target company; the hedge fund then colludes with a so-called
independent stock analyst firm to prepare a false and negative
"research report" on the target; the analyst firm agrees not to
release the report to the public until the hedge fund
accumulates a significant short position in the target's stock;
once the hedge fund has accumulated that large short position,
the report is disseminated widely, causing the intended decline
in the price of the target company's stock. The report that is
disseminated contains no disclosure that the analyst was paid to
prepare the report, or that the hedge fund dictated its
contents, or that the hedge fund had a substantial short
position in the target's stock. Once the false and negative
research report -- misrepresented as "independent" -- has had
its intended effect, the hedge fund then closes its position and
makes an enormous profit, at the expense of the proper
functioning of the markets, harming innocent investors who were
unaware that the game was rigged, and damaging the target
company itself and its employees.
http://judiciary.senate.gov/testimony.cfm?id=1972
Student exercise: Compare and contrast with the movement of AAPL stock shares before and after this report came out.
For example, if your sample is 1000, your 95% confidence interval is 1/sqrt(1000) = +/-3%. So if your 1000 samples showed 250 occurrences, you would know that it's 95% likely that the frequency of occurrence is between 22% and 28%. So the real frequency could be between 220 occurrences or 280 occurrences per thousand. No big deal for year to year comparison purposes. Worst case a 50% drop in sales is measurable because one year you could've been low (220), and the next year high (280/2 = 140), and the change is still statistically significant (outside your confidence interval).
For rare phenomena, this runs into a problem. Say the frequency of occurrence is 0.1%. You take 1000 samples and you measure 1 occurrence. The neophyte statistics student will say "Cool, I meansured 1 occurrence +/- 3%, so I have 95% confidence that the actual rate of occurrences is between 0.97 per thousand and 1.03 per thousand." Unfortunately, that's wrong.
The confidence interval is based on the percentage you measured. Your confidence interval says there's a 95% chance that the actual frequency of occurrence lies between 0% and 3.1%. There is a huge, huge difference between 1 incident in a thousand and 31 incidents in a thousand, especially if you're trying to compare between two samples. One sample (year 2005) you might get 25. Next sample (year 2006) you might get 5. These are both within your confidence interval, but if you're not careful you would erroneously conclude that you have 95% confidence that sales plummeted to just 20% that of the previous year.
Put simply, if you want to accurately measure rare phenomenon, your sample size has to be large enough that your confidence interval is significantly smaller than the rate at which that phenomenon occurs. If iTunes sales account for 0.1% of all credit card sales (which I think is a very high estimate) and you want to compare year to year changes, you probably want an accuracy of at least 1/10th the 0.1%, or a margin of error of +/- 0.01%. Your sample size needs to be large enough that your confidence interval is around the 0.01% range. That is, you need a sample size of a 100 million credit card transactions.
based on a random sampling of 2,000 credit card accounts,
Ummm... Now, I harbor no delusions that my credit card history really counts as a secret - Obviously my CC company has it and uses it to market bizarre crap to me, and they'd turn it over to the government without thinking twice about it.
But how does some guy just go and "randomly sample" 2000 cards' histories? If I wanted to validate his study, could I do the same?
Something doesn't seem right here, and I don't think most people would like the "how" either way.
Seems to me like a pretty clear admission that the sample size is too small to be reliable. He took the data he had available, analyzed it, and presented the results while noting the deficiencies in the method. Doesn't sound much like fraud to me. That's just grade school reading by the way...
So in other words, he knew from the beginning that he was spewing out bullshit. The article never should have gotten past the editors. One can argue back and forth whether the journalist should be disciplined, I'd argue for it and for an investigation of possible conflict of interest, but there's no way the editors should have let the article through as it stood. They should have been canned.Beta is broken and the link to classic doesn't work. Stop wasting our time or there won't be anybody left here.
a new study shows that studies are not factual at all and are used as propaganda by large powerful companies.
there are three kinds of lies in this world: lies, damn lies, and statistics.
No, I don't think you are an exception - that's the same meaning I took from the most specific reading of the sentence. However, I'm saying that I see no point in even mentioning a "real drop" in their sample data when it simply can't be extrapolated to sales as a whole, especially in a research document they are flogging: to me it appears to be a triumph of marketing over valid conclusions. "Gee, look at this dip! (PS. oh, but don't take any notice of it please!)". Cheers.
"Get off the cross - we need the wood" - Tori Amos
I thought it would be fun to compare slashdot comments to the previous posting to see how many geniuses out there fell into it with "I told you so," "It's because Apple is a big meanie," "Songs are no good," and similiar contributions. But I have to say after reading through the previous posting's comments, though there were a few like the above, the vast majority of slashdotters called it correctly and said the previous study was flawed, giving all the reasons why. Impressive!
How about a moderation of -1 pedantic.