iTunes Sales Not 'Collapsing' After All

john82 writes "Earlier this month we had a report from Forrester, based on a random sampling of 2,000 credit card accounts, that purported to show that iTunes sales were crashing. Now comes another survey from Reston, VA-based ComScore which indicates the exact opposite. ComScore's report which is based on actual iTunes sales shows a 84% increase during the first nine months of this year compared to the same period last year. Meanwhile the author of the Forrester report, Josh Bernoff, noted in his blog yesterday that they shouldn't be pummeled just because everyone took what he wrote and ran with it."

Comments

Own up to your reporting

[BWJones]

Meanwhile the author of the Forrester report, Josh Bernoff, noted in his blog yesterday that they shouldn't be pummeled just because everyone took what he wrote and ran with it."

Well, that is why people should be responsible for their reporting. In my business, when you report something, you stand by it. If you present data or a theory with the suspicion that it is incorrect, that is fraud in my line of work. Seriously though, did you *really* think that a sample size of just over 1000 purchases on credit cards obtained through a back channel source is a reliable sample size for the number of iTunes purchases? If I correctly recall, Apple announced back in February that they were selling about 3 million songs/day and if the current estimates of increases on the order of 84% are correct, your sample size is woefully under-representative. Thats just high school statistics by the way...

I am not saying that you should lose your job over this one, but this should be a tacit reminder of how important good reporting is and if you are beyond your means or competence on a particular story or analysis, go find some help before you publish it, do some fact checking and be more careful with stories that can have a significant impact on companies and individuals.

Re:Own up to your reporting

[RichPowers]

Indeed, he should stand by his work. But all of the websites and blogs weren't afraid to use Mr. Bernoff's report to drum up an Apple doom-and-gloom story for the sake of attracting readers.

Re:Own up to your reporting

[dcollins]

"Seriously though, did you *really* think that a sample size of just over 1000 purchases on credit cards obtained through a back channel source is a reliable sample size for the number of iTunes purchases?... Thats just high school statistics by the way..."

I'm a college professor of statistics. I don't think you can actually quote a high school statistics book which says that sample size is too small. In general, a sample size of 1,000 gives 95% confidence that your result is within +/-3% of the actual result. This is *regardless* of population size - that's how statisatics work, due to the Central Limit Theorem.

http://en.wikipedia.org/wiki/Sample_size
http://en.wikipedia.org/wiki/Central_limit_theorem

Now, the first thing that pops into my head is why only credit-card purchases? And even more fundamentally, why would the same people need to buy music, after they just went on a music-buying spree? I would think the opposite. That was the thing that made me skeptical of the report yesterday in the first place.

Re:Own up to your reporting

[lendude]

That sentence you quote is having it both ways:

"Our credit card transaction data shows a real drop (my emphasis) between the January post-holiday peak and the rest of the year, but with the number of transactions we counted it's simply not possible to draw this conclusion . . . as we pointed out in the report."

There is no way that he can use the words "...real drop..." in the same sentence as "...it's simply not possible to draw this conclusion...". Whilst those who uncritically took the information from this 'research' and used it (doubtless with some sensationalistic agenda in mind) deserve scorn, that very sentence itself demonstrates the research to be nothing more than PR to flog the thing at $249.00 a pop. If you take out the words "real drop" and substitute "no meaningful change" then this report was clearly worth fuck-all: at least in terms of the author's now visible desire to have something sexy to sell!

Re:Own up to your reporting "I stand..."

[Elsan]

I stand by this man as long as he isn't proven wrong.

from Josh Bernoff's blog

[defy god]

He concludes with this statement in his blog:

"Finally, a word for Apple. Apple is extremely stingy with information about their business and public comment. Their unwillingness to comment on the record or off about anything they're working on or any industry results beyond the basic statistics fuels speculation, pro and con, from their supporters and detractors. In the research business we like facts -- and every other technology company is more open with them. So maybe it's time for Apple to share a bit more. When the real bad news hits -- and it's inevitable, no company gets everything right -- that openness would pay off."

To a degree, he has a point. With Apple's secrecy, articles like these are run without having all the facts. Sensationalism becomes rampant. Then he has to go and say "In the research business we like facts." All too often we read more about speculation rather than facts from these research companies. They complain secretive companies like Apple or Google don't give them enough information, but I wonder where the actual "research" in research business has gone.