Slashdot Mirror
"Long Tail Effect" Doesn't Work As Advertised, Say Wharton Researchers
Death Metal writes "In a working paper titled, 'Is Tom Cruise Threatened? Using Netflix Prize Data to Examine the Long Tail of Electronic Commerce,' Wharton Operations and Information Management professor Serguei Netessine and doctoral student Tom F. Tan pull information from the movie rental company Netflix to explore consumer demand for smash hits and lesser-known films. Netflix made its data available as part of a $1 million prize competition to encourage the development of new ways that will improve its ability to introduce customers to lesser-known titles they might find appealing." In short, the researchers say that the Long Tail effect described by Chris Anderson is much less important in the real world than popularly held. Says the article: "The key difference between the opinion of [Anderson's] book and the study by Wharton researchers is how they define 'hits' and 'niches.' In the book, Anderson focuses on the definition of hits in absolute terms such as the top 10 or top 1,000 products, while Netessine and Tan argue that, to take growing product variety into account, one has to define popularity in relative terms, such as the top 1% or top 10% of products, to properly assess the presence or absence of the Long Tail."
The long tail doesn't threaten those at the top any more than it isolates those at the bottom. It only describes the shape of the market which necessarily has only a few specific market products which are used by the majority and the rest of the products with very few customers in the "long tail". It's a market definition, not a competition definition.
You can cut the tail off of a gecko at any point, but it doesn't mean that somehow the tail can exist without a fat end and a thin end. Since the tail is simply the appendage attached to the abdomen, wherever it is attached defines its fat end, and where it ends is the thin end. Even if you cut the tail off completely, all that you've done is stimulated the tail regrowth reflex.
OK, I am not a mathematician, but this paper makes me deeply skeptical.
If the input data is indeed heavy tail (non-existing higher moments) or quasi-heavy tail (existing, but extremely large higher moments) how on earth they can use variance, R^2 and other measures? They may not even exist! And if the input is quasi-heavy tail, then of course they exist, but the convergence time could be arbitrarily long!
I had the unpleasure to work with quasi-heavy-tailed data, and it is really enlightening. You watch the evolution of some metric (e.g.: avg) as the function of incoming data, and you see of course convergence. At least for a while. And then in sudden an extreme outlier comes in, and the avg takes a huge jump! Now if your input is heavy tailed enough, you can be never sure that your measure finally came to rest (converged), or the next jump is just over the corner!
I hope a more educated person clarifies this, I am just an engineer.
The crux of Andersen's argument is that, while Amazon et al have the same demand for big-name titles, their tail is longer and higher than a traditional bookstore, and by defining a cut at a certain point (say, those with less than 5% of the peak sales, those outside the top 10% or whatever is appropriate) it can be seen that the low-volume sales represent a larger fraction of the total sales due to the extreme length of the tail.
Quoting from the Wikipedia article on the topic:
In the graph shown above, Amazon's book sales or Netflix's movie rentals would be represented along the vertical axis, while the book or movie ranks are along the horizontal axis. The total volume of low popularity items exceeds the volume of high popularity items.
Andersen was suggesting that, in the limit of infinite items to sell and negligable stocking costs, much more profit is to be derived from the large number of items that sell a few copies than the few items that sell many copies.
Indeed, it went further than that, suggesting that as people got used to having more choice, they would begin to shun the "popular" items in favour of more obscure titles, further fattening the tail. But that's even more speculative and somewhat independent of the other economic predictions.
Of course not. You must have it. Now that you have Dewar's represented, should you also add Dickface Brand for half the price?
You have *no* idea what you're talking about.
Everyone knows that Dickface Brand is a bourbon, not a scotch.
"Slashdot - News and Chat Sites Deviant". (Click "homepage" link above for details).
The thing that's always struck me about the long-tail effect is that you've got to work it, to get value from it. Just having all the books or films by a particular author / actor isn't enough. You have to use that information and have the intelligent algorithms to guide your website visitors (or maybe "entice" would be a better word) to consider those alternate products. Just saying "Uuh, here's all the other stuff that guy's done" isn't enough, it needs enthusiasm and some knowledge of *why* a visitor might like a particular past work. That's where the gold lies: not in the long tail itself, but how you utilitise it.
politicians are like babies' nappies: they should both be changed regularly and for the same reasons