Slashdot Mirror
Evergreens: What The RIAA's Doing Wrong
Chris Johnson writes: "Recently I've been doing some heavy analysis of actual RIAA sales data, from the entire history of Platinum-certified albums. I've worked out a methodology that compensates for the explosion in CD sales and highlights ability to drive sustained sales over years. There's a top 10 list of albums and a top 10 list of the most commercially important artists in history with definite surprises- and the full lists as well, downloadable as text files, with Perl-friendly index numbering, so the analysis can continue with the annoying work already done! Perl folk, go nuts! The actual analysis takes this data and attempts to extrapolate from it and explain the competitive situation the music industry is in relative to Internet music in general, and what they are missing in their assumptions and plans. Should be interesting to see what people make of the Evergreens project! -Chris Johnson" Nice analysis of the history and future of music promotion.
As a result, Revolver would have even less promotional muscle and would flop even worse- "Love You To? Tomorrow Never Knows? What are you guys, trying to blow it here?", and there would be no budget for a symphony orchestra on Sgt. Pepper, which would be orchestrated much like Let It Be- kinda low-fi early-days sounding.
When that flopped, forget getting approval for a double album- forget getting tracks like "Revolution #9" on said album- forget even getting approval for the minimalist cover, and hence the very name 'White Album'- and goodbye to the (according to some! ;) ) most commercially important Beatles album in history.
The rules _are_ different these days.
No problem at all. Nobody is making you. I _did_ sit and manually do it. You think that's worth nothing (or worse than nothing?). I charged you nothing, so we're even.
I'm sorry you're _not_ going to sit and manually do it properly. I would love to have better data to work from- I simply did not have the time to get everything I could.
Assuming the RIAA site remains up and continues to offer that search facility, I may at some point do the whole horrible task over, informed by the criticisms I've seen here- _especially_ if, as I suspected, not a single blessed one of these critics is themselves willing to sit and manually retrieve all the data from the RIAA's balky NT server. Some sort of bot or HTML-parser might be in order: I wrote software to add artist career totals rather than add all the individual albums up with a hand calculator.
However, if I am the only person in the world expected to do this absurd feat of singlehanded data entry and cross-correlation to fix faulty records... you're not going to see it done right away. If you are that determined to say "THIS is how the job must be done", well... you know what to do! It involves sitting. I can confirm that part... >:)
I have a hunch (plainly mentioned in my words on the subject), largely brought on from having seen every cert date in the course of doing the analysis, that in recent years the ability of acts to continue to sell albums through back catalog has _really_ fallen off. I named Warrant- double platinum, then no further certs in ten years. I think there's a pattern there- I saw a lot of that. Some of it must surely be just due to the fact that you can't release _last_ year and get a cert for five years from now- but there's no compelling reason for the times to be so obviously compressed, the entire commercial lifespan of an album compressed into a matter of eight months and then fizzling out completely.
I think this would show in a listing of 'record lifespans' measured by certifications. In particular- I would love to see the same data sorted using this formula:
I think this could potentially turn up some real surprises. It would certainly be a much more effective metric for revealing 'evergreen' albums. You could again add artists' albums together to return career numbers, this time more clearly indicating which artists tend to produce albums that keep selling many years after release.
Even with the limited data I ended up with, there's one trick I didn't get around to doing: instead of summing career totals, average them. I can tell you right off two results- soundtracks and Broadway shows would drop way down in 'score', and Elvis would drop way down in score, because he faded into obscurity on an endless string of lame albums and repackagings. I may just give that a try because now my curiosity is aroused... because I am no sort of professional analyst, I am essentially hacking with the data to see what happens, and trying to come up with interpretations of what I'm seeing. For instance, until I did this analysis I had no idea whatever that movie soundtracks were such astonishingly big business. The bulk of those are _recent_ and still scored shockingly high- and led to some insights on convergence and why this was happening.
Again, if anybody does manage to suck all the data off the RIAA website (remember, huge amounts of it is incomplete or flat-out wrong! That was more than half the work!), I would love to hear about it. I know there's a lot more to it than what I ended up with. I just couldn't do more than what I did, right at the moment.
I'll add that I have been kicking myself since halfway through for only noting down the product of those two fields and not keeping the fields themselves. The 'alphabetical' list is literally what I was typing in, record by record- at first because I wanted to get a quick look at the multiplatinum level, and then I was stuck either going back and starting again, or going on... Please do show up my inadequacy as a data analyst by going to http://www.riaa.com/Gold-Intro-2.cfm and making nice downloadable text files suitable for data munging that contain ALL the information you can get from that source, i.e. artist, album, release date, cert date of each certification. I would love it if you did that! Would have saved me days of exhausting and nonprofessional work. I'll further warn you that loads of the information is either missing or flat wrong, and must be corrected through cross-referencing with other sources- _this_ I did, well I thought.
There is no original files- what you see is literally all I have. http://www.riaa.com/Gold-Intro-2.cfm is the source for mining. If you mean it and have the facilities to do a better job at this, may I please, earnestly and in _total_ _sincerity_, beg you to do it? I'm not a data analyst: I'm a sound engineer dabbling in many other fields. If I'm not mistaken my own limited analysis is _the_ _only_ compiled data source of this information out there and publically available. Now that I've taken a shot at it, maybe someone or some open-sourcey group can get some people together and do it right? Nobody would be more pleased than I. Eighteen-hour days of data entry sucked...
I don't quite understand why this is a relevant measurement of album performance. It weights albums dramatically towards those that have been out longer - witness the absolute domination of the Beatles atop the list of performers, simply because their albums were released first. I like the Beatles, but this method of accounting doesn't make all that much sense to me.
Of course this study is going to indicate lackadaisical performance by the Britney Spears and N'Syncs of today's music world: their albums haven't had nearly enough time to build up the massive multiplier factors awarded to those albums from older, more established bands. If an album were to be released today that became an instant craze, selling 60 million copies in one year, it would take yet another year to eclipse a merely double-platinum album that had been released in 1970.
Clearly, the project is trying to weight away from such insane explosions of buying frenzy, and towards artists who have established careers practicing their craft. This is a noble goal, but explicitly multiplying by number of years since release causes older acts to automatically become more significant simply by virtue of being older; a one-hit wonder from the early eighties will outrank any number of career artists who started in the mid-nineties and show no sign of stopping anytime soon.
Perhaps a better weighting factor for the purposes of the study would be one that awarded different point values to different platinum events. An album's first platinum would be awarded a flat rate; subsequent platinum certifications could be multiplied by the number of years since the album's release. This could be somehow extended to artists, as well - artists who consistently earn platinum status could be rewarded, even if no individual album does extraordinarily well. If you haven't gone platinum in a while, you don't get awarded more points for rocking in your chair and reminiscing about when you were a star.
The "top n" artists aren't their's., They produce nothing. They have no artists or pressing/burning facilities.
They are the music mafia. They exist to maximise profits. New artists cost money to produce and to promote. Old trash is pure gravy to them. The deader the artist, the happier they are.
Ever wonder why Elvis is still selling albums? Who gets the money?
Why would anyone invest in yet another Boxcar Willy record?
Its not because they like 'em. Its that the artist gets squat (public domain,) they shuffle the playlist, it costs squat to make (25 cents a CD including cover art [its some old photo scanned with a different photoshop text layer,]) and they are rolling in it.
They are parasites on the artists' efforts.
But the blood suckers have become so engorged that they now control the show. Its like a leech on your ass guiding you to chairs with holes in them, (you know the room and what happens there,) so they don't get squashed.
End result, you get to listen to shit.
The MPAA's the same. But you get brown shoved in your face instead of having it crammed in your ears.
You're paying for music but you're not the one who bought it. Ever wonder what the elevators are filled with? The malls, that theatres, every public gathering place. Ever wonder why? Its not for your benefit. Its not for your enjoyment. This stuff is bought by people who don't listen to it.
Now the RIAA & MPAA have a major threat to their business model.
CD burning and the internet can absolutely wipe them out, so they get some kid arrested in Europe and drag his ass to New York, they get CD-ROM burner manufacturers to add all kinds of crap to make recording and copying impossible.
Supposedly its anti-piracy, but its not. Pirates use equipment that can do bit copies and never even attempt to interpret the stream. Its anti-YOU. If they lose control they are history.
Like the mafia going to the matresses, there's a lot of collateral damage and they really don't give a shit if a few civilians get killed in China or their rights get trampled in Europe.
They are going to make money if they have to rip your pants off. But there are laws that not even they can break. And if you have no money, they don't get any either...
I've turned off the TV and radio, stopped going to the movies and my advice to the xxAAs out there is to get other jobs.
MSBPodcast.com The opinions expressed here are my own. If you don't like 'em... Think up your own stuff.
98 degrees, N'sync, and the backstreet boys "came out of the blue" ? that is a complete fallacy. The backstreet boys and N'sync are both creations of Lou Pearlman, who manufactures groups like ford manufactures cars. these bands fall COMPLETELY in line with the kinds of changes the author was discussing, and only serve to further his point.
So, in reality, your longwinded rebuttal only serves to strengthen the author's hypotheses. thanks!
EOM
Now, Chris doesn't make any real bucks on this. But someone you might have heard a bit more about does... Heather Alexander, a Celtic fiddler/singer/songwriter, actually makes a living at this, both for herself and her hubby/agent. No, they're not rolling in it. But they're doing what they want to be doing, and making a living at it. I would call that success.
The point is, the music industry outside of RIAA is not dead. There are a lot of small labels working outside the box, and bands who get airplay on alternative stations and rackspace in mom and pop music stores... and a growing audience that does NOT listen to Top 40 anymore. When enough people figure out that there's more to life than "Oops, I did it again," RIAA will lose its stranglehold on the business, and the world will change. A very quiet revolution, but I think it's already happening, given the amount of noise going on in the courts....
No, I didn't get a dime for those shameless plugs, I'm just a fan. Deadhead-style band promoting is alive and well.
But how did you know you liked their first album before you purchased it, and that you didn't like their second without purchasing it?
We rely on industry-controlled promotion (radio play, video play, insertion in soundtracks) to inform us about a lot of what-we-might-like. If it was just Hootie and the Blowfish, it could just be a bad album. But if it's a trend where H&B is simply an example. . .
I am excited by this research, but sadly I must agree.
... OLDER ALBUMS! What you really need to do is look at sales figures year after year and compare those.
For example, the author of the study does not even *mention* the effect of the switch from LPs to CDs. When records went nearly extinct in the 80s, people rushed to re-buy their favorite old music, essentially giving pre-CD artists twice the hits they might otherwise have. I think this reflects in the author's analysis. Also, I'm going to take a stab at "analysis" here and say that multiplying the number of millions time the number of years since release is radically biased towards
I was also disappointed to not see the names of any of these "sporatically important" 90's acts. When he mentioned earlier on slashdot that he was conducting this research, I looked up sales figures of Nirvana's Nevermind myself out of curiousity. Nevermind has sold 10 million copies, with the latest in 1999 (you can check this for yourself). This demonstrates that some artists 90s artists can continue to sell records after they stop being promoted.
The time to look to the music industry for a career is over. There is absolutely no future in it. What this means is, other opportunities will be opening up.
... THEY'RE NOT LISTENING. Facts are Napster has boosted sales in music in some polls, and other polls state it has harmed it. Well only one can be right, and that one will forever be an opinion of someone's.
Opportunities have always been openining up, but this does not mean the music industry is going to die. This was an argument some time back only it was with the film industry when many invested in dot com companies who were going create movies for the Internet and about 99.999% of them all are on FuckedCompany.com.
It's a nice analysis done there but it means little to associations like the RIAA since
Try thinking about a wedding reception, do you see them playing mp3's at the ball? I highly doubt it. Do you think radio stations across the world would adapt to mp3's? Searching for a hit song on Napster or something similar? Nope won't happen, as long as record execs keep people happy with graphical cd jewel cases, promoting overblown parties, etc. Don't wanna make this long, but any argument any way you cut it would never be an unbiased one.
Want Root?
I threw together two scripts to compile the entire database into a CSV file (which may be imported into MS Excel or anything else that likes CSV.)
The first script, riaa_pull.pl uses LWP to pull all the data in it's original HTML format into a file called riaa.htm on your harddrive. Beware, this is 30991603 bytes worth of data.
The second script, riaa_parse.pl parses riaa.htm data for relavant info and creates the CSV file riaa.csv.
The riaa.csv file it created for me was 1596076 bytes and appears to contain the entire 16474 entries.
Scripts are here: http://g27.sourceforge.net/files
numb
In order for Johnson's "For everybody else" advice to succeed, we are missing a crucial element: a place for people to be exposed to music.
Sure, there's MP3.com, which I think is quite successful at introducing people to some new songs, but what is really needed are powerful independent webcasters, and the technology to give them an audience. However big net radio is today, it won't come into its own until the wireless networks allow you to listen to it in your car, or while jogging.
If wireless moves quickly enough, it's possible that non-industry people could even end up dominating this new "digital radio" market before their competitors (old school radio) ever wake up and do something about it. After all, a 128kbps stream is of far higher quality than conventional radio, and though webcasters pay for bandwidth, they don't have the enormous initial costs of a radio station, transmitter, and licenses.
What would this kind of movement do for music? Most probably, it would mean that in any city in the world, you could choose from several thousand radio stations, rather than a dozen. There would be stations that would cater to subgenres, playing music never before heard on commercial radio. And it would be a panacea for artists outside the system, who attract the attention of a DJ with a few thousand listeners, whose attention spreads the music from there.
Even more beneficial, the digital nature makes it easy to push the names of songs and artists to the listener, and facilitates purchasing: a song stream could easily embed an URL.
In short, grassroots digital radio is a promotional path for artists who are outside the RIAA system, and have no wish to become a part of it. It's also a godsend to the thousands of RIAA groups who are too small to be worth pushing on radio, whose talents languish because people are never exposed to them. Contrary to Clear Channel's opinion, some people really do like to listen to different music: even the good stuff gets boring after a while.
The major bar to this wonderful idea is that the RIAA has such a stranglehold on the market that it will be exceedingly difficult for independent operators to build up the critical mass of music that will keep the public listening. It wouldn't be such a big deal if the RIAA would allow webcasters reasonable access to music, but we all know that short of congressional action this is not going to happen. Perhaps the creation of a independently controlled rights management website would help: artists would indicate in some form the rights they give to webcasters, and perhaps the royalties they desire. If such a thing got big enough, even RIAA artists might demand the right to webcast their music in their contracts.
Competing with the RIAA/ClearChannel will be exceedingly hard. But hopefully that critical mass of webcasters, artists, and (!) lawyers is out there. I hope so.
My job is doing analysis using variations on Kohonen Self-organizing-maps and other unsupervised classifications techniques. We do classification and data-mining all many different types of data. I read this article and immediately thought I might be able to use the data to try and find some of the correlations mentioned in the article. Unfortunately, the data doesn't contain enough information. The only files that I saw had some magical number which is the the sum of each product platinum times the number of years since the album came out. The unsupervised learning algorithms should determine the calculated numbers, not a person. Where are the original numbers?
If the evergreen project wants any serious analysis to be done with this data, then they need to include the raw, initial data. This would be the year of each platinum album, the year of the record release, and any other relevant or irrelevant data. Please re-release these files with the original data so that a thourough analysis can be done.
I'd be happy to test out a few algorithms on the data.
Keeping
- Of the bands/albums listed there as "important", none are all that important any more. The success of Beatles "1" aside, hardly any of those artists have released what I would call a "hit" album in the past 10 years.
- No band will ever "come out of the blue" any more and hit it huge. There is no such thing as a non-manufactured band anymore. U2 is currently in the midst of a sold-out tour, but if they were starting today, it's likely they wouldn't exist. They owe their early success to a core groups of fans and a bunch of radio stations that were willing to play their music. Those stations are now essentially run by the RIAA. See these articles:1,2. Bands today such as The Dave Mathews Band are the rare exceptions.
- The RIAA's business model is focused around promoting a small group of artists through the radio and other media which they mostly control. Having "free" music available over the internet breaks this model. The RIAA companies will no longer be able to promote the artists they want to promote (ie, the ones that they feel give them the greatest chance of profit).
- The RIAA's biggest fear is that artists will be able to promote themselves and leave the RIAA out of the loop entirely. It's only a matter of time before this really starts to happen. Good riddance.
-S
--- What parts of "shall make no law", "shall not be infringed", and "shall not be violated" don't you understand?
The ranking index the author uses to measure long-term selling ability (millions sold x years aon the market) would weight an album that sold 1 million copies 5 years ago (and none in subsequent years) the same in long-term selling power as one that sold 2.5 million copies last year, 2.5 million copies this year, and will go on selling 2.5 million copies every year from now. Whether you agree with the conclusion or not, this is shoddy analysis.