Slashdot Mirror


Managing Last.FM's "Mountain of Data"

Rob Spengler writes "Last.FM co-founder Richard Jones says the biggest asset the company owns is 'hundreds of terabytes of user data.' Jones adds, '... playing with that data is one of the most fun things about working at the company.' Last.FM, for those who have been living on Mars for the last two years, is the largest online radio outlet, with millions of listeners per day. The company surpassed Pandora and others largely due to its unique datamining features: 'Audioscrobbler,' the company's song/artist naming algorithm, can correctly determine a track even with tens of thousands of false entries. Jones says sitting on that much data has even helped police: 'thieves listening to music on an Audioscrobbler-powered media player have helped police in the US, UK, and other countries track down users' stolen laptops.' Does sitting on a mountain of data make Last.FM powerful enough to start making a stand against the record industry? CBS certainly thinks so — they bought the company for £140 (~$200) million last year."

35 of 139 comments (clear)

  1. Data is valuable by DNS-and-BIND · · Score: 4, Interesting

    A buddy of mine used to run this matching website for teachers & students. Free for teachers, and the students had to pay a nominal amount to get the teachers' contact info, and after that, it was up to them to arrange for lessons. The site was popular, and he made decent money at it. I bugged him and bugged him to organize parties, and eventually he came around to my way of thinking (he wanted to make some money without his parasite partner getting it). He used the list of emails from his website to send party invitations for a monthly get-together. He made more money from the parties than he did from the website.

    --
    Shutting down free speech with violence isn't fighting fascism. It IS fascism!
    1. Re:Data is valuable by dword · · Score: 3, Insightful

      How I see it: there are people with tons of money. Literally, tons. You can't use only money to make more money - no matter what you do with it, it just won't multiply sitting it's ass on the couch all day, watching TV or in a safe somewhere. So what do you need to give that money more value? The answer is simple: information. The only way to make money multiply is if know what to do with it. You can write the best software in the world, the best OS with the best tools ever, but if you don't know how to make it popular, it will never become popular on it's own. The only way to make it popular is to give people as much information about it as possible. Why do we have ads? To send people information about products. Sure, almost every ad is misleading and they give you fake information, but they do tell you something, which you take into account when you make decisions and you are more likely to buy an advertised product instead of an obscure "noname" (I was cheap enough, often enough, to buy "noname" computer-related products and I was amazed at their quality and I wish someone told me they exist so I wouldn't feel so bad and cheap before buying them).

      This is the age of communication and nothing is more valuable than information and manipulating that information. How do you manipulate it? To know that, you need another kind of information, which is usually based on statistics on large amounts of data (like Last.FM's database, for example).

      So, in today's society, there are three valuable entities: money (manipulated by information, everyone wants it), information (manipulated by more information, any company's dream) and more information (based on statistics, like the Last.FM database) controlling each other in a cascade. Once you have the source you can easily trace it to see how things are flowing, so you may know how to invest your money.

      Repeat after me: "I will not disclose the information I have. Information is more valuable than money. If I own a valuable piece of information and I don't make money off it, I'm stupid."

    2. Re:Data is valuable by Dr_Banzai · · Score: 5, Funny

      I think we have enough teacher-student sex scandals without a matchmaking web site!

    3. Re:Data is valuable by Lord+Kano · · Score: 2, Funny

      I disagree. When I was a senior in HS, we had a smoking hot student teacher. I would have paid to get molested by her.

      LK

      --
      "Hi. This is my friend, Jack Shit, and you don't know him." - Lord Kano
    4. Re:Data is valuable by 4D6963 · · Score: 2, Insightful

      Good points. You had me until you said "entity" (do you know what that means? I doubt it) in the place of, I assume, "commodity".

      Oh and the repeat after me bit is silly. The "information" you have is worthless on its own. It only becomes valuable when it's coupled with lots of other similar "informations" from other people. By retaining this information you're only preventing someone from making money, without any benefit for yourself, which is arguably dickish. Oh and saying that "information is more valuable than money" is stupid. You can't say that something is superior to what measures it.

      --
      You just got troll'd!
    5. Re:Data is valuable by toddestan · · Score: 3, Insightful

      Sounds like a slight variation on those people who have TB's of movies/music/videos/TV episodes/etc that they will never have the time to watch/listen to.

    6. Re:Data is valuable by Jah-Wren+Ryel · · Score: 2, Interesting

      and tell me mr Coward what have you deducted from you pile of information

      So what if he has never done one useful thing with it? People like that provide a public service, its people like that which enabled DejaNews and now Google Groups to reconstruct much of the historical usenet. If his hobby is data hording, then let him horde. It doesn't cost you a dime, but one day it might possibly be of great benefit.

      --
      When information is power, privacy is freedom.
  2. unique order of songs by Blue+Shifted · · Score: 5, Interesting

    what i find most interesting is the order certain songs "go together", like listening to a song from Slayer, followed by, say, "someday i suppose" from the bosstones. when composing songlists, i appreciate how similar songs and moods can flow, but also how the contrast of dissimilar songs can SOMETIMES compliment each other.

    a large database could ferret out such instances that might occur frequently in multiple playlists.

    1. Re:unique order of songs by dword · · Score: 2, Funny

      Excellent point. Think of the impact this could have on psychology!

      To get you into the right mood, think of the impact it could have on mind manipulation ;) Tinfoil hats for sale! Get your tinfoil hats here!

    2. Re:unique order of songs by Nova77 · · Score: 2, Interesting

      What about "songs are mostly played in alphabetical order"? :)

    3. Re:unique order of songs by adolf · · Score: 2, Insightful

      So your contribution, then, is noise.

      But this noise does not affect the signal, which is still there. It's just harder to find.

      Nobody ever said mining a mountain of data like this would be a trivial task.

  3. Now What... by clinko · · Score: 2, Interesting

    I have a similar site that I wrote (pre-audioscrobbler). Granted it's crap, but I have mountains of data also. Closer to 1 tb than hundreds of tb. The question is, how do you monetize the data?

    I just don't see how this data is "worth" 200 million bucks. I have some amazing algorithms to do similar cleaning, caching, and recommendations, but still what is that worth?

    This is a fairly legit question. If you can figure it out, I can explain to my wife why I have 3 servers in my closet.

    1. Re:Now What... by Mad+Merlin · · Score: 4, Insightful

      I have a similar site that I wrote (pre-audioscrobbler). Granted it's crap, but I have mountains of data also. Closer to 1 tb than hundreds of tb. The question is, how do you monetize the data?

      If you could (accurately) answer that question, then you'd act upon the answer...

      Why do you think Google ads are Google's bread and butter as far as cashflow goes? The reason is that Google has a treasure trove of user data, probably more than anyone else, so they can really make contextual ads work. Anyone can write an ad engine, but not everyone has access to mountains and mountains of user data.

      You might be surprised at how important context is when you're trying to promote something. Say you're trying to promote an online RPG like Game!, if you took a random collection of people, probably less than 5% of them would be interested in playing, but if you can target gamers specifically, that number might jump to 50%. If you're paying for every impression, that makes a world of difference.

      So not only do you need to understand your audience, you also need to effectively target them. Now, how do you do that? Data mining of course, and the more data the better.

      Pretty much all data has value, figuring out how to turn that data into money is extremely subjective and might involve some black magic, and definitely requires luck too.

    2. Re:Now What... by Mad+Marlin · · Score: 5, Funny

      Information wants to be free.

      Information wants to be a ballerina.

    3. Re:Now What... by Inner_Child · · Score: 5, Funny

      Information wants to be free.

      Information wants to be a ballerina.

      Then information needs to get her fat ass on a diet or she's never going to fit into that tutu and make Mommy proud!

      --
      Today is red jello day - all workers must eat all of their red jello. Failure to comply will result in five demerits.
    4. Re:Now What... by Cally · · Score: 2, Interesting

      Even more impressive is that the guts of the whole last.fm empire was built by a tiny team - a couple of dozen IIRC. They just fired 20% of their staff, incidentally, bringing the numbers down to... 80.

      --
      "None are more hopelessly enslaved than those who falsely believe they are free." -- Goethe
    5. Re:Now What... by cromar · · Score: 2, Funny

      figuring out how to turn that data into money ... might involve some black magic, and definitely requires luck too.

      So what you are saying is:

      1. Data
      2. ???
      3. Profit!

      :~)

  4. It's so popular... by Anonymous Coward · · Score: 5, Funny

    The summary wasn't insulting enough, so I think I'll just add a bit extra.

    Last.FM is so popular that if you aren't familiar with the service, you must be a drooling, knuckle dragging luddite.

    Apparently I'm not one of the cool kids. I'm sad now, and my feelings are hurt.

    1. Re:It's so popular... by thatskinnyguy · · Score: 2, Funny

      Last.FM is so popular that if you aren't familiar with the service, you must be a drooling, knuckle dragging luddite... a step away from churning your own butter.

      Sorry, had to add my own.

      --
      The game.
  5. all this data yet so much gets missed by form222 · · Score: 2, Insightful

    Last.fm Has all this data and yet so much gets missed. For instance: why doesn't last.fm have a feature to email you when a band you like comes out with a new album?

  6. No revolution by Jah-Wren+Ryel · · Score: 5, Interesting

    CBS certainly thinks so -- they bought the company for £140 (~$200) million last year.

    Which is why whatever comes of them, at best it will be evolutionary. CBS is part of the old guard RIAA corps, they are just one of the faces of Viacom - all controlled by Summer Redstone. They may have brought some money to the table, but they brought a whole ton of baggage with them too. Enough baggage to make this privacy freak decide they couldn't be trusted with all that data they've been collecting (for example, if they can track down a stolen laptop, they can track down someone playing an MP3 from an illegally leaked pre-release album).

    --
    When information is power, privacy is freedom.
  7. So... I've been living on Mars? by Dutch+Gun · · Score: 3, Insightful

    Last.FM, for those who have been living on Mars for the last two years, is the largest online radio outlet, with millions of listeners per day.

    You know, I'm not exactly what you'd call a Luddite, yet I've never heard of Last.FM. Am I the only one? I kind of doubt it.

    I have a general gripe about anyone who writes "for those who have been living on Mars" anytime they reference some moderately popular company, service, or product. It smacks of arrogance, as if to say, if you don't have the same interests as I do, you're obviously disconnected from the mainstream.

    Or perhaps I'm just annoyed for being called out on being a bit older and out of touch? Bah!

    >>goes back to guarding lawn with a shotgun from an old rocking chair...

    --
    Irony: Agile development has too much intertia to be abandoned now.
    1. Re:So... I've been living on Mars? by magarity · · Score: 2, Insightful

      I've never heard of them either and the bit about living on Mars also irks. And for all the arrogance in that, the summary makes it sound like the internet radio outfit needs fancy algorythms to tell what music they're playing. WTF don't they just program the correct name when they add a new song to their database? I'd read the article, but my shuttle back to Mars is leaving...

    2. Re:So... I've been living on Mars? by GreatBunzinni · · Score: 2, Informative

      Well, Amarok has a config menu entry with a big old icon with the label "last.fm" on it. Everyone who ever used Amarok had to pass his's cursor over the label "last.fm", which has been there for a few years, mind you. Other media players also support last.fm, whether through a plugin or even built in. So you may have not been living under a rock but you sure were quite a bit distracted. For at least the last 6 years or so.

      On a side note, I've made a point of turning on the last.fm plugin for a simple reason: it build popularity charts directly from the user's preferences instead of some unscientific, corrupted, payola-based sales chart. It bugged me how some artists were put on the top of the charts although no one was really listening to them. With last.fm the charts were compiled directly from the user's input and that meant that bands like Queen and Pink Floyd are still topping the charts even though they don't come near the "official", record company-compiled charts. That was very refreshing.

      But now that I've learnt that last.fm is not only tracking down contributors but also is owned by one of RIAA's record companies... Well, let's just say that the plugin is off and will never be turned on again.

      Audioscrobbler, and now last.fm, is a beautiful concept. The 200 million they got from it is more than deserved. Too bad it's being corrupted by the RIAA's companies. Maybe the sudden appearance of trash like kayne west and britney spears on the top of last.fm's charts has something to do with it.

      --
      Slashdot, fix your code or at least hire someone who is competent at it to do it for you.
    3. Re:So... I've been living on Mars? by yttrstein · · Score: 2, Insightful

      No, none of us have been living on Mars. This is just the latest permutation of viral marketing, it seems. But this one is kind of weird, because it combines all the "bleeding edge stuff" we've seen before with the oldest of old school hawking techniques, which is this:

      "IF Y'AINT SEEN THIS THEN Y'AINT SEEN NOTHIN!"

      Which is pressed and kneaded as needed to "you have to have been living (under a rock | on mars | in a laundry hamper) for the last (year | ten years | few decades | all your life) if you haven't heard of (this amazing company that can solve all your problems | this great company who has this incredible product | this stupendous chamois which can soak up over seven thousand times its own weight in water).

      Last.FM is pretty OK, but I would much rather do business with a company which doesn't have a co-founder who calls it "fun" to play with my personal data.

    4. Re:So... I've been living on Mars? by ktappe · · Score: 2, Interesting

      I've never heard of them either....I've never seen an ad about them, I've never heard them mentioned in the piles of blogs and articles I read daily, and nobody has ever recommended them to me. Pandora, meanwhile, HAS been in all of the above.

      --
      "We can categorically state we have not released man-eating badgers into the area." - UK military spokesman, July 2007
  8. Surpassing Pandora by Paaskonijn · · Score: 5, Insightful

    The company surpassed Pandora and others largely due to its unique datamining features

    I would think that being available outside of the USA may have helped quite a bit as well.

  9. The real danger by Aerynvala · · Score: 5, Interesting

    with last.fm is how it feeds my OCD issues regarding song playcounts. I nearly lost it when the stupid scrobbler started randomly recording excess playcounts on one album. It screwed with my numbers. Then it stopped counting that album's plays all together.

    Seriously though, I have found using the site to be pretty enjoyable. And the advertisements are actually worth keeping AdBlock turned off for. I found a few new artists, some unsigned, that way. I like all the various widgets and things that can crunch my data. Songbird has a last.fm plugin/addon that makes for very easy integration. It's just really useful. I've also found concerts on the site.

    I rarely use the social side of it, except with friends I already know. But that's me.

    --
    http://transformativeworks.org/
    1. Re:The real danger by DCstewieG · · Score: 2, Interesting

      Haha, if it gives you any comfort, I'm the same way. With how iTunes/iPod work - incrementing the count when the track finishes - I'm constantly waiting for songs to end before picking another one, or leaving tracks that have silence at the end to finish completely. Really wish it incremented at 75% complete or something.

    2. Re:The real danger by Paaskonijn · · Score: 2, Informative

      Yet another reason to use Amarok...

      On his iPod?

      GP was talking about the iTunes play counts, not the Last.fm play counts. Every app/plugin I've tried (including the official Last.fm app) either scrobbles at 50% or allows the user to configure the percentage. Yet another reason to be free to use whichever media player one prefers...

    3. Re:The real danger by 4D6963 · · Score: 2, Informative

      I know what you mean, with my smart playlists that keep out the songs I've played in the last 5 days I always let the songs end too. As for the silence thing just edit the properties of a song in iTunes to start/finish at the time code of your choice. It's very convenient to skip intros and such too.

      --
      You just got troll'd!
  10. Comment removed by account_deleted · · Score: 5, Funny

    Comment removed based on user account deletion

  11. surpassed Pandora ... by Lazy+Jones · · Score: 2, Informative

    The company surpassed Pandora and others largely due to its unique datamining features: 'Audioscrobbler,'

    I'd say they surpassed Pandora only because Pandora locked out all non-US users a while back. For people who just wanted to listen to music and find out about new artists, Pandora was so much better IMO, last.fm has a clunky, overloaded UI and is too much like myspace ...

    --
    "I love my job, but I hate talking to people like you" (Freddie Mercury)
  12. If Last.FM Is So Smart... by meehawl · · Score: 2, Funny

    Then why the hell is it that when I run the "Recommendations" stream the algorithm occasionally freaks out and starts pushing one unlistenable noise attack after another at me with tags like brutal death metal, cybergrind, czech, death metal, deathgrind, goregrind, grind, grindcore, noisecore, porngrind, pornogrind, etc. No matter how many times I click the "Do Not Want" button the stuff just keeps coming. It's like a neighbour from hell. And then there's the days when I get nothing but lesbian deathcore vegan grind.

    The Last.FM brainfarts seem to persist no matter how many times yoy try to train the recommendation engine using the like/ban buttons and the only way to get them to "reset" to something vaguely approximating normality is to log out, log back in, and run the Library stream for a while.

    Still, even with this weirdness it's still better than Pandora at finding new music I actually like.

    --

    Da Blog
  13. Comment removed by account_deleted · · Score: 2, Funny

    Comment removed based on user account deletion