The Perception of 'Random' on the iPod
Robaato writes "Stephen Levy writes in the Guardian about the perception of randomness, or the lack thereof, on an iPod set to shuffle." From the article: "My first iPod loved Steely Dan. So do I. But not as much as my iPod did.... I didn't keep track of every song that played every time I shuffled my tunes, but after a while I would keep a sharp ear out for what I came to call the LTBSD (Length of Time Before Steely Dan) Factor. The LTBSD Factor was always perplexingly short." My first iPod shuffle refused to let me delete (sigh) Weird Al's Polkamon off of the flash memory.
Is the iPod Shuffle Playing Favorites?
How about an analysis of the randomness of Zonk dupes. I guess I should be happy it's not a games story.
http://en.wikipedia.org/wiki/Confirmation_bias
You know, instead of wasting your interviewee's time, you could have installed a five song list on your iPod and set it to shuffle. You'd have to carefully mark down the track number being played and listen to it for 100 songs. Do this a few times and make sure you're very methodical about what you do. Wipe the iPod, put five songs on it in order and then listen to a hundred songs "randomly." If you start to see a pattern developing or one song is obviously favored over the other, it will begin to show up.
But on the more technical side, they have to seed the random variable with something. Whether or not it's an internal clock, I'm not sure. Either way, they have to derive a random number and it's possible that their seed isn't good enough or has too few states or is prone to being seeded at the same state, etc. Based on this information, I hate to break it to you but it is very hard to be truly random.
My work here is dung.
My iPod likes Slayer and Marilyn Manson. I guess its posessed.
...well, used to. Then I made a different playlist and labeled it as "Upbeat" music. Now It loves Motion City Soundtrack. Now, I like all the music I have on my iPod (duh) but its very noticeable when I hear the same song 3 times in one hour 30 minutes worth of driving. I can tell you that over the last 2 days (4 hours of driving) I have heard Foo Fighters' "Enough Space" 6 times out of 231 songs. Does the iPod sense higher played songs/albums/groups or is its randomness just that awful? 2GB Nano 1g
I recall The Steve personally addressing this issue in one of his keynotes (although not with iPods, but iTunes). People thought iTunes' shuffle feature wasn't random enough. Steve assured everyone that it indeed was completely random, but then announced that iTunes had a new "Smart Shuffle" option. The description in iTunes is "Smart shuffle allows you to control how likely you are to hear multiple songs in a row by the same artist or from the same album." There's a slider with "More Likely" on one end, "Less likely" on the other end, and "Random" right in the middle. Although this feature is in iTunes, it has not yet made it onto iPods.
I personally have had it happen where my iPod is in shuffle mode and I've heard not just two songs in a row by the same artist, but a song plays and then the next song from that album follows it. And that's with a library of over 5,000 songs. Naturally it's more likely to happen on a much smaller Shuffle with a fraction of the songs.
Duct tape is like the Force. It has a light side, a dark side, and it holds the universe together.
I agree that it is extremly difficlut to be truly random. There are some good ways of initializing the seed in such a way that the pseudo random number generator behaves differently. A good way, done in telecom terminals is to measure the noise at the receiver and then use it to seed the random number generator. Since, the noise is truly random, that is a good way to seed the random number generator. If the costs, are not too high, then it might even be a good idea to read noise (or any truly random parameter) whenever required. That would be close to really random, provided, we can map the random parameter into a quantitative parameter without big errors and approximations.
Humans innately seek patterns in things that are random. That's why so many people wear smelly socks because they think they're lucky. Once you identify a supposed "pattern" i.e. non-randomness, you're going to keep noticing instances that fit that pattern, and ignore instances that do not. This is deeply ingrained.
Think about it, if you're at the roulette table and black has come up four times in a row, how likely are you to bet black? Most people would bet red, because, I mean hey, there's got to be a pattern. But (as I'm sure you all can understand) black has the same probability of occurring again as red does.
People have had this complaint about all sorts of playlist randomizers (not just iPod), it's just people seeing what isn't there.
Apple should add another random play mode -- one that acts as it does now, and the other mode that grants every song an equal play count. The only thing that would be random is which order. This way users that have a confirmation bias of their iPod favoring certain songs can no longer be paranoid of Apple conspiracies to promote the songs of {{ artist }} or {{ record_label }}.
I have one but only ever use it on long journeys and no I don't have DRM'd tracks so I didn't care about online music purchases. The ipod just happened to be the one that worked the best (scrollwheel is nice and quick) and having a mac I knew it'd work well.
It's a shame how people on Slashdot aren't allowed to just like iPods -- they always feel pressured to justify the purchase.
"Best tool for the job" isn't good enough. You have to be different. But only in a pro-Linux anti-iPod sort of way. Any other kind of being different gets you modded troll or flamebait.
-- I'm old enough to have lived through six different meanings of the word "hacker."
Not sure its so much confirmation bias (alone at least), as it is that the odds of NOT playing a song from the same artists over the next X songs shrinks more rapidly than intuition suggests. That is, for example the odds of NOT having a run of X heads or Y tails when flipping Z coins is very, very small.
The article mentions the "how many people does it take to get to a shared birthday thing" - and the point there is that its not that it takes 40 people to get to one with a SPECIFIC birthday but only 40 or so to find two that SHARE a birthday.
-----
graphically speaking
graphically speaking
The open-source music player I wrote (BSoftPlayer) has a feature called "SmartShuffle". One of the biggest problems with shuffle is that it's difficult to understand when the tracks will change order, and it's difficult to know wheter or not a track is going to be played more than once in a single "cycle". Some shuffle features will play the same track twice before playing through your entire library, and some won't.
With SmartShuffle, the order is randomized, but it remains the same until you "reshuffle".
What, we're losing Anonymous Coward? He's been here since the begining, and he wrote half the posts! OH NOES!!!!1!
"Nobody owns the fucking words man." - James Dean
You can get the technical details here: http://en.wikipedia.org/wiki/Birthday_paradox/
The basic gist is that their are far more possible pairs than we'd intuitively imagine. For example, with 20 albums of 20 songs each, the chance of two songs in a row being from the same album is actually:
400/400 * 20/400 = 1/20
Which makes a lot of sense once you sit down and think about it, but is a lot higher than an uneducated guess.
This is the same reason that collision/timing attacks are feasible.
FTFA:Or
Duct tape is like the Force. It has a light side, a dark side, and it holds the universe together.
I think (and have... thunk?) that "randomness" on the ipod is actually a secret R&D weapon in the apple ipod toolkit. From a psychological standpoint alone, what is the value of all other mp3 players being truly (read unadjusted psuedo random) and the ipod being a little less.. that is, what if they, say, mark the number of times you don't let a song play through, but skip it in the first 10 seconds? There are powerful means by which they can onboard build a profile and i have three things to say about that:
1) that is a FUN project for a team of engineers to do and,
2) Why wouldn't they for the HUGE hidden psycological impact it could have in differentiating the player
3) It's closed source so you can't actually tell, so the five songs with-no-user-input model wouldn't work. Another might...
Regardless, i wouldn't expect them to miss the importance such a feature would have. The iPod just keeps the vibe going, while the competition keeps playing country-house-ambient-country-house-ambient
Also, the "sound-check" would be a good place to do some quick BPM detection to have like tempo's play. The new settings for more- or less-random in iTunes almost scream "we are doing something tricky"
Wouldn't you, if you could?
CS majors know the time/space tradeoff, but they never get taught the 3rd, crucial, tradeoff of the set: comprehension!
A long time ago I was dissatisfied by the lack of random in XMMS so I jumped to the source to see what I could do. I think this was my first contribution to a free software project. Anyway, here is what I found: XMMS keeps two copies of the playlist, one that is in the order you set and one that is "shuffled". This has to be clear, all the tracks in the play list are there exactly once in the shuffled playlist.
I can't recall when the shuffled playlist was reshuffled but in was not that often, maybe only when you added or removed tracks. So if you like Smoke on Water but that Ballroom Blitz is just two song after that, too bad, you'll always get Ballroom Blitz soon after you double click on Smoke on Water. Technically speaking, the shuffling was perfect, the random generator was properly seeded and they divided in the right way to prevent loosing entropy. The lack of reshuffling was entirely responsible to the perceived lack of randomness.
So my patch was just that: trigger reshuffling a lot more often. As far as I know this patch was never merged but my copy of XMMS did have the proper random behavior. I don't know if it's the same problem with the iPod. That's something I like with free software: you can fix it!
(I looked at mine; it was closer to uniform than I'd perceived. There's also a "Skip Count", but it's blank for all my songs.)
Stupid job ads, weird spam, occasional insight at
I know there will be snarky +5 Funny comments underneath this (as well there should be), but this system to decrease the perceived randomness is actually really similar to the algorithm most radio stations use when programming their music.
There's a simple parameter that's set to control, to within one minute, the amount of temporal separation there must be between playing two songs from the same artist, or the same song twice. The radio algorithm is a little more complicated, since songs aren't in just one big batch like the iTunes library, but in different categories, based generally on the perceived desire of target listeners to hear a given new song, or like and identify with a given older song.
The system is built off the (once literal, now metaphorical) use of index cards: The format clocks say, e.g., at the top of the hour, play a P category song, followed by a B category song, then a G, then an A, etc. You'd have a set of rules, like "don't play the same artist within 45 minutes" or "don't play the same current song within 3 hours", and you'd take the first card in the category that fit all the rules, play it, and move the card to the back of the stack.
Basically, what Apple is doing with that slider is enabling artist separation control, which is completely one of the illusions radio stations (used to) use to convince you they had every song under the sun available to them.
Actually it does. There's a counter for the number of times a song has been played through completely. I believe one of the in-built playlists accesses this metadata.
Mind you, as to wether the device uses this information to weight its shuffle function is something I have no idea about.
Maybe iPods are showing us their souls. The inner light is shining through.
How to Download YouTube Videos
Except for mathematicians and programmers, most think of "random" in a *very* different way from its technical definition. To most humans, saying that a particular sequence is "random" means *guaranteeing* certain things about it. Among them: the same element does not occur back-to-back, EVER, even if there are only a few elements total to choose from. Even more, if there are more than about half a dozen elements, the same element never occurs twice within about five positions. (So if you've got songs 1 through 7 on your iPod, and the first seven played are 5, 3, 7, 2, 4, 1, 6, then the next one has to be 5 or 3, or _maybe_ 7, or it doesn't seem "random" to most people. Yet, the order can't be the same every time through, either.) No element occurs substantially more often than any other element, even over the short term. If the elements have a natural order (e.g., alphabetical), then no three elements that are adjascent in that order can ever occur together in that order, nor should they occur together in the reverse order. (This gets particularly difficult to guarantee when the elements have more than one natural order, e.g., if the elements are people, you can't have three of them in a row by either name or age, or people notice and decide that the order is not random.) Even worse, if the elements can all be categorized into a small number of categories (e.g., by gender), you can't have "too many" from one category in a row. (How many is too many depends on the ratio, but if half of the elements are male and half female, having four of either in a row will make people cry foul, the order is not "random".) If certain elements stand out from the others in some significant way, they can neither occur first nor last. (For instance, if test questions are being drawn from a question bank, neither the easiest nor the hardest question should be first or last; if it is, people will say the order was not random.)
I could go on and on, but what it really amounts to is that when most people say "random" they mean "carefully arranged in a thoroughly mixed-up order". This is almost the *opposite* of what a mathematician or computer programmer thinks the word "random" means.
For this reason, when describing a mathematically-random sequence to an end user, I never EVER use the word "random". I generally call it something like "arbitrary" or "unpredictable". This greatly reduces complaints.
Now, as far as song frequency, I like to rate my tracks on a scale of 1-10, and rig my playlist so that anything under a 6 never plays unless I specifically select it, tracks rated 7 play twice as often as those rated 6, and the frequency keeps going up the higher my rating is. (I only have eight tracks rated as a 10, and they're all things I don't mind hearing back-to-back.) Then if I find a track is playing more often than I like, I figure I rated it too high and cut back its rating.
Cut that out, or I will ship you to Norilsk in a box.
And the question remains, why doesnt the DUPE tag work anymore? I liked that tag. Seeing it meant I could avoid the 500 "OMG! Its a Dupe!11!" comments.
In Soviet Russia the insensitive clod is YOU!
too important to be left to chance."
It's a psychological thing. Like walking by a streetlight and having it go on or off. If it happens again within an hour or so, you're SURE it had something to do with you. Even if it's the same streetlight. Even if you know when the bulbs get old, they overheat and cycle on and off. Every week or so I hear two Bob Dylan tunes back to back on my iPod. Not too surprising, I only have about 60 tunes on there and Senor Zimmerman "sings" four of them.
Almost two years ago.
Then they changed their form so they no longer accept feature requests for iTunes, only iPods. As for my request, iTunes 6 doesn't remember where I was in a playlist after closing the program, does version 7?
It only took Apple three or four years to incrementally improve their Shuffle feature. I'm sure I just need to wait another year or two for my request to get implemented.
Maybe in another two or three years enough people will have asked Steve Jobs to get the Shuffle feature to play songs sooner that haven't played in a while. Now that version 7 (are the bugs fixed yet?) notes when a track was skipped, maybe version 8 will actually do this.