Slashdot Mirror


Exploiting Network Captures For Truer Randomness

First time accepted submitter ronaldm writes "As a composer who uses computers for anything and everything from engraving to live performance projects, it's periodically of some concern that computers do exactly what they're supposed to do — what they're told. Introducing imperfections into music to make it sound more 'natural' is nothing new: yet it still troubles me that picking up random data from /dev/random to do this is well, cheating. It's not random. It bugs me. So, short of bringing in and using an atomic source, here's a way to embrace natural randomness — and bring your packet captures to life!"

24 of 189 comments (clear)

  1. Random by somersault · · Score: 4, Insightful

    The imperfections in music aren't perfectly random either, so what's the big deal?

    --
    which is totally what she said
    1. Re:Random by PopeRatzo · · Score: 4, Insightful

      The imperfections in music aren't perfectly random either, so what's the big deal?

      Most insightful comment on this story. Period.

      Even if we could get perfect randomness in our art, it wouldn't really matter because the humans who see it or hear it will just try to impose some order on that randomness. It's what we do.

      Instead of randomness, what I seek to add to my sounds in the music I make is complexity. That's what makes for a rich sound.

      For example, if you look at the harmonics in a struck piano string or plucked guitar or bowed violin, they appear at predictable places. Now look at the harmonics in a free reed instrument, such as a chromatic harmonica. All sorts of weird places, strange ratios. It's what gives the chromatic such a distinctive, heart-rending timbre. Listen to the album Affinity by Bill Evans and Toots Theilemans and you can see why Evans decided to record his masterwork with a "trivial" instrument like the chromatic harp. It's basically a shaped noise generator with pitch.

      Similarly, listen to the digital sound used in "Sky Saw" in Brian Eno's Another Green World album. A simple waveform made extremely complex using god knows what filthy circuitry and it feels like someone is sticking the motor from a pair of hair clippers up your butt (not that I would know what that feels like since I would never, ever do such a thing since I turned 40).

      It can be easy, or hard, using pseudo-random algorithms in MaxDSP but it's the complexity that makes the sound do it's business. Except when it's simple, like a flute which is basically a sine wave. Oh never mind. I hate thinking about this stuff. It's a waste of time and I left my days as a theorist behind me. I'll let the young guys like the lad in the article worry about how pure the randomness is in the sounds he uses. It'll keep him occupied until inspiration comes along.

      --
      You are welcome on my lawn.
    2. Re:Random by The+Askylist · · Score: 2
      And there's nothing to substitute for the variations in timing that come from humans playing the instruments.

      Whether it's Sly and Robbie or George Shearing (sorry new music fans - I really don't like most modern crap), it's that slight movement ahead or behind the beat, and the control of it that adds emotion and a certain thrill to a tune.

      On the harmonica - give me Larry Adler any day :-P

  2. How is that more random? by AuMatar · · Score: 2

    The vast majority of traffic is either html or email. Very structured data. It's sufficiently random to use for a video game or the like, but it's definitely not random from a cryptography point of view. So you're doing things the hard way with no discernible benefit. Total waste of time.

    --
    I still have more fans than freaks. WTF is wrong with you people?
  3. What the cluck? by YodasEvilTwin · · Score: 5, Insightful

    Network captures do not embody "natural randomness". Packets are produced by computers too, not by the entropy of the universe or something. This guy has toked a little too much ganja. They're probably not even as random as a regular pseudorandom number generator. The latter makes some guarantees with regards to what you'll get out and ensures that no basic patterns are present. Network captures don't have these features. Depending on the computer, the network, and so on the incoming packets may be quite deliberate and ordered.

  4. /dev/random by Anonymous Coward · · Score: 3, Informative

    This seems like a fairly lame variant of the environmental entropy gathering which *is* what /dev/random does...

  5. I thought /dev/random already looked for entropy.. by tiffany352 · · Score: 2

    I recall reading that /dev/random will pull from any system modules that are capable of being noisey. Like radios or network equipment. It would make sense too. Also, network packets are not a very good source of entropy. Atmospheric noise from a radio is. Network packets have structured data being sent through them, often in the form of english text.

  6. Why not atomic? by nten · · Score: 2

    The lavarnd.org folks have all the source you need and a reference implementation that literally is webcam stuffed in a dark can. When you can get such high quality entropy for less than US $30, it seems like anything else must just be for fun. Some opaque tape over the camera on many laptops should work fine too.

    --
    refactor the law, its bloated, confusing and unmaintainable.
  7. not nearly as "random" as /dev/random by Dahamma · · Score: 5, Informative

    /dev/random on most OS'ed these days uses an entropy pool generated from a bunch of different sources - timing of keystrokes, mouse movements, disk seeking - and yes, network information. Then it uses cryptographic hashes on those.

    Your implementation basically uses one of those entropy sources, and then doesn't even hash it...

    1. Re:not nearly as "random" as /dev/random by gman003 · · Score: 5, Insightful

      In brief:

      "The generation of random numbers is too important to be left to chance."

      Anyone trying to create a new random number generator with the intent of producing more random numbers, without an extensive and specialized education, is guaranteed to fail.

  8. Confusion... by Junta · · Score: 3, Insightful

    /dev/random is about as random as you'll get. I presume your issue is that the pool is exhausted for the given desire. /dev/urandom is your endless of supply of 'good-enough' random for something like this. If your criticism is that it isn't really 'random', it's no less random than your pcap stream. Besides, given the application 'true' randomness will not be distinguishable from good pseudo-random.

    If you wanted to be random and artistic, then maybe point a webcam at a fireplace or something as an entropy source.

    --
    XML is like violence. If it doesn't solve the problem, use more.
    1. Re:Confusion... by emj · · Score: 2

      Then, I'm XORing the five sources together, which produces a stream with entropy good enough to satisfy my randomness needs.

      I tell you all that you do not want to read about parents "needs". Disturbing!

    2. Re:Confusion... by gman003 · · Score: 2

      Or just grab a computer with a VIA Nano CPU - they have a built-in true random-number generator, based on thermal and/or electric variances inside the processor. They claim "up to 1600 kilobits per second", so it should provide more than enough for music, provided you aren't adding bit-for-bit random noise in real time.

    3. Re:Confusion... by ronaldm · · Score: 3, Informative

      I'm going to reply to just the one poster, as explaining this to each /.'r would take rather a long time! :)

      First and foremost, Slashdot (as you know) unfortunately chooses the URL for your particular story. "Truer[sic] Randomness" is not in fact what I'm going out to somehow magically solve (with my absolute non-background in cryptography etc.). As to why they chose to enter the title of the story as such - I don't know. A bit of sensationalism, perhaps? In any case, I'd originally titled this "Musical Network Captures" - no more, no less!

      Why not use /dev/random? It's not random per se which is required - it's a pseudo-random source which can still be directly influenced by those in the immediate environment. For installation purposes, for example, it's a quick (unorthodox, convoluted, fucked-up, whatever-you-wanna-call it) way of generating an input that can be used to further modulate other inputs/sounds. For this, a stream of random numbers alone is not good enough. There's a million and one ways this could have been done - it just so happens that this is how I decided I'd go about doing it.

      Finally, I'm a bit overwhelmed by the whole Slashdot thing. As it's all worked out, it seems that on my first 'go' at adding something that I thought a couple of people would be interested in, it's seemed to hit the front page - and so at least, next time, I'll know to not be so trigger-happy when I'm 'submitting' something to here! Apologies for all of you who seem to have fallen out with one another and spent half your time bickering over nothing.

      --R

    4. Re:Confusion... by Chelloveck · · Score: 2

      Sorry that you feel like your corn flakes have been pissed in, but you can't go blaming this on the editor's bad choice of headlines. Your own submission says "[...] yet it still troubles me that picking up random data from /dev/random to do this is well, cheating. It's not random. It bugs me." Then you go on to describe a mechanism that's far, far less random than /dev/random or any halfway decent pseudo-random number generator.

      Your blog post doesn't actually try to say that the network captures are random, just that they're a good source of variation for this purpose. I, and probably the majority of the Slasdot crowd, have to complaint with that. It may even be true that the *non* random qualities of the network traffic make it a more pleasing source of noise than a truly random source.

      But that's not what the Slashdot submission says. It baldly states that you don't think /dev/random is random enough, and that "short of bringing in and using an atomic source" the network captures are a good source of randomness. Okay, that doesn't exactly say that you think network captures are more random than /dev/random, but it sure the heck implies it.

      And that's what's raised our collective hackles. If your submission had read, "Hey, here's an interesting source of noise to make computer-generated music more natural-sounding." I don't think anyone would have complained. It is interesting, especially if it turns out that it produces a more pleasing result than truly random noise. The complaint is about your use of the term "random", which has a strict mathematical definition that is not very well satisfied by network traffic.

      So, I'm sorry we beat you up over the summary rather than the actual article, but this is Slashdot. The summary had better be good, because no one ever reads the article.

      --
      Chelloveck
      I give up on debugging. From now on, SIGSEGV is a feature.
  9. Mod parent up by impaledsunset · · Score: 3, Interesting

    /dev/random is already gathering environmental entropy from hardware sources and (except if you're running it on a virtual machine), it should produce data with good entropy that's truly random and is not comping from a pseudo RNG algorithm.

    Now, of course, if you XOR it with the network data you might increase entropy, but if it happens that /dev/random already uses it, you're not gaining anything, or in fact make things worse.

    But, please, if you think that /dev/random isn't providing data that's random enough, suggestions and patches would be welcome. Even if they don't get accepted in the mainline kernel, you can still distribute them.

    Another issue: I'd encrypt the data from the network source or XOR it with a pseudo RNG, because otherwise you might be leaking sensitive data through your "random" numbers.

  10. Re:If I would by JabberWokky · · Score: 5, Informative

    Actually, many people would sell you the answer. And they don't have nobel-prices[sic].

    See http://en.wikipedia.org/wiki/Hardware_random_number_generator for an overview of the devices you're looking for.

    --
    "$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
  11. Re:What universe does this guy live in? by Em+Adespoton · · Score: 2

    Are you sure about those two statements?

    However, you do have a point... while computer theoretical models do exactly what they're told to do, as soon as you introduce a physical implementation, the computer will do whatever its environment tells it to do -- this is not always the same thing as what the computer operator tells it to do.

    Similarly, the ocean does exactly as it is told to do... of course, this interaction is so complex, that a mere human being would be unable to untangle all of the instructions given to the ocean by various external influences.

    Also, I'd argue that it is highly possible for a multiorganism as large as an ocean to have sentience... maybe it's just hiding this fact from us mere humans because it's smart enough to know that's a good move... maybe it's even smarter than that, and doesn't think us lower lifeforms have anything more to contribute than we would consider carrying on a conversation or discussing ethics with a top quark.

  12. Re:If I would by maxwell+demon · · Score: 2

    Those using quantum effects cannot be predicted even if you had a device to monitor the complete surroundings.

    --
    The Tao of math: The numbers you can count are not the real numbers.
  13. Re:Randomness is not an objective thing by FrangoAssado · · Score: 2

    Distinguishing between "false" and "true" randomness is pointless.

    Not really, it's done all the time for many different purposes.

    Take, for example, how computer scientists define it: roughly, a sequence is random if it can't be compressed, that is, any (program+data) that generates it must be at least as large than the sequence itself. It distinguishes between "random" and "not having enough information to predict it": it doesn't matter if it looks random to YOU; if it could in principle be compressed, it's not random.

    That's not pointless hair splitting, it has real consequences for many areas of computer science, some very practical (cryptography, for example).

  14. Re:Randomness is not an objective thing by FrangoAssado · · Score: 2

    It's not my theory; maybe you heard about a guy named Kolmogorov that lived in the last century? I bet the great mind of Robert Coveyou studied a lot of his theory :).

    But, more seriously, of course a random source will output compressible data sometimes. What happens is this: as you collect more output from a truly random source, the probability of it being compressible goes to zero very fast.

    But the point is that it *is* useful to distinguish between "false" and "true" randomness, otherwise it wouldn't be true that "the generation of random numbers is too important to be left to chance".

  15. Re:Random is as random does by dotancohen · · Score: 2

    http://www.random.org/faq/

    Q2.1: How can you be sure the numbers are really random?

    Oddly enough, it is theoretically impossible to prove that a random number generator is really random.

    http://dilbert.com/strips/comic/2001-10-25/

    --
    It is dangerous to be right when the government is wrong.
  16. Better ways to do random by joe_cot · · Score: 2
    As a number of commenters have pointed out, /dev/random is actually way more random than what this article suggests doing. If you want stuff that actually is more random, or need a lot more random data, here are some options.
    • Random.org provides random data generated by radio noise. You can get as much random data as you'd like. Gaming websites download their random data in 5MB chunks to use for card shuffles and dice rolls.
    • HotBits is a similar idea, but uses radioactive decay instead of radio waves
    • If you want to do it in house, you can do so with a smoke detector and a webcam. This was submitted to slashdot in 2006
    • Finally, if you need a ton of random numbers, and they must be random, you can buy RNG hardware

    What do i do? if I don't really care if it's random, I use the RPG from the programming language I'm using, or /dev/random. If I really, really care that it's random, I download a chunk of data off random.org, and either use that for the numbers, or use it to seed my RNG. For the most part, anything more than that is overkill.

  17. Re:I thought /dev/random already looked for entrop by gweihir · · Score: 2

    It does. An a simple "man 4 random" will give you that information. It seems the OP could not even be bothered to do that before posting his clueless BS.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.