Slashdot Mirror
Exploiting Network Captures For Truer Randomness
First time accepted submitter ronaldm writes "As a composer who uses computers for anything and everything from engraving to live performance projects, it's periodically of some concern that computers do exactly what they're supposed to do — what they're told. Introducing imperfections into music to make it sound more 'natural' is nothing new: yet it still troubles me that picking up random data from /dev/random to do this is well, cheating. It's not random. It bugs me. So, short of bringing in and using an atomic source, here's a way to embrace natural randomness — and bring your packet captures to life!"
computers do exactly what they're supposed to do — what they're told.
90% of my day job is a bunch of engineers standing around scratching our heads trying to brainstorm ways to figure out what the hell is going on with our system. We don't even know what it is doing, let along being able to tell it what to do.
http://michaelsmith.id.au
The imperfections in music aren't perfectly random either, so what's the big deal?
which is totally what she said
Can't you just ask the user to make some random mouse movements or keystrokes?
The vast majority of traffic is either html or email. Very structured data. It's sufficiently random to use for a video game or the like, but it's definitely not random from a cryptography point of view. So you're doing things the hard way with no discernible benefit. Total waste of time.
I still have more fans than freaks. WTF is wrong with you people?
shite... if i 'could' not would :(
Network captures do not embody "natural randomness". Packets are produced by computers too, not by the entropy of the universe or something. This guy has toked a little too much ganja. They're probably not even as random as a regular pseudorandom number generator. The latter makes some guarantees with regards to what you'll get out and ensures that no basic patterns are present. Network captures don't have these features. Depending on the computer, the network, and so on the incoming packets may be quite deliberate and ordered.
And "Nobel Prize," not "nobel-price."
This seems like a fairly lame variant of the environmental entropy gathering which *is* what /dev/random does...
I recall reading that /dev/random will pull from any system modules that are capable of being noisey. Like radios or network equipment.
It would make sense too.
Also, network packets are not a very good source of entropy. Atmospheric noise from a radio is.
Network packets have structured data being sent through them, often in the form of english text.
The lavarnd.org folks have all the source you need and a reference implementation that literally is webcam stuffed in a dark can. When you can get such high quality entropy for less than US $30, it seems like anything else must just be for fun. Some opaque tape over the camera on many laptops should work fine too.
refactor the law, its bloated, confusing and unmaintainable.
web cam + lava lamp is much more exciting.
Do you even lift?
These aren't the 'roids you're looking for.
/dev/random on most OS'ed these days uses an entropy pool generated from a bunch of different sources - timing of keystrokes, mouse movements, disk seeking - and yes, network information. Then it uses cryptographic hashes on those.
Your implementation basically uses one of those entropy sources, and then doesn't even hash it...
/dev/random is about as random as you'll get. I presume your issue is that the pool is exhausted for the given desire. /dev/urandom is your endless of supply of 'good-enough' random for something like this. If your criticism is that it isn't really 'random', it's no less random than your pcap stream. Besides, given the application 'true' randomness will not be distinguishable from good pseudo-random.
If you wanted to be random and artistic, then maybe point a webcam at a fireplace or something as an entropy source.
XML is like violence. If it doesn't solve the problem, use more.
/dev/random is already gathering environmental entropy from hardware sources and (except if you're running it on a virtual machine), it should produce data with good entropy that's truly random and is not comping from a pseudo RNG algorithm.
Now, of course, if you XOR it with the network data you might increase entropy, but if it happens that /dev/random already uses it, you're not gaining anything, or in fact make things worse.
But, please, if you think that /dev/random isn't providing data that's random enough, suggestions and patches would be welcome. Even if they don't get accepted in the mainline kernel, you can still distribute them.
Another issue: I'd encrypt the data from the network source or XOR it with a pseudo RNG, because otherwise you might be leaking sensitive data through your "random" numbers.
Something is random if you don't have the information to predict it. Distinguishing between "false" and "true" randomness is pointless.
Actually, many people would sell you the answer. And they don't have nobel-prices[sic].
See http://en.wikipedia.org/wiki/Hardware_random_number_generator for an overview of the devices you're looking for.
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
Sure you can, you just need dedicated hardware for it.
This kind of problem bugs only attention whores.
Yes, /dev/random is collecting randomness. It does not employ a pseudo-random number generator (/dev/urandom does so if there's not enough entropy in the pool). I don't know exactly which sources it draws from, but I guess the network is already used for randomness.But not the content, that's not random (unless you stream random data over your network, of course), but things like packet timings.
The Tao of math: The numbers you can count are not the real numbers.
So what's wrong w/ using /dev/random, at time intervals specified by /dev/random?
Wrong. You cannot get true randomness algorithmically, but every computer is connected to several input devices. For example, the noise on your sound card is exacty that: Physical noise of the analog circuit. That's about as random as any physical process can be.
The Tao of math: The numbers you can count are not the real numbers.
Still not random. If you can (and I am glad to admit this is impossible hard as far as I know) capture the 'surroundings' one on one, this is still not random enough. But still a good read and link.
The hardware consisting of 1 noise diode (yes they do exist) and one Op-Amp.
Pure randomness is easy to get. Just not in a deterministic environment like a computer. I could cobble together a source of true randomness from a smoke detector and a handful of transistors. Quantum random, the best there is. If you want lots of random, you can buy devices that plug into an expansion slot to provide it.
"As a composer who uses computers for anything and everything from engraving..."
What kind of "composer" does engraving, and why does he need a random number generator? And yeah, I read TFA, and it had nothing about applications.
Chaos maximizes locally around me.
Those using quantum effects cannot be predicted even if you had a device to monitor the complete surroundings.
The Tao of math: The numbers you can count are not the real numbers.
Still not random. If you can (and I am glad to admit this is impossible hard as far as I know) capture the 'surroundings' one on one, this is still not random enough. But still a good read and link.
'Capturing the surroundings' still won't help you do any predictions for sources with quantum randomness. At best you can say that a source would exhibit a certain behaviour x% of the time. Quantum systems are not deterministic so even with perfect state information, you can only give probabilities that certain things happen. If you know otherwise, feel free to let others know and collect your nobel prize(s).
"When you sit with a nice girl for two hours, it seems like two minutes. When you sit on a hot stove for two minutes, it
I agree on both counts. Been playing myself for 25 years and I think it's the human feel, emotion as you will, that makes the great music. Then again, I don't like autotune either, nor do I like the Biebers and Spears of this world and yet they are the most succesfull 'artists' if money is what counts (well actually U2 is, which I think happens to be a band with a reasonable singer that does have 'soul'.)
I didn't know the quamtum devies where being sold allready. But for the rest I agree with you. Quantum is the way to go forward as it seems.
So what are you saying - computer random number generators aren't good enough because they aren't 'natural'? That's a completely unsubstantiated point of view that I'd expect to hear from some hippie Arts student. They're actually tested to validate the appropriate statistical properties (run lengths, low auto-correlation, probability density, etc.) and have extremely long repetition periods. You can re-seed them anytime if you're paranoid. This is guaranteed to be as good or better than what you'll get with any of the traditional methods (numbers in a phone book, coin flipping, etc.) or anything else you can dream up.
That's rather a random task to take on as an odd job.
http://www.random.org/faq/
Q2.1: How can you be sure the numbers are really random?
Oddly enough, it is theoretically impossible to prove that a random number generator is really random. Rather, you analyse an increasing amount of numbers produced by a given generator, and depending on the results, your confidence in the generator increases (or decreases, as the case may be). This is explained in more detail on my Statistical Analysis page, which also contains two studies of the numbers generated by RANDOM.ORG, both of which concluded that the numbers are sound. In addition, the continually updated Real-Time Statistics page gives you an indication of the quality of the numbers produced over time.
You keep using that word. I do not think it means what you think it means.
You will be awarded a nobel-price.
A nobel-price? What is that, the cost of a stick of dynamite?
It could be that /dev/random ran out of entropy, and /dev/urandom just didn't sound good. Since the article is about audio, maybe it just sounds better with certain (not true random) tonal noise. Perhaps it just appears more realistic with Brownian or pinked noise.
Is is "cheating" to use PRNG when they work totally fine and computerized music that uses them can't be discerned from "natural" imperfect music. or is it cheating to use a computer to make music whether it's randomized or not?
The RdRand instruction obviates all of this, code name is "Bull Mountain" - check it out.
That explains why my packets disappear when they have too many neighbors.
make imaginary.friends COUNT=100 VISIBLE=false
What kind of clown posts these things?!
Oh... Ronald. I'm sorry dude.
Do people actually enjoy this sort of thing? I listen to Boston - Tom Sholtz made a point to put "no computers were made in the production of this album" This is how music should be: acoustic and awesome. oh and PS randomness is way easier to produce- http://www.random.org/ Mads Haar does it with atmospheric noise. In fact, my high school science fair project involved proving it is a very easy thing to reproduce for personal use.
Radioactive decay alone is enough to get true randomness. And there are several other (non-quantum) where 'capturing the surroundings' is actually considered theoretically impossible. Simple example would be Brownian motion (plus, it would give any amount of randomness desired from as small a sample as you want! Sort of anyways), which is actually what several of those techniques end up using (Brownian motion is just random thermal noise.) Technically, it is only non-random if you are able to predict the motion of billions of atoms simultaneously. That is very nearly utterly impossible.
"None can love freedom heartily, but good men; the rest love not freedom, but license." --John Milton
Just because something is quantum does not mean it is inherently special. Optical devices like photo-transistors have to deal with shot noise caused by quantum effects.
See http://en.wikipedia.org/wiki/Shot_noise#In_optics. Those kinds of effects can be leveraged for randomness.
Most of my network traffic involves downloading porn.
Your music is going to come out sounding like a strip club.
Have gnu, will travel.
Let U be the universe that you believe in, and let R source of true randomness for that universe. Then the universe that you believe in is U(R).
Let R' be one of the pseudo random algorithms that is too computationally complex for you to detect. How ever computationally advanced you are there will be an infinite number of these.
It will be impossible for you to prove that the real universe is not one of the U(R'). Occam's razor is only a human convention which prefers simplicity. It is true that the U(R') universes may be more complex and violate non-locality, but these again are human conventions adopted for simplicity not because we can exclude the U(R') with experiment.
But really you'll never achieve randomness, at least not using a traditional turning machine...
What are you talking about? I make random things with my lathe all the time!
What do i do? if I don't really care if it's random, I use the RPG from the programming language I'm using, or /dev/random. If I really, really care that it's random, I download a chunk of data off random.org, and either use that for the numbers, or use it to seed my RNG. For the most part, anything more than that is overkill.
The OP is clueless. /dev/random has full entropy and is random. /dev/urandom is the watered-down version, which still has some entropy in it.
There is no need for "better randomness". There is need for people to find out what actually exists and is implemented and use it properly.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
What utter nonsense. There is nothing to solve here and plenty of sources for true randomness available.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
It does. An a simple "man 4 random" will give you that information. It seems the OP could not even be bothered to do that before posting his clueless BS.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Still not random.
A true Nobel prize awaits your proof that quantum randomness is not truly random. We are in awe.
=~ s,(.*),<sarcasm>$1</sarcasm>,g if any_point_you_wish();
The OP is somewhat mistaken about entropy sources within a modern system. /dev/random pool and only allows that amount to be taken out. /dev/urandom pool simply allows the user to continue removing bits of data when there are no shannons of entropy left in the pool. /dev/random pool (and hence provide a way for /dev/urandom pool to also acquire more) is to have a physical source of truly random nature.
A modern OS implementation keeps a record of the number of shannons of entropy available in the
The
The only way to acquire more shannons of entropy to fill the
Such sources can include the timing information of key presses which introduce fractions of a shannon of entropy but network activity of any type is disturbingly periodic, hence such sources have largely been removed.
If you want a true random number generator you need something like an Entropy key which uses physical quantum phenomena to generate many shannons of entropy continuously.
If you monitor the complete surroundings, then you are monitoring the output of these devices. Any kind of noise that is unpredictable to your attacker will work; it doesn't need to be OMG QUANTUM!
If /dev/random has run out of entropy, so will this ridiculous hack: /dev/random already sources entropy from the packet buffers, so either there are no packets arriving and neither method works, or their are packets arriving and then you may as well stick to /dev/random
/dev/random to periodically seed a good psuedo-random generator, say a Mersenne Twister implementation. You could call it /dev/urandom.
Even better, use
Wait...
Syllable : It's an Operating System
Randomness is easy, but turning that into unbiased random numbers in the right range is a damn sight harder.
Chernobyl 'not a wildlife haven' - BBC News
If /dev/urandom could be distinguished from /dev/random by the human ear, it would be an extremely shitty excuse for a random number source.
Actually, /dev/random will never pull from network hardware. It was decided that the risk of a timing attack was too high relative to entropy gains.
The sum of the intelligence of the world is constant. The population is, of course, growing.
Indeed.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Well, he is using MacOS. If he were using a real operating system, he might have access to more authentic random numbers from /dev/random.
But what an incredibly ugly hack. If it's so damned important to you, just buy a $20 USB-FM receiver, tune into the cosmic microwave background, and record and hash the audio stream from that. Or point your webcam at a bubbles generated from a fish-tank bubbler and hash the video stream. There are so many easy ways to get moderately random numbers, that would certainly be much more random and easier to program than parsing the not-so-random stuff from tcpdump.
He probably never even bothered to query Google for natural sources of randomness, or he may have run accross LavaRnd.
Please, everyone, don't even think about taking TFA seriously.
No, the money is a testament to the success of their business/marketing/etc. managers. It has little to do with the actual "artist" or, as more would likely refer to them, performer.
I have been using the network for a source of randomness for years. Another good source is the Hard Drives internal servo coefficients, or a TV Stations video or radio station in to an Audio card. If people bug me maybe I will write that up too. On a Linux box this is simple. But bash isn't well suited for audio processing. (although it's possible) For the video in you need C code.
http://churchofbsd.blogspot.com/2011/11/generation-of-random-number-from-ping.html
I am always doing that which I can not do, in order that I may learn how to do it. - Pablo Picasso