Slashdot Mirror
Why Improbable Things Really Aren't
First time accepted submitter sixoh1 writes "Scientific American has an excellent summary of a new book 'The Improbabilty Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day' by David J. Hand. The summary offers a quick way to relate statistical math (something that's really hard to intuit) to our daily experiences with unlikely events. The simple equations here make it easier to understand that improbable things really are not so improbable, which Hand call the 'Improbability Principle:' 'How can a huge number of opportunities occur without people realizing they are there? The law of combinations, a related strand of the Improbability Principle, points the way. It says: the number of combinations of interacting elements increases exponentially with the number of elements. The 'birthday problem' is a well-known example. Now if only we could harness this to make an infinite improbability drive!"
How improbable is the Heart of Gold?
And Zaphod stealing it...
The day before Fukashima happened I was writing a paper for an Industrial Safety class on the subject of Nuclear safety. My conclusion essentially made the argument that "Although individual improbable events are unlikely, the shear number of opportunities to experience an improbable event on a day to day basis are staggering." Any specific improbable event is highly unlikely to occur, but the occurrence of improbable events in general is a practical certainty.
The next day I saw on the news that mother nature had done her best to prove my point. The timing worked out to be an incredibly unlikely coincidence, but on a daily basis I rarely notice when unlikely coincidences fail to occur. :)
My theory of the question for life, the universe, and everything.
The books rely heavily on probability (even as far as powering the faster than light engine as alluded in the summary).
A pair of dice is one of, of not the most common symbol for probability, chance, and luck (at least in Anglo-American culture). And how many pips are on a pair of dice?
I'm out of my mind right now, but feel free to leave a message.....
I found it highly improbable that an article on that topic could be boring. It explained to me in laborious detail why I was wrong.
Slashdot: where don knuth is an idiot because he cant grasp the awesome power of php
Why? Because there are 7 billion people on Earth.
There are well defined techniques for measuring the probability of events happening in industrial safety. Safety Integrity Levels or SIL are used to categorize the possibility of a life threatening event occurring.
The problem is how low a risk do you need and how much will it cost you to get there. Fukashima would probably not have happened if the sea wall had been higher, but the designers had to make the judgement that it was not worth the millions of cost required to build a bigger wall compared to risk of it being breached. Unfortunately decisions like that in hindsight always look flawed.,
Choose your allies carefully, it is highly unlikely you will be held accountable for the actions of your enemies
Very interesting article on it http://www.lotterypost.com/new... been a long time since I've read it (bookmark), but this guy can tell which scratch tickets will pay off by by reading their serial numbers, winning wasn't as improbable as one is led to believe - and yes, of course he's a statistician.
I don't play the lottery, maybe a ticket twice a year, but my son likes the scratch tickets, I told him that they were predictable, he refused to listen; he wouldn't even pick up the link I printed out. He refused to imagine that it wasn't anything but random. It was just an odd reaction, I can't begin to explain the reasoning behind it.
The link is old so I imagine the serial number gig has been fixed (yet I have no clue one way or the other), but supports the improbability disclaimer.
They didn't need higher sea wall. A wall can permanently hold water away and that is overdoing. What they needed to do was to make a watertight, anchored to the ground building for auxiliary generators and connect them to main building by undersea power cables. Basically, build a submarine on land, complete with snorkel. When there is a water surge, it holds generators safe and dry so that they function after the water recedes.
http://xkcd.com/1331/
Law of truly large numbers is the applicable law here, but the mistake is understandable.
Law of truly large numbers is the applicable law here, but the mistake is understandable.
In fact fairly probable
Just a bit of a nitpick. Mother nature did not "prove your point". Statistics infer data for a population from a sample. A single event from that sample does not prove or disprove anything about the population, nor the sample. Had there not been an event at Fukushima that day, your statement would not have been any more true or false, or any less proven. Your point is proven with statistical significance tests on the sample, not by taking one event and saying "here's proof". That's the opposite of statistics.
I understand what you're saying but I think much more care and precision is needed when articulating issues of probability. The bar in most discussions is set so hopelessly low that the general population - the people who least understand statistics and are most in need of some help - end up with insane theories as to how and why things occur.
It makes any rational discussion about risk impossible. I'm sure we've all heard some anecdote along the lines of "They say smoking causes cancer, but I've got an Uncle who smoked his whole life and lived to 102! Those stupid scientists don't know anything!"
People who are in a position to help with understanding these concepts do not clearly articulate the correct ideas, whether unintentionally (as in this post) or maliciously (politicians). We as a society need to become better at this.
Often when the probability of an event gets close to 1-in-100 people just say "impossible", i.e. they round down to zero.
They also forget that one can increase the chances of the event happening by repeating the trial. E.g. funding a 1-in-100 chances of blow-out-success company sounds like a risky bet, but if you fund 100 such companies, it is a rather safe bet. Hence VCs.
This is a counter-intuitive situation in which increasing the occurrences of the risky behaviour makes the whole situation safer. (Contrast this with Russian roulette in which increased trials is definitely a bad thing).
Firstly, Pratchett's comment has nothing to do with a paradox something of the sort. It's a simple claim that scientists are bad at estimating very small probabilities, and typically get them wrong by a factor of hundreds of thousands. This is actually true and rather insightful in a the-emperor-has-no-clothes kind of way, and also not very deep at all.
The concept of the long tail is somewhat more interesting, but not that deep either. It's merely about realizing that many processes aren't Gaussian, unlike what students are lead to believe in highschool and various introductory courses which are not primarily about statistics.
However, your distinction between likely and unlikely events is confused. If you are going to label two events as likely and unlikely, then you are asserting that the likely event is to be observed with higher probability than the unlikely event. This is always true by definition.
What you are trying to say is that, if you restrict yourself to a particular family of events and you compare the probability of occurrence of an unspecified member of the family with the probability of occurrence of a single specified member, then the former can be larger.
As an example, consider the family of events {the hour of your death is N}. It is fairly unlikely that I can predict the hour of your death (not being a serial killer myself), so if I specify the event {the hour of your death is 12am} then the probability of occurrence is small. But if I do not specify the event, by saying {the hour of your death is N, where N is some hour in the day}, then that event is certain. Of course I haven't said anything interesting *with certainty*, whereas in the case of 12am I have said something interesting *with low probability*.
The tragedy of statistics is that the great majority of things we know with high probability aren't interesting, and the majority of things that are interesting have low probabilities or cannot be estimated accurately.
"Feynman discussed this ages ago. And I'm sure he did it better."
That's highly improbable.
Another example is in the curious case of Professor Meadows - a great paediatrician but a shite mathematician.
He endorsed the dictum that “one sudden infant death is a tragedy, two is suspicious and three is murder, until proved otherwise“. The trouble is, given enough numbers, multiple cot deaths are an inevitability.
Unfortunately, his expert testimony convicted an innocent woman. Fortunately, she was released on appeal when the math was reviewed.
--- "We've always been at war with Eastasia."
applied to debunking so-called "Intelligent Design". There are a few high profile proponents who claim that the probability of an organism as complex as humans evolving from single celled ancestors is so small as to be impossible, therefore we must have been "designed" by "someone" (a variation on the God of the gaps principle used by others for the same purpose). They like to point out eyes as organs that are so complex they could not have evolved, even though we have numerous living organisms that have organisms with photosensitive sensitive organs that aren't quite eyes, perhaps on their way to becoming eyes, many generations/mutations down the road.
In a single field of view under a microscope I can see tens of thousands of bacteria swimming around in a drop of water. Multiple that by all the drops of water in the world and you quickly realize that the number of living organisms is a HUGE number. With all that genetic replication (with errors that sometimes result) and gene swapping going on, and all the DNA floating around freely in the waters of the world, it seems inevitable that there will be enough mutations taking place to produce the variety of life we see on earth.