Slashdot Mirror

It's not a matter of smart or stupid here. Even a very smart person will come to the wrong conclusion if they have bad data.

When people are ignorant of statistics, they are unable to collect good information, and unable to figure out what is true among multiple interpretations. The correlation is real, what do they decide?

There is a very real correlation between venemous spiders and spelling bee champions. That is undeniable. Understanding the correlation takes a level of understanding of statistics. What kind of experiment would you develop to determine if spider bites are causing spelling bee champions?

For people who don't understand statistics, the world is a very confusing place, no matter how smart you are.

Strawman argument.

Why? Because there NEVER would have BEEN USB Thumb Drives in the first place, if not for Apple and the iMac.

HAHA good one.

Proof? USB Ports had been on EVERY Wintel motherboard for YEARS with virtually NOTHING to plug into them... UNTIL THE IMAC CAME ALONG.

That's hardly a proof. http://tylervigen.com/spurious...

I applaud your healthy skepticism; however, in this case, the vast majority of non-Apple-Hating tech experts would agree that Apple, through the iMac, pretty much singlehandedly put USB "on the map".

https://www.macworld.com/artic...

" Apple Inc.'s iMac was the first mainstream product with USB and the iMac's success popularized USB itself.[12] Following Apple's design decision to remove all legacy port from the iMac, many PC manufacturers began building legacy-free PCs, which led to the broader PC market using USB as a standard.[13][14][15]"

https://en.wikipedia.org/wiki/...

Citations from above:

12. "Eight ways the iMac changed computing". Macworld. 15 August 2008. Archived from the original on 22 December 2011. Retrieved 5 September 2017.

13. "Compaq hopes to follow the iMac". Archived from the original on 22 October 2006.

14. "The PC Follows iMac's Lead". Business week. 1999. Archived from the original on 23 September 2015.

15. Popular Mechanics: Making Connections. Hearst Magazines. February 2001. p. 59. ISSN 0032-4558. Archived from the original on 15 February 2017.

So, no; in this particular case, the consensus is that the iMac made USB into a viable alternative to parallel and RS-232 Serial (as well as other buses, such as ADB) interfaces popular at the time.

And considering that USB Thumb Drives first appeared on the market in 2000, it is fairly certain that, if the iMac had not come along and, through its popularity, given USB the critical "push" it needed to get past the "chicken and egg" problem faced by every new technology, those USB Thumb Drives would certainly not happened in 2000, and likely not until much, much later.

https://en.wikipedia.org/wiki/...

Heck, I'm sure that you remember: While Apple was just a few months from launching the iMac, Microsoft couldn't so much as plug in a USB SCANNER without Blue-Screening W '98, even though Windows advertises USB Support as early as W '95 OSR2.1!

https://www.theregister.co.uk/...

And, BTW, note that Windows didn't even HAVE Storage Device USB Support until W '98 SE (May, 1999). AFAIK, Apple supported Storage-Class Devices from the first release of the iMac in August, 1998 (running (Classic) MacOS 8.1 !!!).

https://en.wikipedia.org/wiki/...

https://en.wikipedia.org/wiki/...

...and let's not even talk about Linux's sad and late adoption of USB. It wasn't even added until the 2.2 Kernel in 1999, although that was experimental support. Mainstream USB support wasn't really available in Linux until the 2.4 Kernel was released in 2001:

https://kernel.readthedocs.io/...

https://en.wikipedia.org/wiki/...

And even now, USB and Linux is sometimes an exercise in frustration (but I guess that applies to all things Linux, doesn't it?)

https://www.google.com/search?...

Strawman argument.

Why? Because there NEVER would have BEEN USB Thumb Drives in the first place, if not for Apple and the iMac.

HAHA good one.

Proof? USB Ports had been on EVERY Wintel motherboard for YEARS with virtually NOTHING to plug into them... UNTIL THE IMAC CAME ALONG.

That's hardly a proof. http://tylervigen.com/spurious...

http://tylervigen.com/view_correlation?id=1703/

I could swear that people are just looking at http://tylervigen.com/spurious... for ideas to make new headlines with.

I dunno, I believe the divorce/margarine one. Seems like a clear correlation.

I could swear that people are just looking at http://tylervigen.com/spurious... for ideas to make new headlines with.

Yeah, I grew up less than a quarter mile from one of the major city highways and about a half a mile from the other major freeway (both feeding a city pop around 200k). When I started college I was 5'9" and 115 pounds (male, so more than a little skinny), since I left home I'm a more healthy weight but far from any overweight category in my late 30s. So am I just an outlier in this study (yeah I know, one data point out of 7.5 billion and all that...)?

Parents needed a new reason to be paranoid and overprotective. That's never been a bad thing.

On a serious note, I grew up in Pittsburgh. Prior the 1980s, the air was exceptionally unsafe. Since then, it's only been getting cleaner. What's the trend line for obesity look like? Spoiler: obesity is going up and up, while the air is cleaner than it's ever been for nearly 40 years. Therefore if bad air was thing, you'd expect the exact opposite.

Alarmist news reports over spurious correlations do not help people. They only make your average person lose trust in science. That's not particularly a good thing.

I could swear that people are just looking at http://tylervigen.com/spurious... for ideas to make new headlines with.

Well... real correlations with crappy R2 do mean that there's some connection between the two things.

First off, there's spurious correlations--where the most reasonable explanation for a statistically significant correlation (where the R2 is actually a very high value) is that it's just a really, really, really weird coincidence. This site lets you explore examples, and some have a very strong R^2 while also being quite unlikely to actually have a correlation--for example, the divorce rate in Maine correlates to the per capita consumption of margarine with an R^2 of 0.99...but there is no reasonable mechanism by which one could influence the other. That's actually part of why you're generally supposed to include some sort of theory when you're trying to show correlations, unless your goal is to create demonstrations of why it's wrong to assume that correlation even implies connection, no less causation.

Next, a R^2 of 0.02 is...fantastically low. The range is from 0 to 1, because it's basically a ratio. You can think of it very accurately as a measure of how well the line you just drew on the graph paper correlates to the data points--with 1 being the point at which it's perfect and 0 being where it has no relationship whatsoever.

Last? The usual cutoff for statistical significance in psych research is 0.05. Statistical significance is the point where you feel the correlation is strong enough that it isn't just cause by error--and one major cause of error is choosing a too-low threshold. And, well, 0.02 is less than half the normal threshold...

When the painful majority of news articles are clearly based on spurious correlations, such a trite phrase really does warrant repeating.

Multi-core bots are people too.

Hmm, 243 is Venus's rotation period. Coincidence?

Yes. Also, all of these. Put away the tin foil.

Control for other causes among 500K participants over the entire breadth of the island of Great Britain? What do you want??

I want some effort putting into determining whether or not this correlation actually means anything before they report it as if it actually means something. There are all kinds of weird correlations that are statistically significant but don't actually mean anything or have any direct relationship. For example the divorce rate in the State of Maine has a 99% correlation with the per capita consumption of margarine in the US. Those two facts clearly have nothing to do with one another and yet they correlate incredibly strongly with large populations behind each.

I see coincidence, at the very most. No correlations was demonstrated.

http://www.tylervigen.com/spur...

I love "studies" that find a "link" between two things like this. Correlation is king!

Seems all these articles these days are getting overly redundant in an unnecessary and redundant way. Here's the basic formula:

$COMPANY can't find enough cheap workers and comes up with $STUPID_IDEA to try to explain why competent people still cost $TOO_MUCH.

As a bonus for this article, we can have fun pointing out obvious lurking variables because they obviously don't realize things may not be as they seem.

They need to know about this then.

The better explanation for the decreased crime rate is the explosion in private gun ownership.

I think even better explanation is an increase in per capita cheese consumption.

"Suggest" not "Prove". Correlations are everywhere and they don't really mean anything. If someone can give a reason why this is the case then thats a different matter.
http://www.tylervigen.com/spurious-correlations

Be aware! Overconsumption of water can lead to death. Even moderate consumption, if sustained over time, can poison your brain and body. There's no right or wrong, but that is the choice being made here.

I read that on the Internet, so it must be true.

Well, that's clearly worth a spike in strokes and heart attacks.

Did you know that a rise in per capita cheese consumption causes people to die from getting tangled in their bedsheets?

I can prove it.

http://tylervigen.com/discover

Any irrelevant ideological barrow could have been pushed from those numbers just by excluding all those people in China.
eg. So it appears that places where people eat bread instead of rice are less corrupt. Who would have thought?

Time for fun-with-spurious-correlations yet? :)

http://www.tylervigen.com/spur...

Remember it's a Chaebol and includes LG Electronics, LG Display, LG Uplus, LG Chem, LG Life Sciences, and LG Solar Energy.

Given how many industries they're in the chart they provide is a spurious correlation at best.

And in related news, US spending on science, space, and technology correlates with suicides by hanging, strangulation and suffocation. http://tylervigen.com/spurious...

Funny how people are quick to point out "the negative impacts that occur in some religious communities" without saying much about the positive. Some religious communities (it can be hard to separate the religion from the community) are clearly doing something right.

I can't find good current stats on household income by religion, but this 2009 survey breaks it down pretty well.

Hindus come out on top, as they have for some time now: evidence that the more gods you believe in, the more successful you are in life. Or maybe it's something to do with the number of pirates - the data is slim. In any case, religions are recipes for life, and some of them seem to still be pretty good recipes, much as /. would hate to consider such a thing.

If religion is a recipe for life then, in this analogy, all the world's religions involve animal products. I'm now a vegan. ;-)

While you're arguing that the number of gods and happiness per annum are related, I hope you are familiar with http://www.tylervigen.com/spur...

Otherwise you'd have to accept as fact that the divorce rate in Maine is controlled by the per capita consumption of margarine. Worse yet, bedsheet deaths and ski profits are linked. Any argument against my position would also undermine your god(s)=Mo$$$ argument as well.

Please note: if you have any argument against mine, I appreciate the time you spend writing up a rebuttal. I would love the feedback as it either helps me go to your side, or refines my argument. Regardless, it helps me see your side better.

It's a classic example of methodological incompetence. Got your popcorn? Let's start the show.

Their dependent variable is piracy rates (between 0 and 100) as published by the BSA. Not one word about measurement uncertainty in those data. Remember how the MPAA and the BSA used to estimate sales losses due to piracy? That's right: list_price x (penetration_fraction x population of PC's - license sold). I kid you not. And they calmly rely on piracy data from those sources.

Then their explanatory variable: the so-called IQ measure. They cite the "seminal" work of Lyn,R. VanHaanen, T ()

Unfortunately for the authors of the latest "correlation paper", the work of Lyn and VanHaanen is anything but uncontroversial. I quote from one of the Lyn and VanHaanenpapers:

First, we believe that these estab- lish beyond reasonable doubt the validity of our national IQ. This was initially disputed by a number of critics. For in- stance, Ervik (2003, pp. 405â"6) asked âoeare people in rich countries smarter than those in poorer countries?â and con- cluded that âoethe authors fail to present convincing evidence and appear to jump to conclusions.â Nechyba (2004, p. 1178) wrote of the âoerelatively weak statistical evidence and dubious presumptions.â Barnett and Williams (2004, p.) rejected our national IQ as âoevirtually meaninglessâ; Volken (2003, p. 411) described them as âoehighly deficientâ; and Hunt and Sternberg (2006, pp. 133, 136) rejected them as âoetechnically inadequate⦠and meaninglessâ. The answer to these criticisms is that our national IQs are validated by their high correlations with scores in tests of mathematics, science and reading, as shown in Table 1, and also with the numerous other economic and social phenom- ena documented in subsequent tables. These high correla- tions would not be present if our national IQs were meaningless.

Gettit? The fact that there are high correlations "proves" the validity of their inference that there are meaningful relationships. Did they go to Trump University or what?

In this vein I especially like the high correlation (see http://www.tylervigen.com/spur... ) between per-capita cheese consumption and people who died by becoming entangled in their bedsheets.

I wonder if the authors thought to control for that.

As far as serious research is concerned, this is the end of the line, but lets go on and have a look at their model, shall we?

They model the value of a fraction through a straightforward regression model: SP_i = \alpha + \beta IQ_i + \lambda X_i + \epsilon_i

Oops, and there we have the little matter of using straightforward regression to model a fraaaaction, instead of something like logistic regression. For those who don't immediately spot the problem, see e.g. here: http://www.theanalysisfactor.c...

Ordinary linear models are simply unsuitable to model fractions. A point that's common knowledge with statisticians, but one that's apparently lost on the authors (and the authors on which they base their work).

Right, lets continue and look at the graph they show with their regression line. Each country counts as one (China has the same weight as the e.g. Senegal and the US has the same weight as the Comores. Look ma, no weights! Sounds good eh? When you look at their graph, China shows up as one serious outlier with an "IQ" score of about 110 and a "piracy" score of about 80%. Only 1 bln people up there. Close by, in the bottom-right corner of their graph is the good ole US of A, weighing in at about 270 mln people, with almost the same score

There is a nearly perfect correlation between US spending on science and suicides by hanging/strangulation/suffocation.

It is meaningful in this context because the context is that the study discovered a correlation between countries where a tendency to do well in standardised IQ tests correlated with less piracy.

In other contexts I would probably agree with you.

You seem like the kind of person who would have fun with this: http://www.tylervigen.com/spur...

Maybe the US finally increased spending on science, space, and technology?

.... usually results in findings like this:

http://tylervigen.com/spurious... [tylervigen.com]

After viewing the link I must say that, while I understand the point you're trying to make, I think that there probably IS more than just a passing correlation between the number of people who drowned after falling out of fishing boats, and the marriage rate in Kentucky.

Because doing a wide-ranging statistical analysis on something as wide-ranging as "Autism," which is a diagnosis and not a particular disorder, usually results in findings like this:

http://tylervigen.com/spurious...

The vast majority of high quality correlations are also causations. Sometimes a common cause for both can confuse matters, but in the vast majority of cases you can trust a correlation to be a causation.

That is ABSOLUTELY false.

I mean, seriously -- just think about what you said for a minute. There is an insanely large number of possible datasets in the world. And if you can match up any dataset with any other dataset, you'll have a similarly insanely large number of high correlations that just happen by random chance.

That vast majority of such "correlations" are absolutely meaningless.

If we take your statement to be true, we'd have to conclude that most of the correlations "discovered" here are likely to be causal: US spending on science causes suicides by suffocation (or the reverse, r=0.9979), per capita margarine consumption causes variations in the divorce rate in Maine (r=0.9926), annual number of drownings by falling out of a fishing boat causes the marriage rate in Kentucky to go up and down (r=0.95), per capita consumption of mozzarella cheese causes more civil engineering doctorates to be awarded (r=0.96), etc., etc.

The correlation between phasing out leaded petrol and falling crime holds in many countries which banned lead at different times. It is highly unlikely that some other cause happened at the right time in all the countries.

See, now you're starting to get there. When we can track a correlation and match it up with the cause in time or using some other variable which influences both, the causality likelihood starts to increase.

The causality itself is also quite uncontroversial: It is known that exposure to lead means lower average IQ, and lower average IQ means more violence.

And now we get even further along -- the causality mechanism might actually make sense.

I'm NOT arguing against this particular theory, by the way. I'm just pointing out that your assertion that the "vast majority" of "high quality" correlations imply causation is ridiculous. If you just allow any dataset to be matched with any other dataset in the world, the vast majority of correlations will be stupid random nonsense. That's the reason the "p-hacking" is a useless statistical practice that will inevitably result in false relationships.

It's only when you start tracking other variables related to the correlation and establish likely causality through other reasonable mechanisms and data in other variables that you can start proving something.

I'm not arguing the point (because I agree on this one), but your post reminded me of Spurious Correlations.

Did the submitter get this off of Spurious Correlations?

Here, I found an entire web site dedicated to helping find these correlations! http://www.tylervigen.com/spur...

OK, the subject is a bit of a click-bait, but I started thinking about it after I had to fix no less than four different devices within three weeks apart. I strongly suspect that I'm just strongly biased by my "local" sampling. Nevertheless, the fact that the devices were rather unrelated (an almost new compact flash, an SSD, a spinning 3.5' and a spinning 2.5') made me wonder: did anybody noticed a similar pattern recently? Now, if you excuse me, I have to sign a petition for stopping pool drownings by preventing Nicholas Cage to appear in movies.

Don't despair! There are new spurious correlations every day. We just need to feed him the sunspot data.

"I'm merely pointing out that this is more akin to a debate scenario"
Ah, I understand better where you are coming from.

"You can't just say, well it's statistical data so you're probably wrong and then dismiss the claim."
That isn't what I was trying to do. I never meant to say that statistical data is likely wrong and therefore can be dismissed. What I was saying that proof usually has something more concrete than merely someone picking out a data set and then applying graphs that would seem to indicate what they believe is right. Trying to make a one factor correlation when things are much more complex multifactorial and interrelated things seems to me to likely to end up with false conclusions. I read recently that Statistics are like a Drunk seeking a lamppost for support rather than illumination. I found this to be a rather fitting synopsis.

My disputing the OP’s proof lies in the fact that it may likely be a misuse of statistics and therefore invalid. To support my supposition perhaps I was not very clear. It is the OP's position it would seem that more guns = fewer murders based upon the data set chosen. The OP further uses a statistical chart to make the point. So then, the central question becomes does this correlation actually exist or is it some brand of false causality.

From the Wiki:
When a statistical test shows a correlation between A and B, there are usually six possibilities:

A causes B.
B causes A.
A and B both partly cause each other.
A and B are both caused by a third factor, C.
B is caused by C which is correlated to A.
The observed correlation was due purely to chance.

Merely because one can find an apparent correlation in a random data set does not mean that the data set is a proof that the one thing leads to another. Since the supposed proof of the OP is based upon a simple 2 factor correlation then the fact that there are other options in interpretation means that the Proof is not really a Proof, but merely an interpretation which may or may not be valid. Certainly there can be no conclusion drawn from spurious correlation other than to point to the fact that further and more specific studies or data would need to be done regarding the question.

As an example of why this actually needs to further illumination I found these other ridiculous spurious correlations to illustrate my point.
http://www.tylervigen.com/

Things can be shown on charts which have no bearing on reality.

If tylervigen.com has tought me anything it's that you are 100% correct.

For example, did you know?

Motorcycle riders killed in collision with stationary object correlates with Corporate Political Action Committees (US)

Obviously PACs are bad for motorcyclists!

If tylervigen.com has tought me anything it's that you are 100% correct.

For example, did you know?

Motorcycle riders killed in collision with stationary object correlates with Corporate Political Action Committees (US)

Obviously PACs are bad for motorcyclists!

Please explain any possible "C" that explains the correlation between US crude oil imports from Norway versus Drivers killed in collision with railway train. Sometimes correlation is just correlation.

You can find all sorts of weird correlations if you look for them but the mere existence of a correlation is meaningless by itself. In this case my first question would be about money. States with more money will be able to afford both faster internet and better schools. Other factors that need to be controlled for include population density, local industry, demographic makeup, etc to be able to put some meaning to this.

Basically this is a meaningless correlation which provides no context to draw useful conclusions from. Obligatory XKCD.

This is data mining. If you compare enough things you'll find strange correlations. There is little plausible reason to believe there is an actual causal relationship here.

These are also "irrefutable correlations":

US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation:
http://www.tylervigen.com/view...

Number people who drowned by falling into a swimming-pool correlates with Number of films Nicolas Cage appeared in
http://tylervigen.com/view_cor...

Per capita consumption of cheese (US) correlates with Number of people who died by becoming tangled in their bedsheets:
http://tylervigen.com/view_cor...

This is data mining. If you compare enough things you'll find strange correlations. There is little plausible reason to believe there is an actual causal relationship here.

These are also "irrefutable correlations":

US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation:
http://www.tylervigen.com/view...

Number people who drowned by falling into a swimming-pool correlates with Number of films Nicolas Cage appeared in
http://tylervigen.com/view_cor...

Per capita consumption of cheese (US) correlates with Number of people who died by becoming tangled in their bedsheets:
http://tylervigen.com/view_cor...

This is data mining. If you compare enough things you'll find strange correlations. There is little plausible reason to believe there is an actual causal relationship here.

These are also "irrefutable correlations":

US spending on science, space, and technology correlates with Suicides by hanging, strangulation and suffocation:
http://www.tylervigen.com/view...

Number people who drowned by falling into a swimming-pool correlates with Number of films Nicolas Cage appeared in
http://tylervigen.com/view_cor...

Per capita consumption of cheese (US) correlates with Number of people who died by becoming tangled in their bedsheets:
http://tylervigen.com/view_cor...

Who is "John Oliver"? And this post reminds me of Spurious Correlations.

Some random comic makes a joke to a small segment of the TV watching late-nighters, and this somehow took down an FCC web site?

Sure, OK, if it makes you feel better....

More pseudoscience. They say that they're not sure whether this means that porn shrinks your brain, or if the shrunken brain causes porn viewing. But, this leaves out the very real possibility that this correlation means nothing whatsoever. The site below collects correlations that look pretty convincing in the graphs, but quite obviously are unlikely to be cases of causation in either direction:

http://www.tylervigen.com/

Or is there really nothing other than CO2 levels which correlates strongly with the use of portable classrooms and with absenteeism? Perhaps low socioeconomic status has nothing to do with which school districts have more trouble affording permanent buildings? Perhaps higher numbers of children per family are unrelated to which schools are overcrowded?

It's hard to tell, when the bibliography consists of "studies show".

What's sad is that this is still better-than-average science and science reporting. We got an actual transcript, and the correlation seems to be at least a step above the "people who wear parachutes are more likely to die in skydiving accidents!" level which is so good at grabbing headlines.

So, the only thing that scientists agree on are (1-3).

Don't forget to point out THIS.

Alarmists have made a very big habit out of taking anything that might correlate to increase in CO2 and call it "proof". That's just nonsense.

A positive correlation does not imply cause. Period. It can only suggest, and often not even that.

HERE is another example, and HERE is another.

Look! Correlation of 0.99! Obviously, golf causes consumption of cheese, and pets cause football.

So, the only thing that scientists agree on are (1-3).

Don't forget to point out THIS.

Alarmists have made a very big habit out of taking anything that might correlate to increase in CO2 and call it "proof". That's just nonsense.

A positive correlation does not imply cause. Period. It can only suggest, and often not even that.

HERE is another example, and HERE is another.

Look! Correlation of 0.99! Obviously, golf causes consumption of cheese, and pets cause football.

So, the only thing that scientists agree on are (1-3).

Don't forget to point out THIS.

Alarmists have made a very big habit out of taking anything that might correlate to increase in CO2 and call it "proof". That's just nonsense.

A positive correlation does not imply cause. Period. It can only suggest, and often not even that.

HERE is another example, and HERE is another.

Look! Correlation of 0.99! Obviously, golf causes consumption of cheese, and pets cause football.