Cause and Effect: How a Revolutionary New Statistical Test Can Tease Them Apart

KentuckyFC writes Statisticians have long thought it impossible to tell cause and effect apart using observational data. The problem is to take two sets of measurements that are correlated, say X and Y, and to find out if X caused Y or Y caused X. That's straightforward with a controlled experiment in which one variable can be held constant to see how this influences the other. Take for example, a correlation between wind speed and the rotation speed of a wind turbine. Observational data gives no clue about cause and effect but an experiment that holds the wind speed constant while measuring the speed of the turbine, and vice versa, would soon give an answer. But in the last couple of years, statisticians have developed a technique that can tease apart cause and effect from the observational data alone. It is based on the idea that any set of measurements always contain noise. However, the noise in the cause variable can influence the effect but not the other way round. So the noise in the effect dataset is always more complex than the noise in the cause dataset. The new statistical test, known as the additive noise model, is designed to find this asymmetry. Now statisticians have tested the model on 88 sets of cause-and-effect data, ranging from altitude and temperature measurements at German weather stations to the correlation between rent and apartment size in student accommodation.The results suggest that the additive noise model can tease apart cause and effect correctly in up to 80 per cent of the cases (provided there are no confounding factors or selection effects). That's a useful new trick in a statistician's armoury, particularly in areas of science where controlled experiments are expensive, unethical or practically impossible.

Comments

Always

[phantomfive]

So the noise in the effect dataset is always more complex than the noise in the cause dataset....... the additive noise model can tease apart cause and effect correctly in up to 80 per cent of the cases

In other words, not always.

Re:Always

[Mr D from 63]

This is the tricky part, and it seems to work if you know exactly the cause and effect in advance, so you know which data to look at. It is quite clever though, and would seem to have application as an indicator if nothing else.

I recall some equipment monitoring techniques used in my industry. There were reams of data. If a piece of equipment failed, you could go back and look at the data and see that there were indications. But filtering those indications out as useful input was always the problem. Only the blatant, in your face indications were caught. I see a similar problem here, that you might be able to show cause and effect with this data in hindsight, but it won't be so clear when you don't know the answer already.

Re:Always

[phantomfive]

Indeed, it's easy to think of situations where the opposite is true, where the noise is simpler in the 'effect' than in the 'cause,' because there is some attenuation factor in between that reduces the noise. That's more or less what a damper or shock absorber is designed to do. And a low pass filter in audio does the same thing.

Now you might say, "obviously a low-pass filter is in the way, and that's causing the difference" but that gets back to your point, where it's easy to figure out when you already know the system, but if you don't, then it's not so easy.

Re:So, correlation CAN mean causation?

[Anonymous Coward]

Gawd I hate the brain-dead fools who thoughtlessly parrot, "Correlation is not causation!"

The proper term is: "Correlation does not imply causation". Perhaps you are being pendantic, but I'd rather hang around people who think "Correlation is not causation" (since it is more correct), than people who think "Correlation is causation".

Re:So, correlation CAN mean causation?

[Wraithlyn]

I prefer "Correlation does not prove causation".

Edward Tufte suggested "Correlation is not causation but it sure is a hint."