Correlation is Not Causation, but ...
By: bappit , 12:18 AM GMT on June 19, 2013
Next: That Reminds Me ... or The Limitations of Language
Previously: Ludicrous! ... or ... The Elephant Effect
* * * * * *
Researchers have proposed methods of detecting likely causal relationships in data for pairs of correlated variables through a strictly mechanical/statistical process (machine learning). A group of these researchers are conducting a competition on www.kaggle.com, "Cause and Effect Pairs," to encourage the development of techniques and algorithms for this purpose. The basic idea is that when a likely causal connection is detected between two correlated variables, then resources can be allocated to studying that relationship. Similarly, if a correlation is judged not to be a likely causal connection, then resources can be spent studying something else.
The www.kaggle.com web page for the competition gives the following example of two variables that show correlation and how one can intuitively infer a likely direction of cause and effect.
Can you guess which variable might be cause and which effect? The units are not shown because they might give away the answer. Also, I don't think the measurement scales actually use any conventional units anyway. The units are obfuscated.
This plot happens to be of two weather-related variables. The way to infer cause and effect is to notice that there seem to be two values of variable B when variable A is about 130, but there appears to be only one value of variable A for any value of B. That is, A is a function of B. The suggested inference is that since A is a function of B, the direction of cause and effect is likely to run from B to A and not vice versa.
You can check the website to see their answer to this puzzle and read more--if you want. The research papers they reference here are not easy reading.
I suspect that practical results in this area are not quite there, but the concept is interesting.
The views of the author are his/her own and do not necessarily represent the position of The Weather Company or its parent, IBM.