Why a perfect correlation can mean nothing: intuition behind correlation

-
Besides attending a meeting in the morning discussing the Colombia paper, I have read some interesting stuff about correlation.

When we talk about correlation, in most cases we mean "linear correlation". What is a linear correlation?

Let's look at two cases where there is no linear correlation (the correlation coefficient is 0).

In the first case, the two variables follow a quadratic function, a parabole, and the correlation is zero. In the second case, the fitted line can go through (2,2) from any direction so there is no way to fit a certain line. Actually, the correlation is zero as long as the scatterplot is symmetrical by either horizontal or vertical line.

In sum, the strength of the correlation only describes how well the linear model can describe the relationship, or how well a straight line can get closer to all the points. A linear model is relatively easy to interpret and can work well in many real-world cases, but there is no guarantee for it to work everywhere.

To give another case in which linear correlation is misleading: the two variables have a pretty strong negative linear correlation (r=-0.66, r2=0.4) but they actually follow a hyperbolic function. This is why talking about correlation without observing the scatterplot makes little sense. And be cautious next time when the statistician claims that the two variables are highly correlated.
By the way, the Spearman rank-order correlation coefficient which describes the monotonic relationship between two variables is -1 for this case (a perfect monotonical correlation). The linear correlation is also called Pearson product moment correlation. 

Last but not least, it is worth mentioning even a perfect correlation could mean nothing. For example, US spending on science and technology correlates perfectly with suicides by hanging, strangulation, and suffocation (r=1). It just happens. Without question, human minds are so inclined to inject causation into correlation. I cannot bear reasoning why when I read these wonderful examples of spurious correlations. 

I have also run into some excellent explanations on the intuition behind correlation.
Especially this one by William Huber on Cross Validated: in short, he said if you make a rectangle for every pair of points, color them in red if the line connecting them goes upper right, and in blue if the line goes upper left:
An example of 3 points:

Do this for all the pairs of points. Redder the plot, more positive the covariance. A dominating red or blue picture indicates a strong correlation.






Comments

Green57 said…
I remember how Professors studying Statistics repeatedly emphasized that "correlation does not mean causation." Interesting examples where two items are highly correlated but are unlikely to be related.
January 17, 2018 at 4:59 PM
Post a Comment

Popular posts from this blog

How to Draw Heatmap with Colorful Dendrogram

-
Image
Heatmap with Dendrogram: Color Branches and Labels This data visualization example include: * Hierarchical clustering, dendrogram, and heat map based on normalized odds ratios * The dendrogram was built separately to give color to dendrogram’s branches/labels based on clusters using  dendextend * Heatmap is made by  heatmap.2  from  gplots  using the built dendrogram * The rows are sorted by means from highest to lowest, it can be done in either the dendrogram or the  heatmap.2 * Use color palettes from  colorspace Data YRBSS (Youth Risk Behavior Surveillance System) survey data from CDC. This example plots the odds ratios of 63 behavioral questions on one question regarding disordered eating. The odds ratios are median centered by column and log2 transformed Version 1: Color both the branches and labels library (curl) # read file from google drive library (gplots) # heatmap.2 library (dendextend) # make and color dendrogram library (colorspace) #
Read more

Power-law distribution (Pareto)& Zipf's Law: connection and how to fit the distribution of global city population

-
Image
- This post counts for 3 days. - In short,  We use the histogram to outline the distribution, use rank-frequency plot (log-log plot) to identify the power-law distribution, and use maximum likelihood estimation to obtain the parameters. -  I will show 1 st , how the power-law function CDF and PDF can be derived from Zipf CDF; and 2 nd , this least square estimation by excel is wrong and how to get the right one. Historical background on the power-law distribution. - Power-law distribution is as common as normal distribution in the real world. In the 19 th century, Italian economist Pareto realized that 20% of the population owns 80% the wealth, which is the famous 80/20 rule. He later named the power-law distribution as Pareto distribution, which is what we studied in the probability course. - In the 20 th century, another main contributor, Zipf made a similar discovery in linguistics about the frequency of words used. - Besides social wealth, many things follow a hig
Read more

SAS’s Best Subset Selection by Mallows's Cp is actually Stepwise?

-
Image
SAS’s Best Subset is actually Stepwise? I answered this question in my next post:  Best subset selection uses the branch and bound algorithm to speed up The original post explaining my own confusion: I suspect that the function  regsubsets  from R library  leaps  does go through all the possible combinations, and it takes hours for 50 variables. Facing a large number of variables SAS just uses stepwise selection even though the code asks for best subset by adding  selection = cp  to the model part of  proc reg . This is my suspicion, I don’t know whether maybe it never scans all the combinations. I tested with the same dataset from the  last post Without cross-validation, I used all the 598 observations to run  regsubsets : # R: nv_max <- 25 # up-limit of number of variables to test fit_s <- regsubsets(Share_Temporary~., mydata4, really.big = T , nbest= 1 , nvmax = nv_max ) As we can see from the figure above, the number of variables with
Read more