Linear CorrelationConsider two quantitative random variables measured on a population.


If one of the variables has a tendency to change (either increasing or decreasing) by some fixed amount whenever the other variable increases by one unit, we say that the two variables are linearly correlated, and assign a measure for that linear correlation between #-1# and #1#, inclusive.

This measure is generally referred to as the population correlation coefficent and is denoted #\rho#.

Pearson Sample Correlation CoefficientConsider a random sample of size #n# from the population, with measurements on two quantitative variables #X# and #Y# for each element in the sample. This gives us a set of bivariate data #(x_1,y_1),...,(x_n,y_n)#.

The Pearson sample correlation coefficient #r# can then be computed as the average of the products of the standardized measurements over the sample:
\[\begin{array}{rcl}
r &=& \cfrac{1}{n - 1} \displaystyle\sum_{i=1}^n \bigg(\cfrac{x_i - \bar{x}}{s_x}\bigg)\bigg(\cfrac{y_i - \bar{y}}{s_y}\bigg) \\\\
&=& \cfrac{\displaystyle\sum_{i=1}^n \Big(x_i - \bar{x} \Big)\Big(y_i - \bar{y} \Big)}{\sqrt{\displaystyle\sum_{i=1}^n \Big(x_i - \bar{x}\Big)^2 \displaystyle\sum_{i=1}^n \Big(y_i - \bar{y}\Big)^2}}
\end{array}\]

The sample correlation coefficient #r# between #X# and #Y# is an unbiased estimate of the population correlation coefficient #\rho# between these two variables.

As with #\rho#, the sample correlation coefficient #r# is between #-1# and #1#, and #r# is unitless.

Moreover, #r# is symmetric, i.e., the sample correlation coefficient between #X# and #Y# is the same as the sample correlation coefficient between #Y# and #X#.

Spurious RelationshipA spurious relationship between two variables indicates that changes in the values of both variables are caused by changes in the values of other unmeasured variables, which we call confounding variables (or lurking variables).

Hence we say that correlation does not imply causation (although causation does imply correlation).

Inference about the Population Correlation CoefficientWe say that the pair of random variables #(X,Y)# has a bivariate normal distribution if the random variable #aX + bY# has a normal distribution for every pair of constants #a# and #b#. A sample from a bivariate normal distribution will result in a scatterplot with an elliptical shape.

In this setting, we can make inference about the population correlation coefficient #\rho# based on the sample correlation coefficient #r#. It can be shown, when the sample size is large, that the statistic \[W = \cfrac{1}{2} \ln \bigg( \cfrac{1 + r}{1 - r}\bigg)\] has an approximate \[N\Bigg(\cfrac{1}{2} \ln \bigg(\cfrac{1 + \rho}{1 - \rho}\bigg),\cfrac{1}{n - 3}\Bigg)\] distribution.

Confidence interval for #\rho#

We can construct a #(1 - \alpha)100\%# CI for #\rho# as follows: \[\Bigg(\cfrac{e^{2(W-z_{\alpha/2}/\sqrt{n-3})} - 1}{e^{2(W-z_{\alpha/2}/\sqrt{n-3})} + 1},\cfrac{e^{2(W+z_{\alpha/2}/\sqrt{n-3})} - 1}{e^{2(W+z_{\alpha/2}/\sqrt{n-3})} + 1}\Bigg)\] The upper bound should not exceed #1#, and the lower bound should not be smaller than #-1#, since we require #-1 \leq \rho \leq 1#.

Hypothesis test about #\rho#

For any #\rho_0 \neq 0#, we can test

#H_0 : \rho = \rho_0# or #H_0 : \rho \leq \rho_0# or #H_0 : \rho \geq \rho_0#

against

#H_1 : \rho \neq \rho_0# or #H_1 : \rho > \rho_0# or #H_1 : \rho < \rho_0#

respectively, as follows:

Set \[W_0 = \cfrac{1}{2} \ln \bigg(\cfrac{1 + \rho_0}{1 - \rho_0}\bigg)\] and compute \[Z = \cfrac{W - W_0}{\sqrt{1/(n - 3)}}\] where \[W = \cfrac{1}{2 }\ln \bigg(\cfrac{1 + r}{1 - r}\bigg)\] Then #Z \sim N(0,1)#.

Depending on the form of #H_1#, compute the P-value using the standard normal distribution as we have done multiple times already.

However, when #\rho_0 = 0#, the procedure to test

#H_0 : \rho = \rho_0# or #H_0 : \rho \leq \rho_0# or #H_0 : \rho \geq \rho_0#

against

#H_1 : \rho \neq \rho_0# or #H_1 : \rho > \rho_0# or #H_1 : \rho < \rho_0#

respectively, is much simpler.

Set \[U = \cfrac{r\sqrt{n-2}}{\sqrt{1 - r^2}}\] Then #U \sim t_{n-2}#.

Depending on the form of #H_1#, compute the P-value using the Student’s t-distribution as we have done many times previously.