#\text{}#

In section 3.2 we defined #z_p# as the quantile of the standard normal distribution which has probability #p# to its left, such that #\mathbb{P}(Z \leq z_p) = p#. This is the notation used in probability. But in statistics, it will be more convenient to use this notation for the quantile which has probability #p# to its right, such that #\mathbb{P}(Z \geq z_p) = p#.

So from here on in this course, #z_{\alpha/2}# denotes the quantile of the standard normal distribution which has probability #\alpha/2# to its right, in the right tail of the standard normal distribution. Then, because the standard normal distribution is symmetric about zero, #-z_{\alpha/2}# denotes the quantile of the standard normal distribution which has probability #\alpha/2# to its left, in the left tail of the standard normal distribution. So if #\alpha = 0.05# then #z_{0.05/2} = 1.96# and #-z_{0.05/2} = -1.96#. (In R, this distinction is made by adding the setting #\mathrm{lower.tail=FALSE}# or #\mathrm{low=F}# in the #\mathrm{qnorm()}# command.)

Suppose #X# is a quantitative variable which, if #X# could be measured on every element of some population of interest, would have a mean of #\mu#. Because #\mu# represents the value of #X# for a “typical” element of the population, we would like to have a precise estimate of its value, based on a preferred level of confidence.

So we plan to select a random sample of size #n#, and measure #X# on each element of the sample, to obtain #X_1,..., X_n#. From these measurements we will compute the sample mean #\bar{X}# and the sample standard deviation #s#.

There are several options for the method of computing a #(1 - \alpha)100\%# confidence interval for #\mu#, based on the answers to three questions. These questions are:

1. Can we assume that #X# has a normal distribution on the population?
   
   
2. Do we know the value of the population variance #\sigma^2# of #X#?
   
   
3. Can we consider the sample size #n# to be large?

We first consider the situation in which the answer to questions 1 and 2 is “Yes”. If so, the answer to question 3 is irrelevant.

We assume #X \sim N(\mu,\sigma^2)# and the value of #\sigma^2# is known. Then \[Z = \cfrac{\bar{X} - \mu}{\sigma/\sqrt{n}} \sim N(0,1)\] and the middle #(1 - \alpha)100\%# of the distribution of #Z# falls between the quantiles #-z_{\alpha /2}# and #z_{\alpha /2}#.

This means that: \[\mathbb{P}\bigg(-z_{\alpha /2} \leq \cfrac{\bar{X} - \mu}{\sigma/\sqrt{n}} \leq z_{\alpha /2}\bigg) = 1 - \alpha\]

If we rearrange this inequality with some algebraic manipulation, we have:

\[\mathbb{P}\bigg(\bar{X} - z_{\alpha /2}\cfrac{\sigma}{\sqrt{n}} \leq \mu \leq \bar{X} + z_{\alpha /2}\cfrac{\sigma}{\sqrt{n}}\bigg) = 1 - \alpha\]

Hence \[L = \bar{X} - z_{\alpha /2} \cfrac{\sigma}{\sqrt{n}} \,\,\,\,\,\,\,\,\,\,\,\,\,\, \text{and} \,\,\,\,\,\,\,\,\,\,\,\,\,\, U = \bar{X} + z_{\alpha /2}\cfrac{\sigma}{\sqrt{n}}\]

qnorm(0.025,low=F) 

qnorm(0.975)

#\text{}#

Going back to the three questions, suppose the answer to the first question is either “no” or “I’m not sure”, but the answers to the second and third questions are both “yes”. That is, we don’t know if #X# has a normal distribution on the population, but the sample size #n# is large and the value of #\sigma^2# is known.

The Central Limit Theorem tells us that #\bar{X}# has an approximate #N\Big(\mu,\sigma^2\Big)# distribution, so we can assume \[\cfrac{\bar{X} - \mu}{\sigma/\sqrt{n}} \sim N(0,1)\] approximately. Then the formula for computing the upper and lower bounds of the #(1 - \alpha)100\%# CI for #\mu# is exactly the same as in the previous setting.

Now suppose the answer to the first question is still either “no” or “I’m not sure”, the answer to the third question is still “yes”, but the answer to the second question is “no”. That is, we don’t know the actual value of #\sigma^2#, which is actually much more realistic.

If the sample size is large, then the value of the sample variance #s^2# should be close to the actual value of #\sigma^2#. Combined with the Central Limit Theorem, we can say that \[\cfrac{\bar{X} - \mu}{s/\sqrt{n}} \sim N(0,1)\] approximately.

#\text{}#

Finally, suppose the answer to the first question is “yes” and the answers to the second and third questions are “no”. So #X \sim N(\mu,\sigma^2)# but we don’t know the value of #\sigma^2# and the sample size #n# is not large.

In this case, we have to work with #\cfrac{\bar{X} - \mu}{s/\sqrt{n}}# again, but this random variable does not have an approximate #N(0,1)# distribution.

However, it can be shown in this setting that #\cfrac{\bar{X} - \mu}{s/\sqrt{n}}# has the Student’s t-distribution with #n - 1# degrees of freedom. So we would write:

\[\cfrac{\bar{X} - \mu}{s/\sqrt{n}} \sim t_{n-1}\]

Now the structure of the CI is essentially the same, but in place of #z_{\alpha /2}# we use #t_{n-1,\alpha /2}#, the quantile of the #t_{n-1}# distribution for which #1 - \cfrac{\alpha}{2}# of the area below the density curve falls to its left (and thus #\cfrac{\alpha}{2}# of the area falls to its right).

> qt(1-α/2, n-1)

> qt(α/2, n-1, low=F)

> qt(0.005,9,low=F)

> qt(0.995,9)

We didn’t address every possible situation from our three questions (such as if the answers to all three questions are “no”), but there are methods available for those situations as well. Those are not covered in this course.