The Normal Distribution

In this lesson, you will learn about the normal distribution. Many continuous random variables that model natural and social phenomena follow the normal distribution. This is the most important distribution in the field of statistics.

Empirical Rule

If #X \sim N\big(\mu,\sigma^2\big)#, then there is an approximate probability of #0.68# that an observed measurement of #X# will fall into the interval #(\mu - \sigma, \mu + \sigma)#.

Furthermore, there is an approximate probability of #0.95# that an observed measurement of #X# will fall into the interval #(\mu - 2\sigma, \mu + 2\sigma)#.

Lastly, there is an approximate probability of #0.997# that an observed measurement of #X# will fall into the interval #(\mu - 3\sigma,\mu + 3\sigma)#.

Hence it would be very unusual to observe a value of #X# that is more than #3# standard deviations away from the mean.

Normal Distribution and Sampling

If a random sample is selected from a population for which some non-random variable has a normal distribution, then the distribution of the same variable on the sample itself should also be approximately normal (i.e., a histogram of the variable measured on the sample would have a similar shape to the density curve shown above).

Moreover, if a non-random variable has a #N\big(\mu,\sigma^2\big)# distribution on a population, and we plan to randomly select a member of the population and measure that variable, and we let #X# denote the outcome of that measurement, then #X# is a random variable which has a #N\big(\mu,\sigma^2\big)# probability distribution.

Approximating the Normal Distribution

Sometimes a variable is not continuous, but its range consists of many possible values, and the histogram of its frequency distribution over the values has the shape of the density curve of the normal distribution.

It is then often convenient to approximate the distribution of such a variable with the normal distribution.

Standardizing

An important and useful fact that you will need to remember is: If

\[X \sim N \big( \mu, \sigma^2 \big) \]

and we define

\[Z = \frac{X - \mu}{\sigma}\]

then

\[Z \sim N \big( 0, 1 \big) \]

The conversion of #X# to #Z# shown here is called standardizing, and the result is often referred to as the Z-score.

Normal Probability Calculations

To compute the probability for a normally-distributed random variable #X# to be observed in some interval, we would have to compute an integral of its pdf over the interval.

That is, if #X \sim N\big(\mu,\sigma^2\big)#, then:

\[\begin{array}{rcl}
\mathbb{P}(X \leq x) &=& \displaystyle\int_{-\infty}^{x}\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(t-\mu)^2}{2\sigma^2}}dt\\\\
\mathbb{P}(X \geq x) &=& \displaystyle\int_{x}^{\infty}\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(t-\mu)^2}{2\sigma^2}}dt\\\\
\mathbb{P}(x_1 \leq X \leq x_2) &=& \displaystyle\int_{x_1}^{x_2}\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(t-\mu)^2}{2\sigma^2}}dt
\end{array}\]

Quantiles of the Normal Distribution

Suppose #X \sim N\big(\mu,\sigma^2\big)#. We might also want the #p#th quantile #x_p# of #X# that satisfies

\[F(x_p) = \mathbb{P}(X \leq x_p) = p\]

for any #p# in #(0,1)#.

This means that the area beneath the normal density curve to the left of #x_p# equals #p#, i.e.,

\[p = \int_{-\infty}^{x_p}\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(t-\mu)^2}{2\sigma^2}}dt\]

In statistics we will be most interested in quantiles of the standard normal distribution. When we use the standard normal distribution, we use #z# in place of #x#. So, for #Z \sim N\big(0,1\big)#, the #p#th quantile of #Z# is represented by #z_p#.

Because the standard normal distribution is symmetric around #0#, we have the property that #z_p = -z_{1-p}# for every #p# in #(0,1)#.

Linear Combinations of a Single Normally-Distributed Variable

If #X\sim N\big(\mu, \sigma^2\big)# and #a# and #b# are any constants from the real numbers with #a \neq 0# and we define

\[Y= aX +b\]

then

\[Y\sim N\big(a\mu+b, a^2\sigma^2\big)\]

Linear Combinations of Two Normally-Distributed Variables

Now suppose we have two random variables, #X# and #Y#, with #X \sim N\big(\mu_1, \sigma^2_1)# and #Y \sim N\big(\mu_2, \sigma^2_2)#, and suppose #X# and #Y# are independent. Then

\[X+Y \sim N\big(\mu_1 + \mu_2,\,\, \sigma^2_1 + \sigma^2_2\big) \]

and

\[X - Y \sim N\big(\mu_1 - \mu_2,\,\, \sigma^2_1 + \sigma^2_2\big) \]

Note carefully that the variances are added even when we have #X-Y#. This is because, for any real constants #a# and #b#:

\[aX + bY \sim N\big(a\mu_1 + b\mu_2, a^2\sigma^2_1 + b^2\sigma^2_2\big)\]

We would think of #X-Y# as #(1)X + (-1)Y#.

Linear Combinations of n Normally-Distributed Variables

Finally, suppose we have #n# random variables #X_1, X_2, \ldots, X_n#, each of which has the #N\big( \mu,\sigma^2\big) # distribution, i.e.:

\[X_i \sim N\big(\mu, \sigma^2\big)\]

for #i = 1, 2, \ldots, n#.

Suppose further that these #n# random variables are mutually independent. Then the following results hold:

\[\begin{array}{rcccl}
\text{if}& S = \displaystyle\sum_{i=1}^n X_i &\text{then} & S \sim N\big(n\mu, n\sigma^2\big)
\end{array}\]

and

\[\begin{array}{rcccl}
\text{if}& \bar{X} = \cfrac{S}{n} &\text{then} & \bar{X} \sim N\Big(\mu, \cfrac{\sigma^2}{n}\Big)
\end{array}\]

Using R

Normal Probability Calculations

If #X \sim N\big(\mu,\sigma^2\big)# and you want #\mathbb{P}(X \leq x)#, you can compute this in #\mathrm{R}# using the function #\mathtt{pnorm()}#:

> pnorm(x,μ,σ)

Note very carefully that you must use the standard deviation in the #\mathrm{R}# command, not the variance.

In particular, if #Z \sim N\big(0,1\big)# and you want #\mathbb{P}(Z \leq z)#, you can compute this is #\mathrm{R}# using

> pnorm(z)

In other words, if you are working with the standard normal distribution, you don't need to specify the value of the mean or the standard deviation. The #\mathrm{R}# command assumes you are working with the standard normal distribution unless you specify some other values for the mean and standard deviation.

Normal Probability of the Complement

If you want the complement probability, i.e., #\mathbb{P}(X \geq x)# or #\mathbb{P}(Z \geq z)#, you could subtract the above probabilities from #1#.

But you can also use an extra setting in the original #\mathrm{R}# commands given above:

> pnorm(x,μ,σ,low=F)

> pnorm(z,low=F)

The setting #\mathtt{low=F}# gives the area to the right of #x# (or #z#) under the density curve rather than the area to the left.

Calculating Normal Quantiles

If instead you know the probability #p# and you need the quantile #x_p# (or #z_p# for the standard normal), in #\mathrm{R}# you use the function #\mathtt{qnorm()}#:

> qnorm(q,μ,σ)

Or, if you have a standard normal distribution, use:

> qnorm(p)

If you want #p# to be the area to the right of the quantile rather than the area to its left, you can add the setting #\mathtt{low=F}# as above. Or else just replace #p# with #1 - p#.

So

> qnorm(0.975) 

would give the same value as

> qnorm(0.025,low=F)

Generating a Random Sample from a Normal Distribution

You might want to generate a random sample of size #n# from a #N\big(\mu,\sigma^2\big)# distribution. This is done in #\mathrm{R}# using the function #\mathtt{rnorm()}#:

> rnorm(n,μ,σ) 

Or, if you have a standard normal distribution, use:

> rnorm(n)

You may recall that for the binomial distribution there was an #\mathrm{R}# function #\mathtt{dbinom()}# that gives you the probability that #X = x#. There is also an #\mathrm{R}# function #\mathtt{dnorm()}#, but this does not give you the probability that #X = x#, which we know is #0# for a continuous random variable.

Instead, it gives you the value of the normal pdf #f(x)# at any real number #x#. This is useful if you want to make a plot of the normal density curve. But in this statistics course we will not need this function.