Properties of orthonormal systems

Inner Product Spaces: Orthonormal systems

Properties of orthonormal systems

We discuss some properties of orthonormal systems of vectors.

Let $V$ be an inner product space and $\vec{x}$ a vector of $V$ .

Orthonormal systems in $V$ are linearly independent.
If $\basis{\vec{a}_1,\ldots ,\vec{a}_n}$ is an orthonormal basis of $V$ , then the coordinates of $\vec{x}$ with respect to this basis are, successively, $\dotprod{\vec{x}}{\vec{a}_1}, \ldots , \dotprod{\vec{x}}{\vec{a}_n}$ : $\vec{x}=(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n$
The length of $\vec{x}$ is equal to the length of the coordinate vector of $\vec{x}$ relative to the standard inner product:
$\norm{\vec{x}}^2 =\left(\dotprod{\vec{x}}{\vec{a}_1}\right)^2 + \cdots + \left(\dotprod{\vec{x}}{\vec{a}_n}\right)^2$

1. Suppose that $\vec{a}_1,\ldots ,\vec{a}_n$ is an orthonormal system in $V$ . To demonstrate its independence, we start with the equation
$\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n =\vec{0}$ with unknowns $\lambda_1,\ldots , \lambda_n$ . By taking the inner product of each side of the equality with the vector $\vec{a}_j$ with $1\leq j\leq n$ we find
$\dotprod{(\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n)}{ \vec{a}_j} =\dotprod{\vec{0}}{\vec{a}_j}=0$ The left-hand side can be simplified as follows:
$\begin{array}{rcl}\dotprod{(\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n)}{ \vec{a}_j}&=&\lambda_1(\dotprod{\vec{a}_1}{\vec{a}_j})+\cdots + \lambda_n(\dotprod{\vec{a}_n}{\vec{a}_j})\\&&\phantom{xx}\color{blue}{\text{linearity of the inner product}}\\ &=&\lambda_j (\dotprod{\vec{a}_j}{\vec{a}_j})\\&&\phantom{xx}\color{blue}{\text{orthogonality of the system}}\\&=&\lambda_j\\ &&\phantom{xx}\color{blue}{\text{orthonormality of the system}}\\\end{array}$ So the only solution to $\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n =\vec{0}$ is $\lambda_j=0$ for all $j$ , showing the independence of $\vec{a}_1,\ldots ,\vec{a}_n$ .

2. Because $\vec{a}_1,\ldots ,\vec{a}_n$ is a basis of $V$ , there are scalars $\mu_1 , \ldots ,\mu_n$ such that $\vec{x}=\mu_1 \vec{a}_1 +\cdots + \mu_n \vec{a}_n$ . In order to determine $\mu_j$ , we again take the inner product of each side of this equality with $\vec{a}_j$ :
$\dotprod{\vec{x}}{\vec{a}_j}=\dotprod{(\mu_1 \vec{a}_1 +\cdots + \mu_n \vec{a}_n)}{\vec{a}_j}$ After working out the right-hand side in a similar manner as above we find $\dotprod{\vec{x}}{\vec{a}_j}=\mu_j$ .

3. We now know that we can write $\vec{x}$ as $\vec{x}=(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n$ . Since all of these terms are perpendicular to each other, we can apply the Pythagorean theorem, which gives $\begin{array}{rcl}\norm{\vec{x}}^2&=&\norm{(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n}^2\\&&\phantom{xx}\color{blue}{\text{above expression for }\vec{x}}\\&=&\norm{(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1}^2 + \cdots + \norm{(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n}^2\\&&\phantom{xx}\color{blue}{\text{Pythagoras}}\\&=&(\dotprod{\vec{x}}{\vec{a}_1})^2\norm{\vec{a}_1}^2 + \cdots + (\dotprod{\vec{x}}{\vec{a}_n})^2\norm{ \vec{a}_n}^2\\&&\phantom{xx}\color{blue}{\text{linearity of the inner product}}\\&=&(\dotprod{\vec{x}}{\vec{a}_1})^2+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})^2\\&&\phantom{xx}\color{blue}{\norm{\vec{a}_i}= 1 \text{ thanks to orthonormality of the system}}\end{array}$

Basis

From the mutual independence of the vectors in an orthonormal system it follows that, if we have an orthonormal system of $n$ vectors in an $n$ -dimensional inner product space, this system forms a basis for this space.

Later we will show that an orthonormal basis can be found for any finite-dimensional inner product space.

The system $\frac{1}{\sqrt{2}}\rv{1,-1,0}, \, \frac{1}{\sqrt{2}}\rv{1,1,0},\,\rv{0,0,1}$ forms an orthonormal system, and so is independent. The three vectors form a basis of $\mathbb{R}^3$ because the dimension of the space equals $3$ . The coordinates of $\vec{a}=\rv{2,2,2}$ with respect to this basis are
$\dotprod{\vec{a}}{\frac{1}{\sqrt{2}}\rv{1,-1,0}}=0, \phantom{xx} \dotprod{\vec{a}}{ \frac{1}{\sqrt{2}}\rv{1,1,0}}=2\sqrt{2},\phantom{xx} \dotprod{\vec{a}}{\rv{0,0,1}}=2$ Consequently,
$\rv{2,2,2}=2\sqrt{2}\cdot\frac{1}{\sqrt{2}}\,\rv{1,1,0} + 2\,\cdot \rv{0,0,1}$

The following statement is of interest for computing inner products if we have an orthonormal basis at our disposal.

Let $\basis{\vec{e}_1,\ldots ,\vec{e}_n}$ be an orthonormal basis for an inner product space $V$ , and let

$\vec{x}=\sum_{i=1}^n x_i\vec{e}_i,\qquad \vec{y}=\sum_{j=1}^n y_j\vec{e}_j$ be vectors of $V$ , written as linear combinations of the basis vectors. Then the inner product of $\vec{x}$ and $\vec{y}$ can be expressed as follows: $\dotprod{\vec{x}}{\vec{y}}=\sum_{i=1}^n x_i\cdot y_i$

Because of the definition of a basis we can write the vectors $\vec{x}$ and $\vec{y}$ as
$\vec{x}=\sum_{i=1}^n x_i\vec{e}_i,\qquad \vec{y}=\sum_{i=1}^n y_i\vec{e}_i$ We now work out the inner product: $\begin{array}{rcl}\dotprod{\vec{x}}{\vec{y}}&=&\displaystyle\dotprod{\left(\sum_{i=1}^n x_i\vec{e}_i\right)}{\left(\sum_{i=1}^n y_i\vec{e}_i\right)}\\&&\phantom{xx}\color{blue}{\text{written in terms of the basis}}\\&=&\displaystyle\sum_{i=1}^n\sum_{j=1}^n x_i y_j \cdot(\dotprod{\vec{e}_i}{ \vec{e}_j})\\&&\phantom{xx}\color{blue}{\text{linearity of the inner product}}\\&=&\displaystyle \sum_{i=1}^nx_iy_i\cdot(\dotprod{\vec{e}_i}{\vec{e}_i})\\&&\phantom{xx}\color{blue}{\dotprod{\vec{e}_i}{\vec{e}_j}=0\text{ if }i\neq j}\\&=&\displaystyle\sum_{i=1}^n x_i y_i\\&&\phantom{xx}\color{blue}{\dotprod{\vec{e}_i}{\vec{e}_j}=1\text{ if }i=j}\end{array}$

Just as for the standard inner product, the inner product can be written as a product of a $(1\times n)$ -matrix and an $(n\times 1)$ -matrix. We write $\vec{c}_x$ for coordinate vector $\vec{x}$ with respect to the basis $\vec{e}_1,\ldots ,\vec{e}_n$ , viewed as a column vector. Similarly, we write $\vec{c}_y$ for the coordinate vector $\vec{y}$ written as a column vector. Thus, these vectors are $(n\times1)$ -matrices. Now we can write the inner product $\dotprod{\vec{x}}{\vec{y}}$ as $\dotprod{\vec{x}}{\vec{y}} = \vec{c}_x^{\top}\,\vec{c}_y$ For example, $\dotprod{\rv{2,3,1}}{\rv{-3,5,2}} = 2\cdot(-3)+3\cdot 5 +1\cdot 2= \matrix{2&3&1}\,\matrix{-3\\ 5\\2}$

The theorem tells us that if we start calculating in $V$ with coordinates with respect to an orthonormal basis, we can express the inner product of two vectors in $V$ in terms of the standard inner product on ${\mathbb{R}^n}$ .

We consider the vector space ${\mathbb{R}^3}$ with the standard inner product. Suppose that the following orthonormal basis is given.
$\begin{array}{rll} \displaystyle \vec{v}_1&=&\displaystyle \rv{ {{2}\over{\sqrt{6}}} , {{1}\over{\sqrt{6}}} , -{{1}\over{\sqrt{6}}} } \\ \vec{v}_2&=&\displaystyle \rv{ 0 , {{1}\over{\sqrt{2}}} , {{1}\over{\sqrt{2}}} } \\ \vec{v}_3&=&\displaystyle \rv{ {{1}\over{\sqrt{3}}} , -{{1}\over{\sqrt{3}}} , {{1}\over{\sqrt{3}}} } \end{array}$
Determine the coordinate vector $\vec{c}=\rv{c_1,c_2,c_3}$ with respect to the given basis of the vector
$\vec{x}=\rv{ 4 , 1 , 3 }$

$\vec{c}=$ $\rv{\sqrt{6},2\sqrt{2},2\sqrt{3}}$

The given basis is orthonormal. According to the Properties of orthonormal systems of vectors, the coordinates of a vector $\vec{x}$ with respect to an orthonormal basis equal the dot products of $\vec{x}$ with the basis vectors. We calculate each of these inner products individually.
$\begin{array}{rcl} c_1&=&\displaystyle\dotprod{\vec{v}_1}{\vec{x}}\\ &&\phantom{xx}\color{blue}{\text{theorem}}\\ &=&\displaystyle\dotprod{\rv{ {{2}\over{\sqrt{6}}} , {{1}\over{\sqrt{6}}} , -{{1}\over{\sqrt{6}}} } }{\rv{ 4 , 1 , 3 } }\\ &&\phantom{xx}\color{blue}{\text{explicit vectors filled in}}\\ &=&\displaystyle\left({{2}\over{\sqrt{6}}}\cdot4\right)+\left({{1}\over{\sqrt{6}}}\cdot 1\right)+\left(-{{1}\over{\sqrt{6}}}\cdot3\right)\\ &&\phantom{xx}\color{blue}{\text{definition standard inner product}}\\ &=&\displaystyle\sqrt{6}\\ &&\phantom{xx}\color{blue}{\text{simplified}}\\ \end{array}$
$\begin{array}{rcl} c_2&=&\displaystyle\dotprod{\vec{v}_2}{\vec{x}}\\ &&\phantom{xx}\color{blue}{\text{theorem}}\\ &=&\displaystyle\dotprod{\rv{ 0 , {{1}\over{\sqrt{2}}} , {{1}\over{\sqrt{2}}} } }{\rv{ 4 , 1 , 3 } }\\ &&\phantom{xx}\color{blue}{\text{explicit vectors filled in}}\\ &=&\displaystyle\left(0\cdot4\right)+\left({{1}\over{\sqrt{2}}}\cdot 1\right)+\left({{1}\over{\sqrt{2}}}\cdot3\right)\\ &&\phantom{xx}\color{blue}{\text{definition standard inner product}}\\ &=&\displaystyle2\sqrt{2}\\ &&\phantom{xx}\color{blue}{\text{simplified}}\\ \end{array}$
$\begin{array}{rcl} c_3&=&\displaystyle\dotprod{\vec{v}_3}{\vec{x}}\\ &&\phantom{xx}\color{blue}{\text{theorem}}\\ &=&\displaystyle \dotprod{\rv{ {{1}\over{\sqrt{3}}} , -{{1}\over{\sqrt{3}}} , {{1}\over{\sqrt{3}}} } }{\rv{ 4 , 1 , 3 } }\\ &&\phantom{xx}\color{blue}{\text{explicit vectors filled in}}\\ &=&\displaystyle\left({{1}\over{\sqrt{3}}}\cdot4\right)+\left(-{{1}\over{\sqrt{3}}}\cdot 1\right)+\left({{1}\over{\sqrt{3}}}\cdot3\right)\\ &&\phantom{xx}\color{blue}{\text{definition standard inner product}}\\ &=&\displaystyle2\sqrt{3}\\ &&\phantom{xx}\color{blue}{\text{simplified}}\\ \end{array}$
So the coordinate vector $\vec{c}$ is given by $\rv{\sqrt{6},2\sqrt{2},2\sqrt{3}}$ .

New example