Lesson 21: Vector Projection

Objectives

Given two non-zero vectors, compute the projection of one onto the other, by hand and in R, or describe why it is not possible.
Describe a projection as the closest possible linear combination of one vector to the other.
Calculate the residual vector between a vector and its projection.
Given two 2-D non-zero vectors, sketch the projection and residual by hand.

1. Projection of One Vector onto Another

Objective. Given two non-zero vectors, compute the projection of one onto the other, by hand and in R, or describe why it is not possible.

The projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ is a scalar multiple $x\overline{\vphantom{b}u}$ that lies on the line through $\overline{\vphantom{b}u}$ , with the scalar $x$ given by the definition below. The dashed segment from the tip of $x\overline{\vphantom{b}u}$ to the tip of $\overline{\vphantom{b}v}$ meets that line at a right angle; this segment is the residual vector $\overline{\vphantom{b}r}$ , defined in section 3. The dot product, length, and orthogonality used below are on the Lesson 20 sheet.

Definition 1 (Projection).

For a vector $\overline{\vphantom{b}v}$ and a nonzero vector $\overline{\vphantom{b}u}$ with the same number of components, the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ is the vector $x\overline{\vphantom{b}u}$ , where

x = \frac{\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}}{\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}}.

The vector $x\overline{\vphantom{b}u}$ is the projection vector, and the number $x$ is the projection scalar.

The vector $\overline{\vphantom{b}v} = \left(\begin{smallmatrix} 3\\4 \end{smallmatrix}\right)$ has no projection onto the zero vector $\overline{\vphantom{b}0} = \left(\begin{smallmatrix} 0\\0 \end{smallmatrix}\right)$ , because $\overline{\vphantom{b}0}\cdot\overline{\vphantom{b}0} = 0$ and the quotient $(\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}0})/(\overline{\vphantom{b}0}\cdot\overline{\vphantom{b}0}) = 0/0$ is undefined. The definition requires $\overline{\vphantom{b}u}$ to be nonzero.

Why it works: the formula gives the closest linear combination

Take $\overline{\vphantom{b}u}$ other than the zero vector. Then $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u} > 0$ and $x$ is defined. The residual $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - x\overline{\vphantom{b}u}$ is orthogonal to $\overline{\vphantom{b}u}$ , by the formula for $x$ :

\overline{\vphantom{b}r}\cdot\overline{\vphantom{b}u} = \overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u} - x\,(\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}) = 0.

For every scalar $t$ , the difference $\overline{\vphantom{b}v} - t\overline{\vphantom{b}u}$ equals $\overline{\vphantom{b}r} + (x - t)\overline{\vphantom{b}u}$ . The dot product distributes over a sum the way multiplication does, and expanding gives

\lVert \overline{\vphantom{b}v} - t\overline{\vphantom{b}u} \rVert^2 = \lVert \overline{\vphantom{b}r} \rVert^2 + 2(x - t)\,\overline{\vphantom{b}r}\cdot\overline{\vphantom{b}u} + (x - t)^2\lVert \overline{\vphantom{b}u} \rVert^2.

The middle term is 0, because $\overline{\vphantom{b}r}\cdot\overline{\vphantom{b}u} = 0$ , which leaves

\lVert \overline{\vphantom{b}v} - t\overline{\vphantom{b}u} \rVert^2 = \lVert \overline{\vphantom{b}r} \rVert^2 + (x - t)^2\lVert \overline{\vphantom{b}u} \rVert^2 \ge \lVert \overline{\vphantom{b}r} \rVert^2

with equality only at $t = x$ . Every other linear combination of $\overline{\vphantom{b}u}$ is farther from $\overline{\vphantom{b}v}$ .

Order of the vectors. The projection of $\overline{\vphantom{b}u}$ onto $\overline{\vphantom{b}v}$ is a multiple of $\overline{\vphantom{b}v}$ , and in general it differs from the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ .

Sign of the scalar. The denominator $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}$ is positive for every $\overline{\vphantom{b}u}$ other than the zero vector. The scalar $x$ therefore takes the sign of $\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}$ . In each graph below, $\overline{\vphantom{b}u} = \left(\begin{smallmatrix} 2\\1 \end{smallmatrix}\right)$ points along a road through the origin, and the projection of $\overline{\vphantom{b}v}$ marks the point of the road closest to $\overline{\vphantom{b}v}$ .

Positive scalar

For $\overline{\vphantom{b}v} = \left(\begin{smallmatrix} 2\\3 \end{smallmatrix}\right)$ , the dot product $\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}$ is 7, and the scalar $x = 1.4$ is positive. The projection $1.4\overline{\vphantom{b}u}$ , in orange, points the same way as $\overline{\vphantom{b}u}$ : the closest point lies ahead of the origin on the road.

Negative scalar

For $\overline{\vphantom{b}v} = \left(\begin{smallmatrix} -2\\1 \end{smallmatrix}\right)$ , the dot product $\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}$ is $-3$ , and the scalar $x = -0.6$ is negative. The projection $-0.6\overline{\vphantom{b}u}$ points opposite $\overline{\vphantom{b}u}$ : the closest point lies behind the origin on the road.

Zero scalar

For $\overline{\vphantom{b}v} = \left(\begin{smallmatrix} -1\\2 \end{smallmatrix}\right)$ , the dot product $\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}$ is 0, and so is the scalar $x$ . The vectors are orthogonal, and the projection is the zero vector: the closest point is the origin itself.

Example 1

Compute the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ by hand, for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 3 \\ 4 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 2 \\ 1 \end{pmatrix*}.

ShowHide solution

By the definition of projection, the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ is

x\overline{\vphantom{b}u} = \left(\frac{\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}}{\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}}\right)\overline{\vphantom{b}u}.

Substitute the components of $\overline{\vphantom{b}v}$ and $\overline{\vphantom{b}u}$ into the formula for the projection scalar:

x = \frac{3(2) + 4(1)}{2(2) + 1(1)} = \frac{6 + 4}{4 + 1} = \frac{10}{5} = 2.

Scale $\overline{\vphantom{b}u}$ by $x$ to get the projection vector:

x\overline{\vphantom{b}u} = 2\begin{pmatrix*}[r] 2 \\ 1 \end{pmatrix*} = \begin{pmatrix*}[r] 4 \\ 2 \end{pmatrix*}.

Example 2

Compute the projection of $\overline{\vphantom{b}u}$ onto $\overline{\vphantom{b}v}$ by hand, for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 3 \\ 4 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 2 \\ 1 \end{pmatrix*}.

ShowHide solution

The projection formula, with the roles of $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ exchanged, gives the projection $y\overline{\vphantom{b}v}$ . Its scalar is named $y$ to keep it apart from the scalar $x$ of the projection onto $\overline{\vphantom{b}u}$ :

y\overline{\vphantom{b}v} = \left(\frac{\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v}}{\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}v}}\right)\overline{\vphantom{b}v},

y = \frac{2(3) + 1(4)}{3(3) + 4(4)} = \frac{10}{25} = 0.4,

y\overline{\vphantom{b}v} = 0.4\begin{pmatrix*}[r] 3 \\ 4 \end{pmatrix*} = \begin{pmatrix*}[r] 1.2 \\ 1.6 \end{pmatrix*}.

This projection is a multiple of $\overline{\vphantom{b}v}$ , unlike the projection $\left(\begin{smallmatrix} 4\\2 \end{smallmatrix}\right)$ of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ in Example 1.

Example 3

Compute the projection scalar and the projection vector of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ by hand, for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 1 \\ 3 \\ 8 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 3 \\ 0 \\ -1 \end{pmatrix*}.

ShowHide solution

Substitute the components into the formula for $x$ , then scale $\overline{\vphantom{b}u}$ by $x$ :

x = \frac{1(3) + 3(0) + 8(-1)}{3(3) + 0(0) + (-1)(-1)} = \frac{-5}{10} = -0.5,

x\overline{\vphantom{b}u} = -0.5\begin{pmatrix*}[r] 3 \\ 0 \\ -1 \end{pmatrix*} = \begin{pmatrix*}[r] -1.5 \\ 0 \\ 0.5 \end{pmatrix*}.

The projection scalar is $-0.5$ , and the projection vector points opposite $\overline{\vphantom{b}u}$ , because the scalar is negative.

In R, project(v ~ u) returns the projection scalar $x$ , and x * u scales $\overline{\vphantom{b}u}$ by it to give the projection vector. The tilde means “onto”.

Comma in place of the tilde. The command project(v, u) returns the projection vector itself and prints no error. When that output is stored as x, the line x * u multiplies the projection vector by $\overline{\vphantom{b}u}$ component by component. For the vectors of Example 1, the result is $\left(\begin{smallmatrix} 8\\2 \end{smallmatrix}\right)$ in place of the projection vector $\left(\begin{smallmatrix} 4\\2 \end{smallmatrix}\right)$ :

> v = c(3, 4)
> u = c(2, 1)
> project(v, u)
[1] 4 2
> x = project(v, u)
> x * u
[1] 8 2

Example 4

Compute the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ in R, for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 3 \\ -1 \\ 2 \\ 6 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 1 \\ 1 \\ -1 \\ 1 \end{pmatrix*}.

ShowHide solution

Store the two vectors, compute the projection scalar with project(v ~ u), and scale $\overline{\vphantom{b}u}$ by it:

v = c(3, -1, 2, 6)
u = c(1, 1, -1, 1)
x = project(v ~ u)
x * u
[1]  1.5  1.5 -1.5  1.5

The projection vector is $1.5\overline{\vphantom{b}u} = \left(\begin{smallmatrix} 1.5\\1.5\\-1.5\\1.5 \end{smallmatrix}\right)$ , and the scalar stored in x is

x = \frac{3 - 1 - 2 + 6}{1 + 1 + 1 + 1} = \frac{6}{4} = 1.5.

As a check, R prints [1] 1.5 1.5 -1.5 1.5 for the formula dot(v, u) / dot(u, u) * u as well.

Rounding. The scalar $x$ stays unrounded, as a fraction by hand or as the full value R stores in x. Rounding $x$ first can change the third decimal of a component.

Example 5

Compute the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ from MMAC 3.4 Example 7(b):

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 1 \\ 5 \\ -2 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 2 \\ 3 \\ 1 \end{pmatrix*}.

ShowHide solution

Substitute the components into the formula, keep the projection scalar as an exact fraction, then scale $\overline{\vphantom{b}u}$ by it:

x = \frac{1(2) + 5(3) + (-2)(1)}{2(2) + 3(3) + 1(1)} = \frac{15}{14},

x\overline{\vphantom{b}u} = \frac{15}{14}\begin{pmatrix*}[r] 2 \\ 3 \\ 1 \end{pmatrix*} = \begin{pmatrix*}[r] 30/14 \\ 45/14 \\ 15/14 \end{pmatrix*} \approx \begin{pmatrix*}[r] 2.143 \\ 3.214 \\ 1.071 \end{pmatrix*}.

Rounding $x$ to 1.071 first gives $1.071\left(\begin{smallmatrix} 2\\3\\1 \end{smallmatrix}\right) = \left(\begin{smallmatrix} 2.142\\3.213\\1.071 \end{smallmatrix}\right)$ . Its first two components are off in the third decimal. R stores the full value of $\frac{15}{14}$ in x:

v = c(1, 5, -2)
u = c(2, 3, 1)
x = project(v ~ u)
x * u
[1] 2.142857 3.214286 1.071429

Example 6

Compute the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ , or describe why it is not possible, for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 4 \\ 1 \\ 2 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 1 \\ 3 \end{pmatrix*}.

ShowHide solution

The projection is not possible, because the dot product $\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}$ of a 3-component vector and a 2-component vector is undefined. In R, project() stops with an error:

> v = c(4, 1, 2)
> u = c(1, 3)
> project(v ~ u)
Error in model.frame.default(x, data = data) :
  variable lengths differ (found for 'u')

Next: 2. Projection as the Closest Linear Combination Lesson map

2. Projection as the Closest Linear Combination

Objective. Describe a projection as the closest possible linear combination of one vector to the other.

A linear combination of the single vector $\overline{\vphantom{b}u}$ is a multiple $t\overline{\vphantom{b}u}$ , for a scalar $t$ (Lesson 17 section 1), and every linear combination of $\overline{\vphantom{b}u}$ lies on the line through $\overline{\vphantom{b}u}$ . When $\overline{\vphantom{b}v}$ lies off that line, no linear combination of $\overline{\vphantom{b}u}$ equals $\overline{\vphantom{b}v}$ , and the closest linear combination of $\overline{\vphantom{b}u}$ to $\overline{\vphantom{b}v}$ is the one at the smallest distance from $\overline{\vphantom{b}v}$ .

The distance from $t\overline{\vphantom{b}u}$ to $\overline{\vphantom{b}v}$ is the length $\lVert \overline{\vphantom{b}r} \rVert$ of the residual vector $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - t\overline{\vphantom{b}u}$ , defined in section 3.

Theorem 2 (Closest linear combination).

For a vector $\overline{\vphantom{b}v}$ and a nonzero vector $\overline{\vphantom{b}u}$ with the same number of components, the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ is the closest linear combination of $\overline{\vphantom{b}u}$ to $\overline{\vphantom{b}v}$ . Every other linear combination of $\overline{\vphantom{b}u}$ is farther from $\overline{\vphantom{b}v}$ .

Example 7

An artillery gun is restricted to a straight road through the origin. Km marker $t$ on the road sits at $t\overline{\vphantom{b}u}$ , and the target sits at $\overline{\vphantom{b}v}$ , in km:

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 2 \\ 4 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 0.8 \\ 0.6 \end{pmatrix*}.

Compare the distance from the gun to the target at km markers $t = -1$ , $3$ , $4$ , and $5$ , and identify the closest km marker.

ShowHide solution

At each km marker, find the distance to the target in three steps:

Compute $t\overline{\vphantom{b}u}$ .
Compute the residual vector $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - t\overline{\vphantom{b}u}$ .
Compute its length $\lVert \overline{\vphantom{b}r} \rVert = \sqrt{\overline{\vphantom{b}r}\cdot\overline{\vphantom{b}r}}$ , the distance to the target.

At $t = -1$ , for example, the distance is

\lVert \overline{\vphantom{b}r} \rVert = \sqrt{(2.8)^2 + (4.6)^2} = \sqrt{7.84 + 21.16} = \sqrt{29} \approx 5.385.

$t$	$t\overline{\vphantom{b}u}$	$\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - t\overline{\vphantom{b}u}$	$\lVert \overline{\vphantom{b}r} \rVert$
$-1$	$\left(\begin{smallmatrix} -0.8\\-0.6 \end{smallmatrix}\right)$	$\left(\begin{smallmatrix} 2.8\\4.6 \end{smallmatrix}\right)$	$\approx 5.385$
$3$	$\left(\begin{smallmatrix} 2.4\\1.8 \end{smallmatrix}\right)$	$\left(\begin{smallmatrix} -0.4\\2.2 \end{smallmatrix}\right)$	$\approx 2.236$
$4$	$\left(\begin{smallmatrix} 3.2\\2.4 \end{smallmatrix}\right)$	$\left(\begin{smallmatrix} -1.2\\1.6 \end{smallmatrix}\right)$	2
$5$	$\left(\begin{smallmatrix} 4\\3 \end{smallmatrix}\right)$	$\left(\begin{smallmatrix} -2\\1 \end{smallmatrix}\right)$	$\approx 2.236$

Of the four markers, marker 4 is the closest, because its distance of 2 km is the smallest. The projection scalar shows that marker 4 is the closest point on the whole road:

x = \frac{\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}}{\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}} = \frac{2(0.8) + 4(0.6)}{(0.8)(0.8) + (0.6)(0.6)} = \frac{1.6 + 2.4}{0.64 + 0.36} = \frac{4}{1} = 4.

By the closest linear combination theorem, the projection $4\overline{\vphantom{b}u} = \left(\begin{smallmatrix} 3.2\\2.4 \end{smallmatrix}\right)$ is closer to the target than every other point $t\overline{\vphantom{b}u}$ on the road. At $t = 4$ the residual vector $\overline{\vphantom{b}r} = \left(\begin{smallmatrix} -1.2\\1.6 \end{smallmatrix}\right)$ is orthogonal to $\overline{\vphantom{b}u}$ , because $(-1.2)(0.8) + 1.6(0.6) = 0$ .

Example 8

Find the linear combination of $\overline{\vphantom{b}u}$ closest to $\overline{\vphantom{b}v}$ and its distance from $\overline{\vphantom{b}v}$ , and compare it with $1\overline{\vphantom{b}u}$ and $3\overline{\vphantom{b}u}$ , for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 2 \\ 1 \\ 3 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 1 \\ 1 \\ 1 \end{pmatrix*}.

ShowHide solution

By the closest linear combination theorem, the closest linear combination is the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ . Compute its scalar and the projection vector:

x = \frac{2(1) + 1(1) + 3(1)}{1(1) + 1(1) + 1(1)} = \frac{6}{3} = 2, \qquad x\overline{\vphantom{b}u} = 2\begin{pmatrix*}[r] 1 \\ 1 \\ 1 \end{pmatrix*} = \begin{pmatrix*}[r] 2 \\ 2 \\ 2 \end{pmatrix*}.

The distance is the length of the residual vector $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - 2\overline{\vphantom{b}u}$ :

\overline{\vphantom{b}r} = \begin{pmatrix*}[r] 0 \\ -1 \\ 1 \end{pmatrix*}, \qquad \lVert \overline{\vphantom{b}r} \rVert = \sqrt{0^2 + (-1)^2 + 1^2} = \sqrt{2} \approx 1.414.

The other two linear combinations are farther, as the theorem states:

\lVert \overline{\vphantom{b}v} - 1\overline{\vphantom{b}u} \rVert = \lVert \begin{pmatrix*}[r] 1 \\ 0 \\ 2 \end{pmatrix*} \rVert = \sqrt{5} \approx 2.236, \qquad \lVert \overline{\vphantom{b}v} - 3\overline{\vphantom{b}u} \rVert = \lVert \begin{pmatrix*}[r] -1 \\ -2 \\ 0 \end{pmatrix*} \rVert = \sqrt{5} \approx 2.236.

R gives the same scalar and distance. The line for d computes $\lVert \overline{\vphantom{b}v} - x\overline{\vphantom{b}u} \rVert = \sqrt{(\overline{\vphantom{b}v} - x\overline{\vphantom{b}u})\cdot(\overline{\vphantom{b}v} - x\overline{\vphantom{b}u})}$ :

v = c(2, 1, 3)
u = c(1, 1, 1)
x = project(v ~ u)
x * u
[1] 2 2 2
d = sqrt(dot(v - x * u, v - x * u))
d
[1] 1.414214

The closest linear combination is $2\overline{\vphantom{b}u} = \left(\begin{smallmatrix} 2\\2\\2 \end{smallmatrix}\right)$ , at a distance of $\sqrt{2} \approx 1.414$ from $\overline{\vphantom{b}v}$ .

Next: 3. Residual Vector Lesson map

3. Residual Vector

Objective. Calculate the residual vector between a vector and its projection.

Definition 3 (Residual vector).

For vectors $\overline{\vphantom{b}v}$ and $\overline{\vphantom{b}u}$ and a scalar $t$ , the residual vector for the linear combination $t\overline{\vphantom{b}u}$ and the target vector $\overline{\vphantom{b}v}$ is $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - t\overline{\vphantom{b}u}$ .

For $\overline{\vphantom{b}v} = \left(\begin{smallmatrix} 3\\4 \end{smallmatrix}\right)$ and $\overline{\vphantom{b}u} = \left(\begin{smallmatrix} 2\\1 \end{smallmatrix}\right)$ , the difference $\overline{\vphantom{b}v} - \left(\begin{smallmatrix} 1\\0 \end{smallmatrix}\right) = \left(\begin{smallmatrix} 2\\4 \end{smallmatrix}\right)$ is not a residual vector for $\overline{\vphantom{b}u}$ , because $\left(\begin{smallmatrix} 1\\0 \end{smallmatrix}\right)$ is not a multiple $t\overline{\vphantom{b}u}$ of $\overline{\vphantom{b}u}$ . The definition subtracts a linear combination $t\overline{\vphantom{b}u}$ .

Proposition 4 (Projection and residual).

For the projection $x\overline{\vphantom{b}u}$ of $\overline{\vphantom{b}v}$ onto a nonzero vector $\overline{\vphantom{b}u}$ , the residual vector $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - x\overline{\vphantom{b}u}$ is orthogonal to $\overline{\vphantom{b}u}$ and to the projection $x\overline{\vphantom{b}u}$ , and the projection and the residual vector add to the target vector:

x\overline{\vphantom{b}u} + \overline{\vphantom{b}r} = \overline{\vphantom{b}v}.

In R, dot(r, u) prints 0 or a number within rounding of 0, such as 2.220446e-15, R’s notation for $2.220446 \times 10^{-15}$ .

Example 9

Compute the residual vector for the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ by hand, and check that the residual is orthogonal to $\overline{\vphantom{b}u}$ , for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 3 \\ 4 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 2 \\ 1 \end{pmatrix*}.

ShowHide solution

By the definition of the residual vector, with $t = x$ the projection scalar, the residual vector is $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - x\overline{\vphantom{b}u}$ . Compute the projection first:

x = \frac{3(2) + 4(1)}{2(2) + 1(1)} = \frac{10}{5} = 2, \qquad x\overline{\vphantom{b}u} = 2\begin{pmatrix*}[r] 2 \\ 1 \end{pmatrix*} = \begin{pmatrix*}[r] 4 \\ 2 \end{pmatrix*},

\overline{\vphantom{b}r} = \begin{pmatrix*}[r] 3 \\ 4 \end{pmatrix*} - \begin{pmatrix*}[r] 4 \\ 2 \end{pmatrix*} = \begin{pmatrix*}[r] -1 \\ 2 \end{pmatrix*}.

The residual vector is orthogonal to $\overline{\vphantom{b}u}$ , because

\overline{\vphantom{b}r}\cdot\overline{\vphantom{b}u} = (-1)(2) + (2)(1) = 0.

Example 10

Compute the residual vector for the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ by hand, and check that the residual is orthogonal to $\overline{\vphantom{b}u}$ , for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 1 \\ 3 \\ 8 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 3 \\ 0 \\ -1 \end{pmatrix*}.

ShowHide solution

Compute the projection as in Example 3, then subtract it from $\overline{\vphantom{b}v}$ :

x = \frac{1(3) + 3(0) + 8(-1)}{3(3) + 0(0) + (-1)(-1)} = \frac{-5}{10} = -0.5, \qquad x\overline{\vphantom{b}u} = -0.5\begin{pmatrix*}[r] 3 \\ 0 \\ -1 \end{pmatrix*} = \begin{pmatrix*}[r] -1.5 \\ 0 \\ 0.5 \end{pmatrix*},

\overline{\vphantom{b}r} = \begin{pmatrix*}[r] 1 \\ 3 \\ 8 \end{pmatrix*} - \begin{pmatrix*}[r] -1.5 \\ 0 \\ 0.5 \end{pmatrix*} = \begin{pmatrix*}[r] 2.5 \\ 3 \\ 7.5 \end{pmatrix*}.

The residual vector is orthogonal to $\overline{\vphantom{b}u}$ , because

\overline{\vphantom{b}r}\cdot\overline{\vphantom{b}u} = (2.5)(3) + (3)(0) + (7.5)(-1) = 0.

Example 11

Compute the residual vector for the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ in R, and check that the residual is orthogonal to $\overline{\vphantom{b}u}$ , for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 3 \\ -1 \\ 2 \\ 6 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 1 \\ 1 \\ -1 \\ 1 \end{pmatrix*}.

ShowHide solution

R computes the projection scalar, then the residual vector and its dot product with $\overline{\vphantom{b}u}$ . The line r = v - x * u is the hand step $\overline{\vphantom{b}r} = \overline{\vphantom{b}v} - x\overline{\vphantom{b}u}$ :

v = c(3, -1, 2, 6)
u = c(1, 1, -1, 1)
x = project(v ~ u)
r = v - x * u
r
[1]  1.5 -2.5  3.5  4.5
dot(r, u)
[1] 0

The residual vector is $\overline{\vphantom{b}r} = \left(\begin{smallmatrix} 1.5\\-2.5\\3.5\\4.5 \end{smallmatrix}\right)$ , and it is orthogonal to $\overline{\vphantom{b}u}$ , because dot(r, u) prints 0.

Next: 4. Sketching a Projection and Its Residual Lesson map

4. Sketching a Projection and Its Residual

Objective. Given two 2-D non-zero vectors, sketch the projection and residual by hand.

Sketching a projection and its residual.

Compute $x = \dfrac{\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}}{\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}}$ and $x\overline{\vphantom{b}u}$ .
Draw $\overline{\vphantom{b}u}$ from the origin, and draw the line through $\overline{\vphantom{b}u}$ , extended both ways.
Draw $\overline{\vphantom{b}v}$ from the origin.
Draw $x\overline{\vphantom{b}u}$ from the origin along the line. Its tip is the point where a segment from the tip of $\overline{\vphantom{b}v}$ meets the line at a right angle.
Draw $\overline{\vphantom{b}r}$ from the tip of $x\overline{\vphantom{b}u}$ to the tip of $\overline{\vphantom{b}v}$ .

Example 12

Sketch the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ and its residual, for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 4 \\ 3 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 1 \\ 2 \end{pmatrix*}.

ShowHide solution

The projection scalar and the two vectors to draw are

x = \frac{4(1) + 3(2)}{1(1) + 2(2)} = \frac{10}{5} = 2,

x\overline{\vphantom{b}u} = 2\begin{pmatrix*}[r] 1 \\ 2 \end{pmatrix*} = \begin{pmatrix*}[r] 2 \\ 4 \end{pmatrix*}, \qquad \overline{\vphantom{b}r} = \begin{pmatrix*}[r] 4 \\ 3 \end{pmatrix*} - \begin{pmatrix*}[r] 2 \\ 4 \end{pmatrix*} = \begin{pmatrix*}[r] 2 \\ -1 \end{pmatrix*}.

In the sketch, $\overline{\vphantom{b}r}$ meets the line through $\overline{\vphantom{b}u}$ at a right angle.

Example 13

Sketch the projection of $\overline{\vphantom{b}v}$ onto $\overline{\vphantom{b}u}$ and its residual, for

\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 0 \\ -4 \end{pmatrix*}, \qquad \overline{\vphantom{b}u} = \begin{pmatrix*}[r] 2 \\ 2 \end{pmatrix*}.

ShowHide solution

The projection scalar and the two vectors to draw are

x = \frac{0(2) + (-4)(2)}{2(2) + 2(2)} = \frac{-8}{8} = -1,

x\overline{\vphantom{b}u} = -1\begin{pmatrix*}[r] 2 \\ 2 \end{pmatrix*} = \begin{pmatrix*}[r] -2 \\ -2 \end{pmatrix*}, \qquad \overline{\vphantom{b}r} = \begin{pmatrix*}[r] 0 \\ -4 \end{pmatrix*} - \begin{pmatrix*}[r] -2 \\ -2 \end{pmatrix*} = \begin{pmatrix*}[r] 2 \\ -2 \end{pmatrix*}.

In the sketch, $x\overline{\vphantom{b}u}$ points opposite $\overline{\vphantom{b}u}$ , because $x$ is negative.

Lesson map