Lesson 20: Dot Product · Tangent Space

Objectives

Given two vectors with the same number of components, find their dot product by hand and in R.
Describe a vector’s length as the square root of its dot product with itself.
Use the dot product to determine whether two vectors are orthogonal.

Dot Product and Orthogonality Open full screen

1. The Dot Product

Objective. Given two vectors with the same number of components, find their dot product by hand and in R.

Definition 1 (Dot product).

For two vectors $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ with the same number of components, the dot product $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v}$ is the scalar obtained by multiplying each component of $\overline{\vphantom{b}u}$ by the corresponding component of $\overline{\vphantom{b}v}$ and summing the products:

\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} = \begin{pmatrix*}[r] u_1 \\ u_2 \\ \vdots \\ u_n \end{pmatrix*}\cdot \begin{pmatrix*}[r] v_1 \\ v_2 \\ \vdots \\ v_n \end{pmatrix*} = u_1v_1 + u_2v_2 + \dots + u_nv_n.

The vectors $\left(\begin{smallmatrix} 1\\-2\\2 \end{smallmatrix}\right)$ and $\left(\begin{smallmatrix} 3\\4 \end{smallmatrix}\right)$ have no dot product, because the first has three components and the second has two. The definition requires the same number of components, so that each component of $\overline{\vphantom{b}u}$ pairs with one component of $\overline{\vphantom{b}v}$ .

Proposition 2 (Order and scalars).

Let $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ be vectors with the same number of components, and let $m$ be a scalar. The dot product gives the same scalar in either order of the two vectors, and a scalar multiple of either vector can be moved outside the product:

\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} = \overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}, \qquad\qquad (m\overline{\vphantom{b}u})\cdot\overline{\vphantom{b}v} = \overline{\vphantom{b}u}\cdot(m\overline{\vphantom{b}v}) = m(\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v}).

Example 1

Find the dot product:

\begin{pmatrix*}[r] 1 \\ -2 \\ 2 \end{pmatrix*}\cdot\begin{pmatrix*}[r] 4 \\ 0 \\ -1 \end{pmatrix*}.

ShowHide solution

By the definition of the dot product, the dot product is the sum of the products of corresponding components:

(1)(4) + (-2)(0) + (2)(-1) = 4 + 0 - 2 = 2.

Example 2

Find the dot product from MMAC 3.4 Example 2(b):

\begin{pmatrix*}[r] 3 \\ 1 \\ -2 \\ 0 \end{pmatrix*}\cdot\begin{pmatrix*}[r] -2 \\ 4 \\ 5 \\ 3 \end{pmatrix*}.

ShowHide solution

The four products of corresponding components sum to

(3)(-2) + (1)(4) + (-2)(5) + (0)(3) = -6 + 4 - 10 + 0 = -12.

Example 3

For the vectors

\overline{\vphantom{b}u} = \begin{pmatrix*}[r] 2 \\ 5 \end{pmatrix*}, \quad \overline{\vphantom{b}v} = \begin{pmatrix*}[r] -3 \\ 1 \end{pmatrix*},

compute $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v}$ , $\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u}$ , and $(2\overline{\vphantom{b}u})\cdot\overline{\vphantom{b}v}$ .

ShowHide solution

Computing each dot product from the components gives

\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} = (2)(-3) + (5)(1) = -6 + 5 = -1, \qquad \overline{\vphantom{b}v}\cdot\overline{\vphantom{b}u} = (-3)(2) + (1)(5) = -6 + 5 = -1,

(2\overline{\vphantom{b}u})\cdot\overline{\vphantom{b}v} = \begin{pmatrix*}[r] 4 \\ 10 \end{pmatrix*}\cdot\begin{pmatrix*}[r] -3 \\ 1 \end{pmatrix*} = (4)(-3) + (10)(1) = -12 + 10 = -2.

The last product equals $2(\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v})$ :

2(\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v}) = 2(-1) = -2,

as the order and scalars proposition states.

In R, the command dot(u, v) returns the dot product $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v}$ of two stored vectors.

Example 4

Find the dot product of $\overline{\vphantom{b}a} = \left(\begin{smallmatrix} 1\\-2\\2 \end{smallmatrix}\right)$ and $\overline{\vphantom{b}b} = \left(\begin{smallmatrix} 4\\0\\-1 \end{smallmatrix}\right)$ in R.

ShowHide solution

Store the two vectors, and compute the dot product three ways: with dot(), with sum(a * b), and with dot() in the other order.

a = c(1, -2, 2)
b = c(4, 0, -1)
dot(a, b)
[1] 2
sum(a * b)
[1] 2
dot(b, a)
[1] 2

The expression a * b multiplies corresponding components of a and b, and sum() adds the products. All three lines give $\overline{\vphantom{b}a}\cdot\overline{\vphantom{b}b} = 2$ , the value Example 1 finds by hand.

Vectors with different numbers of components in R. The vectors $\left(\begin{smallmatrix} 1\\-2\\2 \end{smallmatrix}\right)$ and $\left(\begin{smallmatrix} 3\\4 \end{smallmatrix}\right)$ have no dot product. R still returns a number, because dot() reuses the vector with fewer components, starting again from its first component:

> dot(c(1, -2, 2), c(3, 4))
[1] 1
Warning message:
In u * v : longer object length is not a multiple of shorter object length
> dot(c(1, 2), c(3, 0, -1, 5))
[1] 12

In the warning, u and v are the names dot() gives its two inputs, and “length” is R’s word for the number of components. The second call prints no warning, because 4 is a multiple of 2, yet both numbers are meaningless. Count the components of both vectors before calling dot().

Next: 2. Length as a Dot Product Lesson map

2. Length as a Dot Product

Objective. Describe a vector’s length as the square root of its dot product with itself.

Lesson 15 defined the length of a vector as $\|\overline{\vphantom{b}v}\| = \sqrt{v_1^2 + v_2^2 + \dots + v_n^2}$ . The dot product of a vector with itself is the sum under that square root:

\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}v} = v_1v_1 + v_2v_2 + \dots + v_nv_n = v_1^2 + v_2^2 + \dots + v_n^2.

For $\overline{\vphantom{b}v} = \left(\begin{smallmatrix} 3\\4 \end{smallmatrix}\right)$ , the dot product of $\overline{\vphantom{b}v}$ with itself is

\overline{\vphantom{b}v}\cdot\overline{\vphantom{b}v} = (3)(3) + (4)(4) = 9 + 16 = 25,

and $\sqrt{25} = 5$ is the length of the hypotenuse of the right triangle with legs 3 and 4 in the figure.

Proposition 3 (Length as a dot product).

The length of a vector $\overline{\vphantom{b}u}$ is

\|\overline{\vphantom{b}u}\| = \sqrt{\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}}.

Example 5

Use the dot product to find the length of

\overline{\vphantom{b}a} = \begin{pmatrix*}[r] 1 \\ -2 \\ 2 \end{pmatrix*}

by hand and in R.

ShowHide solution

The length of $\overline{\vphantom{b}a}$ is the square root of $\overline{\vphantom{b}a}\cdot\overline{\vphantom{b}a}$ :

\|\overline{\vphantom{b}a}\| = \sqrt{\overline{\vphantom{b}a}\cdot\overline{\vphantom{b}a}} = \sqrt{(1)(1) + (-2)(-2) + (2)(2)} = \sqrt{9} = 3.

R gives the same dot product and length:

a = c(1, -2, 2)
dot(a, a)
[1] 9
sqrt(dot(a, a))
[1] 3

Example 6

Find the length of $\overline{\vphantom{b}u} = \left(\begin{smallmatrix} 2\\-1\\3\\1 \end{smallmatrix}\right)$ in R.

ShowHide solution

Compute $\sqrt{\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}u}}$ in R, as in Example 5:

u = c(2, -1, 3, 1)
sqrt(dot(u, u))
[1] 3.872983

By hand, the length is the square root of the sum of the squared components:

\|\overline{\vphantom{b}u}\| = \sqrt{(2)^2 + (-1)^2 + (3)^2 + (1)^2} = \sqrt{4 + 1 + 9 + 1} = \sqrt{15} \approx 3.873.

Example 7

The table gives monthly active Twitter users worldwide in millions, by quarter (MMAC 3.3 Example 4). Year counts years since 2000.

Year	11	11.25	11.5	11.75	12.75
Users	68	85	101	117	185

The line $\text{Users} = m\,\text{Year} + b$ through the first two data points does not pass through the other three. Find the line’s $m$ and $b$ in R, subtract the line’s outputs from the observed Users, and find the length of the resulting vector.

ShowHide solution

The problem has three stages: find $m$ and $b$ , subtract the line’s outputs from Users, and take the length of the difference. The $m$ and $b$ of the line through the first two points solve the vector equation built from those two points:

m\begin{pmatrix*}[r] 11 \\ 11.25 \end{pmatrix*} + b\begin{pmatrix*}[r] 1 \\ 1 \end{pmatrix*} = \begin{pmatrix*}[r] 68 \\ 85 \end{pmatrix*}.

In R, solve the matching $2 \times 2$ matrix equation:

u1 = c(11, 11.25)
u2 = c(1, 1)
A = matrix(c(u1, u2), nrow = 2, ncol = 2)
solve(A, c(68, 85))
[1]   68 -680

The line is $\text{Users} = 68\,\text{Year} - 680$ . The miss vector is the observed Users minus the line’s output at each Year. The line’s outputs at the last three Years are

68(11.5) - 680 = 102, \qquad 68(11.75) - 680 = 119, \qquad 68(12.75) - 680 = 187,

and the miss vector is

\overline{\vphantom{b}\text{miss}} = \begin{pmatrix*}[r] 68 \\ 85 \\ 101 \\ 117 \\ 185 \end{pmatrix*} - \begin{pmatrix*}[r] 68 \\ 85 \\ 102 \\ 119 \\ 187 \end{pmatrix*} = \begin{pmatrix*}[r] 0 \\ 0 \\ -1 \\ -2 \\ -2 \end{pmatrix*}.

The first two components are 0, because the line passes through the first two points. The last three are negative, because those points sit below the line. The length of the miss vector is

\|\overline{\vphantom{b}\text{miss}}\| = \sqrt{\overline{\vphantom{b}\text{miss}}\cdot\overline{\vphantom{b}\text{miss}}} = \sqrt{0^2 + 0^2 + (-1)^2 + (-2)^2 + (-2)^2} = \sqrt{0 + 0 + 1 + 4 + 4} = \sqrt{9} = 3.

R computes the same miss vector and length:

Year = c(11, 11.25, 11.5, 11.75, 12.75)
Users = c(68, 85, 101, 117, 185)
miss = Users - (68*Year - 680)
miss
[1]  0  0 -1 -2 -2
sqrt(dot(miss, miss))
[1] 3

The length 3 combines the misses of 1, 2, and 2 million users at the last three points into one number for the whole dataset.

Next: 3. Orthogonal Vectors Lesson map

3. Orthogonal Vectors

Objective. Use the dot product to determine whether two vectors are orthogonal.

Definition 4 (Orthogonal).

Two vectors $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ are orthogonal when $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} = 0$ , where “orthogonal” is an extension of the two-dimensional idea of “perpendicular” to any number of dimensions.

The vectors $\left(\begin{smallmatrix} 1\\2 \end{smallmatrix}\right)$ and $\left(\begin{smallmatrix} 2\\1 \end{smallmatrix}\right)$ are not orthogonal, because their dot product is $(1)(2) + (2)(1) = 4$ , and the definition requires a dot product of 0.

Example 8

Determine whether $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ are orthogonal:

\overline{\vphantom{b}u} = \begin{pmatrix*}[r] 1 \\ 2 \end{pmatrix*}, \quad \overline{\vphantom{b}v} = \begin{pmatrix*}[r] -1 \\ 1/2 \end{pmatrix*}.

ShowHide solution

By the definition of orthogonal, the vectors are orthogonal when their dot product is 0. The dot product is

\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} = (1)(-1) + (2)\left(\tfrac{1}{2}\right) = -1 + 1 = 0.

The vectors are orthogonal, because $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} = 0$ . Drawn as arrows from the origin, $\overline{\vphantom{b}u}$ ends at $(1, 2)$ and $\overline{\vphantom{b}v}$ ends at $\left(-1, \tfrac{1}{2}\right)$ , and the two arrows meet at a right angle.

Example 9

Determine which pairs of the vectors $\overline{\vphantom{b}u} = \left(\begin{smallmatrix} 2\\-1\\3\\1 \end{smallmatrix}\right)$ , $\overline{\vphantom{b}v} = \left(\begin{smallmatrix} 1\\1\\1\\1 \end{smallmatrix}\right)$ , and $\overline{\vphantom{b}w} = \left(\begin{smallmatrix} 2\\2\\1\\-5 \end{smallmatrix}\right)$ from MMAC 3.4 Example 4 are orthogonal, by hand and in R.

ShowHide solution

The three dot products are

\begin{align*} \overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} &= (2)(1) + (-1)(1) + (3)(1) + (1)(1) = 2 - 1 + 3 + 1 = 5,\\ \overline{\vphantom{b}u}\cdot\overline{\vphantom{b}w} &= (2)(2) + (-1)(2) + (3)(1) + (1)(-5) = 4 - 2 + 3 - 5 = 0,\\ \overline{\vphantom{b}v}\cdot\overline{\vphantom{b}w} &= (1)(2) + (1)(2) + (1)(1) + (1)(-5) = 2 + 2 + 1 - 5 = 0. \end{align*}

The vector $\overline{\vphantom{b}w}$ is orthogonal to both $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ , because both dot products are 0. The vectors $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ are not orthogonal, because $\overline{\vphantom{b}u}\cdot\overline{\vphantom{b}v} = 5$ . R gives the same three dot products:

u = c(2, -1, 3, 1)
v = c(1, 1, 1, 1)
w = c(2, 2, 1, -5)
dot(u, v)
[1] 5
dot(u, w)
[1] 0
dot(v, w)
[1] 0

Example 10

Find $z$ so that $\overline{\vphantom{b}u}$ and $\overline{\vphantom{b}v}$ are orthogonal:

\overline{\vphantom{b}u} = \begin{pmatrix*}[r] 4 \\ -3 \\ 1 \end{pmatrix*}, \quad \overline{\vphantom{b}v} = \begin{pmatrix*}[r] z \\ 3 \\ 5 \end{pmatrix*}.

ShowHide solution

Set the dot product equal to 0 and solve for $z$ :

(4)(z) + (-3)(3) + (1)(5) = 4z - 9 + 5 = 4z - 4 = 0, \qquad 4z = 4, \qquad z = 1.

As a check, with $z = 1$ the dot product is $(4)(1) - 9 + 5 = 0$ .

Lesson map