Linear-Algebra-Done-Right · Unit 17 · Video 3 · Interactive Practice

T†w: The Best Approximate Solution of Smallest Norm

IKey Formulas

Formula	Name	When equality holds
$\\|T(T^\dagger w) - w\\| \le \\|Tv - w\\|$ for every $v \in V$	Best fit	$v \in T^\dagger w + \operatorname{null} T$
$\\|T^\dagger w\\| \le \\|v\\|$ for every $v \in T^\dagger w + \operatorname{null} T$	Smallest norm	$v = T^\dagger w$
$\\|Tv - w\\|^2 = \\|Tv - TT^\dagger w\\|^2 + \\|TT^\dagger w - w\\|^2$	Pythagoras in $W$	Always: first piece $\in \operatorname{range} T$ , second $\in (\operatorname{range} T)^\perp$
$T^\dagger(x, y, z) = \tfrac{1}{11}(5x - 2y,\; 5x - 2y,\; x + 4y,\; -2x + 3y)$	The example	Always, for $T(a, b, c, d) = (a + b + c,\; 2c + d,\; 0)$

Key Insight: $T^\dagger w$ solves $Tv = P_{\operatorname{range} T}\, w$ , not $Tv = w$ — the two agree only when $w \in \operatorname{range} T$ . Among all solutions of the projected equation it is the one in $(\operatorname{null} T)^\perp$ , hence the shortest.

IIVisualization 1 — The Foot of the Perpendicular Is the Best Fit

For $T(x, y) = (x + y,\; x + y)$ on $\mathbf{R}^2$ , no point of $\operatorname{range} T$ is closer to $w$ than $T(T^\dagger w)$ .

IIIVisualization 2 — The Shortest Best Fit Lies in $(\operatorname{null} T)^\perp$

For the same $T$ and $w = (5, 1)$ , the best fits fill the line $T^\dagger w + \operatorname{null} T$ , and only $T^\dagger w$ is shortest.

IVVisualization 3 — The Video's Example in $\mathbf{F}^4$

Adding any $n \in \operatorname{null} T$ to $T^\dagger w$ leaves $\|Tv - w\|$ unchanged and adds exactly $\|n\|^2$ to $\|v\|^2$ .

w: x 11 w: y 0 w: z 7

b 0 c 0

💡 Challenge: with $w = (11, 0, 7)$ , find the $b$ and $c$ that give $v = (11, 0, 0, 0)$ , then check that $55 + \|n\|^2 = 121$ .

VQuiz Questions

Problem 1 · Apply the Formula

Given: $T \in \mathcal{L}(\mathbf{F}^4, \mathbf{F}^3)$ , $T(a, b, c, d) = (a + b + c,\; 2c + d,\; 0)$ , and $w = (0, 11, 5)$ — find $T^\dagger w$ .

✅ Correct!

T(-2, -2, 4, 3) = (0, 11, 0) = P_{\operatorname{range} T}\, w

, and

(-2, -2, 4, 3)

is orthogonal to both basis vectors of

\operatorname{null} T

❌ A best fit, but not $T^\dagger w$ . This vector does map to

(0, 11, 0)

, but it is not orthogonal to

(-1, 0, 1, -2)

, so it is not in

(\operatorname{null} T)^\perp

— and its norm is larger than

\sqrt{33}

❌ Check the signs.

T(2, 2, -4, -3) = (0, -11, 0)

, which is not the projection

(0, 11, 0)

Show solution

The formula $T^\dagger(x, y, z) = \tfrac{1}{11}(5x - 2y,\; 5x - 2y,\; x + 4y,\; -2x + 3y)$ with $x = 0$ , $y = 11$ gives

T^\dagger w = \tfrac{1}{11}(-22,\; -22,\; 44,\; 33) = (-2, -2, 4, 3).

Check. $T(-2, -2, 4, 3) = (-2 - 2 + 4,\; 8 + 3,\; 0) = (0, 11, 0)$ , the projection of $w$ onto $\operatorname{range} T$ . Orthogonality: $\langle (-2, -2, 4, 3), (-1, 1, 0, 0) \rangle = 2 - 2 = 0$ and $\langle (-2, -2, 4, 3), (-1, 0, 1, -2) \rangle = 2 + 4 - 6 = 0$ .

Options (a) and (b) also map to $(0, 11, 0)$ , so they are best fits too, but their norms are $11$ and $\sqrt{41}$ , both larger than $\|T^\dagger w\| = \sqrt{33}$ .

Problem 2 · A Best Fit That Is Not $T^\dagger w$

Given: the same $T$ , $w = (11, 0, 7)$ with $T^\dagger w = (5, 5, 1, -2)$ , and $v = (11, 0, 0, 0)$ — which statement is true?

✅ Correct!

v - T^\dagger w = (6, -5, -1, 2) \in \operatorname{null} T

, so

v

fits exactly as well as

T^\dagger w

; the second part of the theorem then says

\|T^\dagger w\| < \|v\|

❌ Fit and size are separate questions. Being a best fit depends only on

\|Tv - w\|

; here

Tv = (11, 0, 0) = T(T^\dagger w)

, so the fit is optimal regardless of

\|v\|

❌ Not quite.

Tv = (11, 0, 0) \ne (11, 0, 7)

; no vector solves

Tv = w

, because

w \notin \operatorname{range} T

❌ $T$ is not injective.

Tv = T(T^\dagger w)

only gives

v - T^\dagger w \in \operatorname{null} T

, and

\operatorname{null} T

is two-dimensional.

Show solution

$Tv = (11 + 0 + 0,\; 0 + 0,\; 0) = (11, 0, 0)$ , which equals $T(T^\dagger w) = T(5, 5, 1, -2) = (11, 0, 0)$ . So $\|Tv - w\| = \|(0, 0, -7)\| = 7$ , the minimum, and $v$ is a best fit.

The difference $n = v - T^\dagger w = (6, -5, -1, 2)$ lies in $\operatorname{null} T$ , while $T^\dagger w \in (\operatorname{null} T)^\perp$ . By Pythagoras,

\|v\|^2 = \|T^\dagger w\|^2 + \|n\|^2 = 55 + 66 = 121,

so $\|v\| = 11 > \sqrt{55} \approx 7.42$ . It is a best fit, but not the one of smallest norm.

Problem 3 · Fit Error and Norm

Given: the same $T$ and $w = (1, 8, -4)$ — find $T^\dagger w$ , the smallest possible value of $\|Tv - w\|$ , and $\|T^\dagger w\|$ .

What is $T^\dagger w$ ?

What are $\|T(T^\dagger w) - w\|$ and $\|T^\dagger w\|$ ?

✅ Correct!

T^\dagger w = (-1, -1, 3, 2)

maps to

(1, 8, 0)

, which misses

w

4

in the third coordinate, and

\|T^\dagger w\| = \sqrt{15}

is the smallest norm among all best fits.

❌ A best fit, but not $T^\dagger w$ . This vector maps to

(1, 8, 0)

, but it is not orthogonal to both

(-1, 1, 0, 0)

and

(-1, 0, 1, -2)

❌ Check the signs.

T(1, 1, -3, -2) = (-1, -8, 0)

, not

(1, 8, 0)

❌ $w$ is not in $\operatorname{range} T$ . Its third coordinate is

-4 \ne 0

, so

\|T(T^\dagger w) - w\| = |-4| \ne 0

❌ Squared versus unsquared.

\|(0, 0, 4)\| = 4

and

\|T^\dagger w\|^2 = 15

, so

\|T^\dagger w\| = \sqrt{15}

❌ Check $T^\dagger w$ . Substitute

x = 1

y = 8

into the formula.

❌ Check the two lengths. Compute

T(T^\dagger w) - w

first.

Show solution

Step 1. With $x = 1$ , $y = 8$ :

T^\dagger w = \tfrac{1}{11}(5 - 16,\; 5 - 16,\; 1 + 32,\; -2 + 24) = \tfrac{1}{11}(-11, -11, 33, 22) = (-1, -1, 3, 2).

Step 2. $T(-1, -1, 3, 2) = (-1 - 1 + 3,\; 6 + 2,\; 0) = (1, 8, 0)$ , so $T(T^\dagger w) - w = (0, 0, 4)$ and the smallest possible value of $\|Tv - w\|$ is $4$ .

Step 3. $\|T^\dagger w\|^2 = 1 + 1 + 9 + 4 = 15$ , so $\|T^\dagger w\| = \sqrt{15}$ .

Options (a) and (c) in the first group are $T^\dagger w + n$ with $n = (-1, 1, 0, 0)$ and $n = (1, 1, -2, 4)$ ; their squared norms are $15 + 2 = 17$ and $15 + 22 = 37$ .

Problem 4 · The Surjective Case

Given: $S \in \mathcal{L}(\mathbf{R}^4, \mathbf{R}^2)$ is surjective and $w \in \mathbf{R}^2$ — which statement is true?

✅ Correct! Surjectivity makes the projection onto

\operatorname{range} S = \mathbf{R}^2

the identity, so every best fit is an exact solution, and

S^\dagger w

is the shortest of them.

❌ The first half is right, the second is not.

\operatorname{range} S = \mathbf{R}^2

, so projecting onto it changes nothing:

S(S^\dagger w) = w

❌ $S$ has no inverse.

\dim \operatorname{null} S = 4 - 2 = 2

, so

S

is not injective;

S^\dagger = S^{-1}

only when

S

is invertible.

❌ Too many solutions. The solutions form

S^\dagger w + \operatorname{null} S

, a plane in

\mathbf{R}^4

;

S^\dagger w

is just its shortest member.

Show solution

Since $S$ is surjective, $\operatorname{range} S = \mathbf{R}^2$ and $P_{\operatorname{range} S} = I$ . Hence $S(S^\dagger w) = P_{\operatorname{range} S}\, w = w$ : the best fit is an exact solution.

By the dimension formula, $\dim \operatorname{null} S = 4 - 2 = 2$ , so $S$ is not injective and $Sv = w$ has infinitely many solutions, namely $S^\dagger w + \operatorname{null} S$ . They are exactly the best fits (each has $\|Sv - w\| = 0$ ), and by the second part of the theorem $S^\dagger w$ is the unique one of smallest norm.

$S^\dagger = S^{-1}$ would require $S$ to be invertible, which is impossible from $\mathbf{R}^4$ to $\mathbf{R}^2$ .

Solved: 0 / 4

T†w: The Best Approximate Solution of Smallest Norm

IKey Formulas

IIVisualization 1 — The Foot of the Perpendicular Is the Best Fit

IIIVisualization 2 — The Shortest Best Fit Lies in (null⁡T)⊥(\operatorname{null} T)^\perp(nullT)⊥

IVVisualization 3 — The Video's Example in F4\mathbf{F}^4F4

VQuiz Questions

IIIVisualization 2 — The Shortest Best Fit Lies in $(\operatorname{null} T)^\perp$

IVVisualization 3 — The Video's Example in $\mathbf{F}^4$