LINEAR-ALGEBRA · Unit 15 · Video 1 · Interactive Practice

How One Matrix Splits Every Vector Into Two Perfect Pieces

IKey Formulas

Formula	Name	When it matters
$P = A(A^\top A)^{-1}A^\top$	Projection matrix	Projects any $\mathbf{b}$ onto $C(A)$
$\mathbf{b} = P\mathbf{b} + (I - P)\mathbf{b}$	Orthogonal decomposition	Splits $\mathbf{b}$ into two $\perp$ pieces
$P^2 = P$ , $P^\top = P$	Idempotent & symmetric	Both required for orthogonal projection
$(I-P)^2 = I-P$ , $(I-P)^\top = I-P$	Complement properties	$I-P$ projects onto $N(A^\top)$

IIVisualization 1 — The Orthogonal Split: p + e = b

Every $\mathbf{b}$ breaks into $\mathbf{p} = P\mathbf{b}$ inside $C(A)$ plus $\mathbf{e} = (I-P)\mathbf{b}$ perpendicular to it.

Subspace angle θ 30°

IIIVisualization 2 — Energy Shifts Between the Two Extremes

As $\mathbf{b}$ rotates from $C(A)$ toward $N(A^\top)$ , the energy $\|\mathbf{b}\|^2$ drains from one component into the other.

Angle α from C(A) 30°

💡 Challenge: find the angle where the two energies are equal.

IVVisualization 3 — Idempotency: Project Once, Project Forever

Projecting again changes nothing — $P^2 = P$ — so the fan freezes after the first application.

Projection Applications k 0

VQuiz Questions

Question 1

If $\mathbf{b}$ is already in the column space $C(A)$ — meaning $\mathbf{b} = A\mathbf{x}$ for some vector $\mathbf{x}$ — what is $P\mathbf{b}$ ?

✅ Correct! A vector already in the column space is unchanged by projection.

❌ Not quite. Think about what happens when the vector is already lying flat on the subspace — does its shadow differ from itself?

Show solution

Solution:

Since $\mathbf{b} = A\mathbf{x}$ , substitute into the projection formula:

$P\mathbf{b} = A(A^\top A)^{-1}A^\top (A\mathbf{x}) = A \cdot \underbrace{(A^\top A)^{-1}(A^\top A)}_{= I} \cdot \mathbf{x} = A\mathbf{x} = \mathbf{b}$

The inverse cancels with its matrix, leaving the identity. Projecting a vector that's already in the column space returns the vector unchanged — the shadow perfectly matches the original.

Question 2

True or False: Every idempotent matrix ( $M^2 = M$ ) is an orthogonal projection matrix.

✅ Correct! Idempotency alone gives a projection, but you also need symmetry ( $M^\top = M$ ) for it to be orthogonal.

❌ Not quite. Recall from the video: what additional property (besides $M^2 = M$ ) is needed to ensure the projection is orthogonal?

Show solution

Solution: False

An orthogonal projection matrix must satisfy two properties:

Idempotent: $M^2 = M$ (projecting twice = projecting once)
Symmetric: $M^\top = M$ (the decomposition is into perpendicular pieces)

A non-symmetric idempotent matrix produces an oblique projection — it still splits vectors into two components, but those components are not perpendicular to each other. Both conditions together are necessary for an orthogonal projection.

Question 3

Suppose $P\mathbf{b} = \begin{pmatrix} 3 \\ 1 \\ 2 \end{pmatrix}$ and $\mathbf{b} = \begin{pmatrix} 4 \\ 3 \\ 1 \end{pmatrix}$ . What is $(I - P)\mathbf{b}$ ?

✅ Correct! The perpendicular component is simply b − Pb.

❌ Not quite. Use the decomposition: (I − P)b = b − Pb. Subtract component by component.

Show solution

Solution:

From the decomposition $\mathbf{b} = P\mathbf{b} + (I-P)\mathbf{b}$ , rearrange:

$(I-P)\mathbf{b} = \mathbf{b} - P\mathbf{b} = \begin{pmatrix} 4 \\ 3 \\ 1 \end{pmatrix} - \begin{pmatrix} 3 \\ 1 \\ 2 \end{pmatrix} = \begin{pmatrix} 1 \\ 2 \\ -1 \end{pmatrix}$

The perpendicular component is simply $\mathbf{b}$ minus its projection — no need to know $P$ explicitly!

Why the other options are wrong:

$(7, 4, 3)$ : This is $\mathbf{b} + P\mathbf{b}$ (added instead of subtracted)
$(3, 1, 2)$ : This is $P\mathbf{b}$ itself, not the perpendicular part
$(0, 0, 0)$ : This would mean $\mathbf{b} \in C(A)$ , but $\mathbf{b} \neq P\mathbf{b}$ here

Question 4

The matrix $I - P$ is itself a projection matrix. Which subspace does $I - P$ project onto?

✅ Correct! I − P projects onto the orthogonal complement of C(A), which is the left null space N(Aᵀ).

❌ Not quite. Think about which vectors satisfy (I − P)b = b. These are vectors that P sends to zero — where do they live?

Show solution

Solution:

$I - P$ projects onto $N(A^\top)$ , the left null space — the orthogonal complement of the column space.

Why? If $\mathbf{b} \in N(A^\top)$ , then $A^\top \mathbf{b} = \mathbf{0}$ . Substituting into the projection formula:

$P\mathbf{b} = A(A^\top A)^{-1} \underbrace{A^\top \mathbf{b}}_{= \mathbf{0}} = \mathbf{0}$

So $(I - P)\mathbf{b} = \mathbf{b} - \mathbf{0} = \mathbf{b}$ . The matrix $I - P$ preserves vectors in $N(A^\top)$ and annihilates vectors in $C(A)$ — the exact mirror image of what $P$ does.

Together, $P$ and $I - P$ give a complete orthogonal decomposition:

$P \to C(A)$
$I - P \to N(A^\top)$

Solved: 0 / 4