LINEAR-ALGEBRA · Interactive Practice | Unit 28 · Video 4

The Perfect Basis: SVD and the Four Fundamental Subspaces

IKey Formulas

Formula	Name	Description
$A v_i = \sigma_i u_i$	SVD pairing	Matrix sends $v_i$ to scaled $u_i$
$A^T A\, v_i = \lambda_i v_i$	Right singular vectors	$v_i$ are eigenvectors of $A^T A$
$\sigma_i = \sqrt{\lambda_i}$	Singular values	Square roots of $A^T A$ eigenvalues
$u_i = \dfrac{A v_i}{\sigma_i}$	Left singular vectors	Automatically orthonormal in column space
$A = U \Sigma V^T$	Full SVD	Diagonalizes $A$ across two spaces

IIVisualization 1 — Orthonormal Pairs Under $A$

A perpendicular input pair usually lands non-perpendicular after $A$ — except at two special angles.

Input angle θ 20°

input · unit circle (drag q₁)

output · multiply by A

💡 The two input angles whose output stays perpendicular are the right singular vectors $v_1, v_2$ .

IIIVisualization 2 — The Unit Circle Becomes an Ellipse

$A$ sends the unit circle to an ellipse whose semi-axes $\sigma_1 u_1, \sigma_2 u_2$ are the images of $v_1, v_2$ .

A₁₁ 3 A₁₂ 0 A₂₁ 4 A₂₂ 5

input · unit circle + v₁,v₂

output · ellipse + σ₁u₁,σ₂u₂

💡 Drive the rows nearly parallel so $\sigma_2 \to 0$ : the ellipse flattens to a segment — a rank-1, singular matrix.

IVVisualization 3 — Three Steps of $A = U \Sigma V^T$

$A$ acts in three moves: rotate by $V^T$ , stretch by $\Sigma$ , rotate by $U$ .

Step 0

Stage 0 — Start

Unit circle carrying the orthonormal pair

v_1, v_2

(right singular vectors).

Stage 1 — Apply $V^T$

Rotation sending

v_i \to e_i

; a circle stays a circle.

Stage 2 — Apply $\Sigma$

Stretch axis

i

\sigma_i

(

\sigma_1 = 3\sqrt{5} \approx 6.71

\sigma_2 = \sqrt{5} \approx 2.24

): an axis-aligned ellipse.

Stage 3 — Apply $U$

Rotation sending

e_i \to u_i

, so

A v_i = \sigma_i u_i

: the final tilted ellipse.

VQuiz Questions

Question 1

For an $m \times n$ matrix $A$ of rank $r$ , the SVD produces vectors $v_1, \ldots, v_n$ and $u_1, \ldots, u_m$ . Which statement correctly identifies the four fundamental subspaces?

✅ Correct! The first $r$ $v$ 's sit in the row space, the first $r$ $u$ 's sit in the column space.

❌ Not quite. Remember: $v$ 's live in $\mathbb{R}^n$ (input side, row/null space) and $u$ 's live in $\mathbb{R}^m$ (output side, column/left-null space). The first $r$ of each are the active directions.

Show solution

Solution:

The first $r$ singular vectors give orthonormal bases for the spaces $A$ actually "uses":

$v_1, \ldots, v_r$ span the row space (in $\mathbb{R}^n$ )
$v_{r+1}, \ldots, v_n$ span the null space (in $\mathbb{R}^n$ )
$u_1, \ldots, u_r$ span the column space (in $\mathbb{R}^m$ )
$u_{r+1}, \ldots, u_m$ span the left null space (in $\mathbb{R}^m$ )

This is why people call the SVD the "perfect" factorization: one decomposition gives orthonormal bases for all four fundamental subspaces simultaneously.

Question 2

Suppose $A$ is a matrix with $A^T A$ having eigenvalues $\lambda_1 = 45$ and $\lambda_2 = 5$ . What are the singular values $\sigma_1$ and $\sigma_2$ of $A$ ?

✅ Correct! $\sigma_i = \sqrt{\lambda_i}$ — singular values are square roots of the $A^T A$ eigenvalues.

❌ Not quite. A common error is to forget the square root. The eigenvalues of $A^T A$ measure $\|A v_i\|^2$ , not $\|A v_i\|$ .

Show solution

Solution:

Singular values are the square roots of the eigenvalues of $A^T A$ : $\sigma_i = \sqrt{\lambda_i}$

So: $\sigma_1 = \sqrt{45} = 3\sqrt{5}, \qquad \sigma_2 = \sqrt{5}$

Why square roots? Take $A v_i = \sigma_i u_i$ and compute the squared length: $\|A v_i\|^2 = (A v_i)^T (A v_i) = v_i^T A^T A v_i = \lambda_i v_i^T v_i = \lambda_i$ So $\|A v_i\| = \sqrt{\lambda_i}$ , which equals $\sigma_i$ since $u_i$ is a unit vector.

Question 3

True or False: Once you choose $v_i$ as eigenvectors of $A^T A$ , the resulting $u_i = A v_i / \sigma_i$ are automatically orthogonal in the column space — no separate Gram-Schmidt step is needed.

✅ Correct! The eigenvector orthogonality of $A^T A$ forces $(Av_i) \cdot (Av_j) = \lambda_j v_i^T v_j = 0$ automatically.

❌ Not quite. Try the computation $(Av_i)^T(Av_j) = v_i^T A^T A v_j$ . Since $v_j$ is an eigenvector of $A^T A$ , this simplifies to $\lambda_j v_i^T v_j$ , which is zero when $i \neq j$ .

Show solution

Solution:

True. This is the entire reason the SVD works so cleanly. For $i \neq j$ :

$(A v_i)^T (A v_j) = v_i^T A^T A v_j = v_i^T (\lambda_j v_j) = \lambda_j (v_i^T v_j) = 0$

The orthogonality of the $v$ 's as eigenvectors of $A^T A$ automatically forces orthogonality of $A v_i$ and $A v_j$ in the column space. Dividing by $\sigma_i$ just normalizes them to unit length:

$u_i = \frac{A v_i}{\sigma_i}$

So the SVD gives you "orthogonality that survives multiplication by $A$ " for free — no Gram-Schmidt required.

Question 4

For the matrix $A = \begin{bmatrix} 3 & 0 \\ 4 & 5 \end{bmatrix}$ , the right singular vector $v_1 = \tfrac{1}{\sqrt{2}}(1, 1)^T$ and $\sigma_1 = 3\sqrt{5}$ . What is the corresponding left singular vector $u_1$ ?

✅ Correct! $A v_1 = (1/\sqrt{2})(3, 9)^T$ , and dividing by $\sigma_1 = 3\sqrt{5}$ gives $(1/\sqrt{10})(1, 3)^T$ .

❌ Not quite. Carefully compute $A v_1$ first, then divide by $\sigma_1$ . Watch out for swapping coordinates or sign errors.

Show solution

Solution:

Use $u_1 = \dfrac{A v_1}{\sigma_1}$ .

Step 1. Compute $A v_1$ : $A v_1 = \begin{bmatrix} 3 & 0 \\ 4 & 5 \end{bmatrix} \cdot \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ 1 \end{bmatrix} = \frac{1}{\sqrt{2}}\begin{bmatrix} 3 \\ 9 \end{bmatrix}$

Step 2. Divide by $\sigma_1 = 3\sqrt{5}$ : $u_1 = \frac{1}{3\sqrt{5}} \cdot \frac{1}{\sqrt{2}}\begin{bmatrix} 3 \\ 9 \end{bmatrix} = \frac{1}{\sqrt{10}}\begin{bmatrix} 1 \\ 3 \end{bmatrix}$

Check. $\|u_1\| = \sqrt{(1/\sqrt{10})^2 + (3/\sqrt{10})^2} = \sqrt{1/10 + 9/10} = 1$ . Unit vector as expected.

Solved: 0 / 4

The Perfect Basis: SVD and the Four Fundamental Subspaces

IKey Formulas

IIVisualization 1 — Orthonormal Pairs Under AAA

IIIVisualization 2 — The Unit Circle Becomes an Ellipse

IVVisualization 3 — Three Steps of A=UΣVTA = U \Sigma V^TA=UΣVT

VQuiz Questions

IIVisualization 1 — Orthonormal Pairs Under $A$

IVVisualization 3 — Three Steps of $A = U \Sigma V^T$