LINEAR-ALGEBRA · Interactive Practice

SVD in Action: When Singular Values Vanish (and Why Signs Matter)

IKey Formulas Reference

Formula	Name	Description
$A = U \Sigma V^\top$	SVD factorization	Orthonormal $U, V$ ; diagonal $\Sigma \geq 0$
$A^\top A \, v_i = \sigma_i^2 \, v_i$	Right singular vectors	Eigenvectors of $A^\top A$ ; eigenvalues are $\sigma_i^2$
$A v_i = \sigma_i u_i$	Sign-locked $U$	Defines $u_i$ from $v_i$ — fixes the sign
$\text{rank}(A) = \#\{\sigma_i > 0\}$	Rank from spectrum	Zero singular values $\Leftrightarrow$ null space

IIVisualization 1 — The Unit Circle Becomes an Ellipse

A matrix stretches the unit circle into an ellipse whose semi-axis lengths are its singular values.

σ₂ 4.24

When $\sigma_2 = 0$ the input direction $v_2$ maps to the origin — the ellipse degenerates to the column space of $A$ , a single line.

IIIVisualization 2 — The Four Fundamental Subspaces

SVD hands you orthonormal bases for all four fundamental subspaces at once — sorted by which singular values vanish.

Matrix

The number of zero singular values equals the dimension of the null space — and of the left null space.

IVVisualization 3 — Why Signs Matter in U

The equation $A v_i = \sigma_i u_i$ fixes the sign of $u_i$ ; the wrong sign breaks the factorization.

Sign of u₂

The wrong-sign $u_2$ is still a unit eigenvector of $AA^\top$ — always anchor $U$ through $A v_i = \sigma_i u_i$ , not $AA^\top$ alone.

VQuiz Questions

Question 1

For the matrix $A = \begin{bmatrix} 4 & 3 \\ 8 & 6 \end{bmatrix}$ , we computed $A^\top A = \begin{bmatrix} 80 & 60 \\ 60 & 45 \end{bmatrix}$ .

Its eigenvalues are $125$ and $0$ . What are the singular values of $A$ ?

✅ Correct! Singular values are square roots of $A^\top A$ 's eigenvalues, and $\sqrt{0} = 0$ .

❌ Not quite. Remember: $\sigma_i = \sqrt{\lambda_i}$ , not $\lambda_i$ itself.

Show solution

Solution:

The eigenvalues of $A^\top A$ are the squared singular values: $\lambda_i = \sigma_i^2 \quad\Longrightarrow\quad \sigma_i = \sqrt{\lambda_i}.$

So $\sigma_1 = \sqrt{125}$ and $\sigma_2 = \sqrt{0} = 0$ . A zero singular value is perfectly well-defined — it's just telling you the matrix has a 1-dimensional null space.

Question 2

In Example 1, we found $v_2 = \tfrac{1}{\sqrt{2}}(1, -1)$ and $\sigma_2 = \sqrt{18}$ . Computing $A v_2$ gave $\tfrac{1}{\sqrt{2}}(0, -6)$ .

A student writes $u_2 = (0, 1)$ because that's a unit eigenvector of $A A^\top$ . Is this a valid SVD column?

✅ Correct! Signs in $U$ are locked by $A v_i = \sigma_i u_i$ , not free.

❌ Not quite. Both $(0, 1)$ and $(0, -1)$ are unit eigenvectors of $AA^\top$ , but only one satisfies the SVD constraint.

Show solution

Solution:

We need $A v_2 = \sigma_2 u_2$ . With $\sigma_2 = \sqrt{18}$ and $A v_2 = \tfrac{1}{\sqrt{2}}(0, -6)$ :

$u_2 = \frac{A v_2}{\sigma_2} = \frac{(0, -6/\sqrt{2})}{\sqrt{18}} = (0, -1).$

The vector $(0, 1)$ is also a unit eigenvector of $A A^\top$ (eigenvectors come in $\pm$ pairs), but it gives the wrong sign — assembling $U \Sigma V^\top$ with $u_2 = (0, 1)$ does not reproduce $A$ .

❌ Wrong: $u_2 = (0, 1)$ — sign mismatch.
The correct choice is $u_2 = (0, -1)$ .

Question 3

A $3 \times 5$ matrix $A$ has singular values $\sigma_1 = 7$ , $\sigma_2 = 2$ , $\sigma_3 = 0$ .

What is the dimension of the null space of $A$ ?

✅ Correct! $\dim(\text{null}(A)) = n - \text{rank}(A) = 5 - 2 = 3$ .

❌ Not quite. Use rank-nullity: $\dim(\text{null}) = n - \text{rank}$ , where $\text{rank}$ = number of nonzero singular values.

Show solution

Solution:

The rank-nullity theorem says $\dim(\text{null}(A)) = n - \text{rank}(A)$ , where $n$ is the number of columns.

Number of columns: $n = 5$ .
$\text{rank}(A) =$ number of nonzero singular values $= 2$ .
$\dim(\text{null}(A)) = 5 - 2 = 3$ .

Don't be fooled by the "3" listed singular values — for a $3 \times 5$ matrix there are $\min(3, 5) = 3$ singular values, but the null space of $A$ lives in $\mathbb{R}^5$ and has dimension $5 - \text{rank} = 3$ . The columns of $V$ corresponding to zero singular values plus the columns beyond the listed ones (here, $v_4$ and $v_5$ , which automatically have $\sigma = 0$ ) all span the null space.

Question 4

For the rank-one matrix $A = \begin{bmatrix} 4 & 3 \\ 8 & 6 \end{bmatrix}$ with $\sigma_1 = \sqrt{125}$ , $\sigma_2 = 0$ , $u_1 = \tfrac{1}{\sqrt 5}(1,2)$ , $v_1 = (0.8, 0.6)$ :

Which expression correctly reconstructs $A$ ?

✅ Correct! A zero singular value kills its rank-one piece — that's how SVD records rank deficiency.

❌ Not quite. Multiply out $\sum_i \sigma_i u_i v_i^\top$ and notice what happens to the term with $\sigma_2 = 0$ .

Show solution

Solution:

The SVD expands $A$ as a sum of rank-one outer products: $A = \sum_{i} \sigma_i\, u_i v_i^\top.$

When $\sigma_2 = 0$ , that term vanishes, so only one rank-one piece remains: $A = \sigma_1\, u_1 v_1^\top = \sqrt{125} \cdot \tfrac{1}{\sqrt 5}\binom{1}{2}\, (0.8,\ 0.6) = 5 \begin{bmatrix} 0.8 & 0.6 \\ 1.6 & 1.2 \end{bmatrix} = \begin{bmatrix} 4 & 3 \\ 8 & 6 \end{bmatrix}.$

Note the order: $u_i v_i^\top$ (column times row) gives a matrix, while $v_i u_i^\top$ has the wrong shape pattern. This "one term suffices" idea is exactly why SVD powers low-rank approximation and PCA.

Solved: 0 / 4