LINEAR-ALGEBRA · Interactive Practice | Unit 28 · Video 1

Two Bases, One Matrix: The Geometric Heart of the SVD

IKey Formulas

Formula	Name	Description
$A = U\Sigma V^T$	Singular Value Decomposition	Works for any $m \times n$ matrix
$A v_i = \sigma_i u_i$	SVD defining relation	$A$ maps orthonormal $v_i$ to orthonormal $u_i$ , scaled by $\sigma_i$
$\sigma_i = \sqrt{\lambda_i(A^T A)}$	Singular value	Square root of an eigenvalue of $A^T A$ ; always $\geq 0$
$A = Q\Lambda Q^T$	Spectral theorem (symmetric PD)	Special case where $U = V = Q$ and $\Sigma = \Lambda$

IIVisualization 1 — Rotate, Stretch, Rotate

$A = U\Sigma V^T$ turns the unit circle into an ellipse in three moves: a rotation, a stretch, another rotation.

σ₁ 2.5 σ₂ 1 θ_V 30° θ_U −45° stage 3.00

💡 Challenge: set $\sigma_2 = 0$ — the ellipse collapses to a segment, so $A$ has rank 1 and $v_2$ lands in the null space.

IIIVisualization 2 — The Basis That Stays Orthogonal

For $A = \bigl(\begin{smallmatrix} 2 & 1 \\ 0.5 & 1.5 \end{smallmatrix}\bigr)$ , only one input basis comes out of $A$ still perpendicular — the singular vectors $v_1, v_2$ .

basis angle φ 20°

💡 Challenge: drag $b_1$ until $Av_1 \perp Av_2$ — that input angle, $\varphi \approx 39.9\degree$ , is exactly the SVD basis.

IVVisualization 3 — Singular Values vs Eigenvalues

Singular values are the image ellipse's semi-axes — always real and $\geq 0$ ; the eigenvalues can be negative or even complex.

a₁₁ 2 a₁₂ 1 a₂₁ −1 a₂₂ 2

💡 Try: a 90° rotation $a_{11}=0,\ a_{12}=-1,\ a_{21}=1,\ a_{22}=0$ — both singular values equal $1$ , yet the eigenvalues are $\pm i$ .

VQuiz Questions

Question 1

Which of the following statements about the SVD $A = U\Sigma V^T$ is always true for a real $m \times n$ matrix $A$ ?

✅ Correct! This is exactly the structure of the SVD, and it holds for every real matrix.

❌ Not quite. The SVD has no restriction on shape or invertibility — that's why it's called the 'final factorization.'

Show solution

Solution:

The SVD works for every real matrix $A$ — square, rectangular, singular, or nonsingular. The factorization $A = U\Sigma V^T$ has:

$U$ is $m \times m$ orthogonal (so $U^T U = I$ )
$V$ is $n \times n$ orthogonal (so $V^T V = I$ )
$\Sigma$ is $m \times n$ diagonal, with entries $\sigma_1 \geq \sigma_2 \geq \cdots \geq 0$

There is no requirement that $A$ be square or invertible. Eigenvalues may not even exist for rectangular $A$ , but singular values always do.

The case $U = V$ only happens for symmetric positive definite matrices.

Question 2

Consider the rotation matrix $A = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ , which rotates vectors by 90°.

What are its singular values?

✅ Correct! A rotation preserves lengths, so both singular values equal 1, even though the eigenvalues are imaginary.

❌ Not quite. Remember: singular values are $\sqrt{\lambda_i(A^T A)}$ , always non-negative real numbers — never complex, never negative.

Show solution

Solution:

Singular values are the square roots of eigenvalues of $A^T A$ : $A^T A = \begin{pmatrix} 0 & 1 \\ -1 & 0 \end{pmatrix}\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} = I.$

The eigenvalues of $I$ are both 1, so $\sigma_1 = \sigma_2 = 1$ . Singular values are always non-negative real numbers, even when $A$ 's eigenvalues are complex ( $\pm i$ in this case).

Geometrically: a rotation maps the unit circle to itself, so the "ellipse" has both semi-axes equal to 1.

Question 3

True or False: For every real matrix $A$ , the right singular vectors equal the left singular vectors (that is, $U = V$ ).

✅ Correct! The SVD uses two distinct orthonormal bases. $U = V$ only in the symmetric positive definite case.

❌ Not quite. Think about a rectangular matrix — $U$ is $m \times m$ and $V$ is $n \times n$ . They can't even be equal in general.

Show solution

Solution:

False. In general, $U \neq V$ .

$V$ 's columns live in the row space of $A$ (the input space, $\mathbb{R}^n$ ).
$U$ 's columns live in the column space of $A$ (the output space, $\mathbb{R}^m$ ).

For a rectangular $m \times n$ matrix, $U$ and $V$ are even different sizes! They can only be equal in the special case where $A$ is symmetric positive definite. In that case, $A = Q\Lambda Q^T$ from the spectral theorem, and we get $U = V = Q$ , $\Sigma = \Lambda$ .

The whole point of the SVD is that we need two orthonormal bases, one for input and one for output — not one basis doing double duty.

Question 4

A matrix $A$ has SVD $A = U\Sigma V^T$ with $\sigma_1 = 4$ , $\sigma_2 = 2$ , and $\sigma_3 = 0$ .

Which statement is correct about the geometry and rank of $A$ ?

✅ Correct! Rank = number of nonzero singular values. The zero singular value collapses one dimension, flattening the ellipsoid into an ellipse.

❌ Not quite. The rank equals the number of nonzero singular values, and any zero singular value means a direction gets crushed to zero (null space).

Show solution

Solution:

The rank of $A$ equals the number of nonzero singular values. Here $\sigma_1 = 4$ and $\sigma_2 = 2$ are nonzero, but $\sigma_3 = 0$ , so $\text{rank}(A) = 2$ .

Geometrically, $A$ takes the unit sphere in 3D to a flat ellipse in the plane spanned by $u_1$ and $u_2$ , with semi-axes of length 4 and 2. The third direction $v_3$ is squashed to zero: $A v_3 = \sigma_3 u_3 = 0.$

So $v_3$ lies in the null space of $A$ .

Note: $v_3$ is not generally an eigenvector of $A$ — it's a right singular vector, and $A v_3 = 0$ doesn't mean $A v_3 = \lambda v_3$ unless $A$ is square. Eigenvectors and singular vectors are different concepts.

Solved: 0 / 4