LINEAR-ALGEBRA · Interactive Practice | Unit 31 · Video 4

The Inverse That Always Exists: Building the Pseudo-Inverse from the SVD

IKey Formulas

Formula Name Description
A=UΣVTA = U \Sigma V^T Singular Value Decomposition U,VU, V orthogonal; Σ\Sigma diagonal (m×nm\times n)
U−1=UT,V−1=VTU^{-1} = U^T,\quad V^{-1} = V^T Orthogonal inverse Inverse equals transpose — free!
Σ+: σi→1σi\Sigma^+ : \ \sigma_i \to \tfrac{1}{\sigma_i} Pseudo-inverse of Σ\Sigma Invert nonzero entries, transpose the shape
A+=VΣ+UTA^+ = V \Sigma^+ U^T Moore–Penrose pseudo-inverse The "inverse that always exists"

IIVisualization 1 — Building Σ+\Sigma^+ from Σ\Sigma

Reciprocate each nonzero singular value, leave the zeros untouched, and transpose the shape.

💡 As a singular value shrinks toward 00, its reciprocal 1/σ1/\sigma explodes — which is why real algorithms threshold tiny singular values to 00 rather than inverting them.

IIIVisualization 2 — Projections, Not the Identity

The products ΣΣ+\Sigma\Sigma^+ and Σ+Σ\Sigma^+\Sigma carry exactly rr ones on the diagonal — projections, never the full identity.

💡 A matrix with a null space cannot have a true two-sided inverse; these projections are the closest to the identity the null spaces allow.

IVVisualization 3 — The Pseudo-Inverse Solves Least Squares

With more equations than unknowns, x=A+bx = A^+ b returns the line that minimizes the squared residuals.

VQuiz Questions

Question 1

A diagonal matrix Σ\Sigma has a nonzero singular value σ1=4\sigma_1 = 4 on its diagonal. In the pseudo-inverse Σ+\Sigma^+, what value sits in the corresponding diagonal position?

✅ Correct! Each nonzero singular value is inverted: 4 → 1/4.

❌ Not quite. The pseudo-inverse takes the reciprocal of each nonzero singular value.

Show solution

Solution:

To build Σ+\Sigma^+, each nonzero singular value is replaced by its reciprocal: σi  ⟶  1σi\sigma_i \;\longrightarrow\; \frac{1}{\sigma_i}

So σ1=4\sigma_1 = 4 becomes 14\tfrac{1}{4}. (Zeros stay zero — we never divide by zero.)

Question 2

Suppose Σ\Sigma is a 3×43 \times 4 matrix. What are the dimensions of its pseudo-inverse Σ+\Sigma^+?

✅ Correct! The shape transposes: a 3×4 matrix gives a 4×3 pseudo-inverse.

❌ Not quite. Remember that building Σ⁺ also transposes the shape from m×n to n×m.

Show solution

Solution:

Building Σ+\Sigma^+ has two steps: invert the nonzero entries, and transpose the shape.

If Σ\Sigma is m×nm \times n, then Σ+\Sigma^+ is n×mn \times m.

For Σ\Sigma that is 3×43 \times 4, the pseudo-inverse Σ+\Sigma^+ is 4×34 \times 3.

Question 3

True or False: For a rank-deficient matrix, the product ΣΣ+\Sigma \Sigma^+ always equals the identity matrix.

✅ Correct! It is a projection with only r ones on the diagonal — not the identity.

❌ Not quite. ΣΣ⁺ is a projection onto the column space; the null space directions become zeros.

Show solution

Solution:

The statement is False. The product ΣΣ+\Sigma \Sigma^+ is a projection, not the identity.

It is an m×mm \times m matrix with exactly rr ones on the diagonal (where rr is the rank) and zeros for the remaining null-space directions. For a rank-deficient matrix, rr is smaller than the matrix size, so there are trailing zeros and the result cannot be the identity.

A matrix with a null space cannot have a true two-sided inverse — the projections ΣΣ+\Sigma\Sigma^+ (onto the column space) and Σ+Σ\Sigma^+\Sigma (onto the row space) are the closest thing to identity the null spaces allow.

Question 4

Starting from A=UΣVTA = U \Sigma V^T and using the fact that UU and VV are orthogonal, which expression gives the pseudo-inverse A+A^+?

✅ Correct! Reversing the order and using the orthogonal inverses gives A⁺ = V Σ⁺ Uᵀ.

❌ Not quite. Reverse the factor order and use V⁻¹=Vᵀ (so (Vᵀ)⁻¹=V) and U⁻¹=Uᵀ.

Show solution

Solution:

To invert a product, reverse the order and invert each factor: A+=(UΣVT)+=(VT)−1 Σ+ U−1.A^+ = (U \Sigma V^T)^+ = (V^T)^{-1}\, \Sigma^+\, U^{-1}.

Now use orthogonality:

  • VV is orthogonal, so (VT)−1=V(V^T)^{-1} = V.
  • UU is orthogonal, so U−1=UTU^{-1} = U^T.

Everything collapses cleanly to: A+=V Σ+ UT\boxed{A^+ = V\, \Sigma^+\, U^T}

The transposes fall out for free because UU and VV are orthogonal — all the difficulty stayed quarantined inside Σ+\Sigma^+.

Solved: 0 / 4