LINEAR-ALGEBRA · Interactive Practice

The Inverse That Always Exists: The Pseudo-Inverse

IKey Formulas

Formula	Name	Description
$\dim(\text{Row}) = \dim(\text{Col}) = r$	Matching dimensions	Row space and column space share the rank $r$
$\dim(\text{Null}) = n - r,\quad \dim(\text{Left Null}) = m - r$	Subspace dimensions	What the null spaces "lose"
$A^{+}A = P_{\text{Row}}$	Projection identity	$A^{+}A$ projects onto the row space (not $I$ !)
$AA^{+} = P_{\text{Col}}$	Projection identity	$AA^{+}$ projects onto the column space
$\hat{x} = A^{+}b$	Pseudo-inverse solution	Minimum-norm least-squares solution

Recall: Row space $\perp$ Null space, and Column space $\perp$ Left null space. The pseudo-inverse $A^{+}$ inverts the bijection Row $\to$ Column, and sends the left null space to zero.

IIVisualization 1 — The Four Fundamental Subspaces

For a matrix of rank $r$ , how large is each of the four fundamental subspaces?

m (rows) 3 n (columns) 2 rank r 2

💡 $A^{+}$ exists because that shared dimension $r$ lets $A$ act as a bijection from the row space onto the column space.

IIIVisualization 2 — Row Part, Null Part, and the Mapping

Every input $v$ splits into a row-space part and a null-space part — which one survives $Av$ ?

💡 Because the null part is destroyed, $A$ has no true inverse; $A^{+}$ can only ever recover the row-space part of a solution.

IVVisualization 3 — Least Squares and the Pseudo-Inverse

The pseudo-inverse fits the best line through the data and never breaks — even when $A^{T}A$ is singular.

💡 Challenge: drag all three points to the same $x$ : the columns of $A$ become dependent and $A^{T}A$ turns singular, yet $A^{+}b$ still returns the minimum-norm fit.

VQuiz Questions

Question 1

A matrix $A$ has $m = 4$ rows, $n = 3$ columns, and rank $r = 2$ .

What is the dimension of the null space of $A$ ?

✅ Correct! The null space has dimension $n - r = 3 - 2 = 1$ .

❌ Not quite. The null space dimension is $n - r$ (use columns, not rows).

Show solution

Solution:

The null space lives in the input space $\mathbb{R}^n$ and has dimension $n - r$ : $\dim(\text{Null}) = n - r = 3 - 2 = 1.$

(For reference: the column space has dimension $r = 2$ , and the left null space has dimension $m - r = 4 - 2 = 2$ .)

Question 2

True or False: For every matrix $A$ , the product $A^{+}A$ equals the identity matrix.

✅ Correct! $A^{+}A$ projects onto the row space; it equals $I$ only with full column rank.

❌ Not quite. $A^{+}A$ is a projection onto the row space, not the identity in general.

Show solution

Solution:

$A^{+}A$ is the projection onto the row space, not the identity. Split any $x \in \mathbb{R}^n$ into a row-space part plus a null-space part; $A^{+}A$ keeps only the row-space part and discards the null-space part: $A^{+}A = P_{\text{Row}}.$

You only get $A^{+}A = I$ when the null space is trivial — i.e. $A$ has full column rank. So the statement is False in general.

Question 3

For $A = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}$ , the row space is spanned by $(1, 1)$ and the null space is spanned by $(1, -1)$ .

What is the dot product $(1, 1) \cdot (1, -1)$ , and what does it confirm?

✅ Correct! The dot product is $0$ , so the row space and null space are orthogonal.

❌ Not quite. Compute $(1)(1)+(1)(-1)$ — the result is $0$ , confirming orthogonality.

Show solution

Solution:

$(1,1)\cdot(1,-1) = (1)(1) + (1)(-1) = 1 - 1 = 0.$

A dot product of $0$ means the two directions are perpendicular. This confirms that the row space and the null space are orthogonal complements — the only vector lying in both is the zero vector. That orthogonality is exactly what makes the restriction of $A$ to the row space one-to-one.

Question 4

In a regression you accidentally include the same predictor twice, giving two identical columns $A = \begin{bmatrix} 1 & 1 \\ 2 & 2 \\ 3 & 3 \end{bmatrix}$ . The ordinary least-squares formula $\hat{x} = (A^{T}A)^{-1}A^{T}b$ now fails.

Which statement best explains why — and how the pseudo-inverse helps?

✅ Correct! $A^{T}A$ is singular, so $(A^{T}A)^{-1}$ fails; $\hat{x} = A^{+}b$ rescues it.

❌ Not quite. The dependent columns make $A^{T}A$ singular ( $\det = 0$ ); the pseudo-inverse still works.

Show solution

Solution:

The duplicated predictor makes the two columns identical (perfect collinearity), so $A$ is rank-deficient and $A^{T}A$ is singular: $\det(A^{T}A) = 0$ , which means $(A^{T}A)^{-1}$ is undefined and the OLS formula collapses.

The pseudo-inverse always exists, so we replace the broken formula with $\hat{x} = A^{+}b,$ which gracefully delivers the minimum-norm least-squares solution.

Solved: 0 / 4