LINEAR-ALGEBRA · Unit 15 · Video 4 · Interactive Practice

Why $A^TA$ Is Invertible — and When It Equals $I$

IKey Formulas

Formula	Name	Key Idea
$A^TAx = 0 \;\Rightarrow\; \\|Ax\\|^2 = 0 \;\Rightarrow\; x = 0$	Null-space trick	Proves $A^TA$ invertible when $A$ has independent columns
$q_i \cdot q_j = 0$ for $i \neq j$ , and $\\|q_i\\| = 1$	Orthonormal columns	Perpendicular and unit length
Orthonormal columns $\Rightarrow A^TA = I$	Orthonormal identity	Entry $(i,j)$ of $A^TA$ is $q_i \cdot q_j = \delta_{ij}$
$P = A(A^TA)^{-1}A^T \;\longrightarrow\; P = AA^T$	Projection simplification	Inverse disappears for orthonormal columns

IIVisualization 1 — A Rotation Frame Keeps $A^TA = I$

Rotate the frame to any angle $\theta$ : its columns stay perpendicular unit vectors, so $A^TA$ never leaves the identity.

Angle θ 43°

IIIVisualization 2 — Orthogonal Isn't Orthonormal

Perpendicular columns make $A^TA$ diagonal, but only unit lengths turn those diagonal entries into $1$ 's.

‖v₁‖ 1.50 ‖v₂‖ 2.00

💡 Challenge: set the lengths to the video's $(3, 5)$ — $A^TA$ becomes $\mathrm{diag}(9, 25)$ : invertible, but nowhere near $I$ .

IVVisualization 3 — Independent Columns Make $A^TA$ Invertible

Drag the two columns of $A$ : $A^TA$ stays invertible right up until the columns line up.

💡 When the columns collapse onto one line, some nonzero $x$ satisfies $Ax = 0$ — the null-space trick breaks and $A^TA$ turns singular.

VQuiz Questions

Question 1

Let $A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$ . What is $A^TA$ ?

(Hint: Check the columns — are they orthonormal?)

✅ Correct! The columns $(0,1)$ and $(-1,0)$ are orthonormal, so $A^TA = I$ .

❌ Not quite. Try checking whether the columns have unit length and are perpendicular.

Show solution

Solution:

The columns of $A$ are $q_1 = (0, 1)$ and $q_2 = (-1, 0)$ .

Check unit length:

$\|q_1\| = \sqrt{0^2 + 1^2} = 1$ ✓
$\|q_2\| = \sqrt{(-1)^2 + 0^2} = 1$ ✓

Check perpendicularity:

$q_1 \cdot q_2 = (0)(-1) + (1)(0) = 0$ ✓

The columns are orthonormal, so $A^TA = I$ .

Verification by direct computation:

$A^TA = \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} = \begin{bmatrix} 0\cdot0+1\cdot1 & 0\cdot(-1)+1\cdot0 \\ (-1)\cdot0+0\cdot1 & (-1)(-1)+0\cdot0 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}$

Question 2

True or False: The vectors $(3, 0)$ and $(0, 5)$ form an orthonormal set.

✅ Correct! They are perpendicular (orthogonal) but their lengths are 3 and 5, not 1. Orthonormal requires both conditions.

❌ Not quite. Orthonormal means orthogonal AND unit-length. Check the norms.

Show solution

Solution: False

Orthonormal requires both conditions:

Orthogonal (perpendicular)? $(3, 0) \cdot (0, 5) = 3 \cdot 0 + 0 \cdot 5 = 0$ ✓
Normal (unit length)?
- $\|(3, 0)\| = 3 \neq 1$ ✗
- $\|(0, 5)\| = 5 \neq 1$ ✗

The vectors are orthogonal but not orthonormal. To make them orthonormal, normalize: $(1, 0)$ and $(0, 1)$ .

With the original vectors, $A^TA = \text{diag}(9, 25)$ — invertible but not the identity.

Question 3

In the null-space trick, we start with $A^TAx = 0$ and need to prove $x = 0$ . What is the key step that converts the matrix equation into a statement about length?

✅ Correct! Multiplying by $x^T$ manufactures the dot product $\|Ax\|^2$ , converting the matrix equation into a geometric statement about length.

❌ Not quite. The trick creates a squared-length expression. Which operation turns $x^TA^TAx$ into $(Ax)^T(Ax) = \|Ax\|^2$ ?

Show solution

Solution:

The key step is multiplying both sides by $x^T$ on the left, which manufactures a dot product:

$A^TAx = 0$ $x^T(A^TAx) = x^T \cdot 0 = 0$

Regroup using $(x^TA^T) = (Ax)^T$ :

$(Ax)^T(Ax) = 0$ $\|Ax\|^2 = 0$

Since squared length $\geq 0$ and equals 0 only for the zero vector: $Ax = 0$

Column independence then gives $x = 0$ . ∎

The insight: multiplying by $x^T$ converts algebra into geometry — a matrix equation becomes a statement that a squared length is zero.

Question 4

The general projection formula is $P = A(A^TA)^{-1}A^T$ . When $A$ has orthonormal columns, what does this simplify to?

✅ Correct! Since $A^TA = I$ , the inverse vanishes and $P = AA^T$ . No matrix inversion needed!

❌ Not quite. Substitute $A^TA = I$ into $P = A(A^TA)^{-1}A^T$ and simplify step by step.

Show solution

Solution:

When $A$ has orthonormal columns, $A^TA = I$ , so $(A^TA)^{-1} = I^{-1} = I$ .

Substituting into the projection formula:

$P = A\underbrace{(A^TA)^{-1}}_{= I}A^T = A \cdot I \cdot A^T = AA^T$

The inverse disappears entirely — a massive computational simplification.

Important note: $P = AA^T$ is generally not the identity matrix (unless $A$ is square and orthogonal). It is the projection matrix onto the column space of $A$ .

This simplification underpins QR factorization, Gram-Schmidt, and spectral decomposition.

Solved: 0 / 4

Why ATAA^TAATA Is Invertible — and When It Equals III

IKey Formulas

IIVisualization 1 — A Rotation Frame Keeps ATA=IA^TA = IATA=I

IIIVisualization 2 — Orthogonal Isn't Orthonormal

IVVisualization 3 — Independent Columns Make ATAA^TAATA Invertible

VQuiz Questions

Why $A^TA$ Is Invertible — and When It Equals $I$

IIVisualization 1 — A Rotation Frame Keeps $A^TA = I$

IVVisualization 3 — Independent Columns Make $A^TA$ Invertible