LINEAR-ALGEBRA · Unit 16 · Video 2 · Interactive Practice

No Inverses Needed: How Orthonormal Bases Turn Projections into Dot Products

IKey Formulas

Formula	Name	When to Use
$P = A(A^TA)^{-1}A^T$	General projection matrix	Any basis — requires matrix inversion
$P = QQ^T$	Orthonormal projection matrix	When columns of $Q$ are orthonormal
$\hat{x}_i = q_i^T b$	Coefficient as dot product	Finding each least-squares coefficient
$Q^TQ = I$	Orthonormality condition	Columns are unit-length and mutually perpendicular

IIVisualization 1 — When $Q^TQ = I$

$Q$ has orthonormal columns exactly when $Q^TQ = I$ : unit-length columns on the diagonal, perpendicular columns off it.

💡 The diagonal entries of $Q^TQ$ are the squared lengths $\|q_i\|^2$ ; the off-diagonal entry is the dot product $q_1 \cdot q_2$ . Identity demands unit length and perpendicularity together.

IIIVisualization 2 — Projecting $b$ onto Col $(Q)$

With $q_1 = \tfrac{1}{\sqrt{2}}(1,1,0)^T$ , $q_2 = (0,0,1)^T$ , the map $p = QQ^Tb$ splits $b$ into an in-plane part $p$ and a perpendicular error $e = b - p$ .

b_1

b_2

b_3

💡 $p$ is the closest point of the plane to $b$ — which is exactly why $e$ leaves perpendicular. The coefficients $\hat{x}_1 = q_1^Tb$ and $\hat{x}_2 = q_2^Tb$ are plain dot products, no inverse needed.

IVVisualization 3 — Every Coefficient Is a Dot Product

In the orthonormal basis $q_1 = \tfrac{1}{\sqrt{2}}(1,1)^T$ , $q_2 = \tfrac{1}{\sqrt{2}}(-1,1)^T$ , each coordinate of $b$ is just a dot product: $b = (q_1^Tb)\,q_1 + (q_2^Tb)\,q_2$ .

💡 A general basis would force you to solve $A^TA\hat{x} = A^Tb$ ; orthonormality collapses that linear solve into two dot products — the same idea that defines Fourier coefficients.

VQuiz Questions

Question 1

Given orthonormal column $q_1 = \left(\frac{1}{\sqrt{2}},\; \frac{1}{\sqrt{2}},\; 0\right)^T$ and vector $b = (1, 2, 3)^T$ , compute the coefficient $\hat{x}_1 = q_1^T b$ .

✅ Correct! $\hat{x}_1 = 3/\sqrt{2} \approx 2.121$ . Just one dot product!

❌ Not quite. Compute $q_1^T b$ by multiplying corresponding entries and summing: $\frac{1}{\sqrt{2}} \cdot 1 + \frac{1}{\sqrt{2}} \cdot 2 + 0 \cdot 3$ .

Show solution

Solution:

$\hat{x}_1 = q_1^T b = \frac{1}{\sqrt{2}}(1) + \frac{1}{\sqrt{2}}(2) + 0 \cdot (3) = \frac{1}{\sqrt{2}} + \frac{2}{\sqrt{2}} = \frac{3}{\sqrt{2}} \approx 2.121$

The third component of $q_1$ is 0, so the third component of $b$ doesn't contribute. Each coefficient is just a dot product — no system of equations to solve!

Question 2

True or False: If $Q$ is a $5 \times 3$ matrix with orthonormal columns, then $QQ^T = I_5$ (the $5 \times 5$ identity matrix).

✅ Correct! $Q^TQ = I_3$ always holds for orthonormal columns, but $QQ^T$ is only a projection matrix onto Col $(Q)$ — it equals $I$ only when $Q$ is square.

❌ Not quite. Think carefully: $Q^TQ = I$ (small identity) always holds, but does $QQ^T = I$ (big identity) follow when $Q$ has fewer columns than rows?

Show solution

Solution: False.

When $Q$ is $5 \times 3$ with orthonormal columns:

$Q^TQ = I_3$ ✅ (the $3 \times 3$ identity — this always holds for orthonormal columns)
$QQ^T \neq I_5$ ❌ (this is a $5 \times 5$ projection matrix onto the 3D column space, not the identity)

$QQ^T = I$ only when $Q$ is square ( $n \times n$ ), meaning the columns span the entire space $\mathbb{R}^n$ . In that case $Q$ is called an orthogonal matrix and $Q^T = Q^{-1}$ .

Key distinction to remember:

$Q^TQ = I$ → always true for orthonormal columns
$QQ^T = I$ → only when $Q$ is square

Question 3

Let $Q$ be the $3 \times 2$ matrix with orthonormal columns $q_1 = \frac{1}{\sqrt{2}}(1,\; 1,\; 0)^T$ and $q_2 = (0,\; 0,\; 1)^T$ .

Compute the projection $p = QQ^T b$ for $b = (4, 0, 6)^T$ .

✅ Correct! The projection keeps the $q_2$ component (z = 6) and averages the first two entries via $q_1$ : both become 2.

❌ Not quite.

Show solution

Solution:

Step 1 — Coefficients as dot products:

$\hat{x}_1 = q_1^T b = \frac{1}{\sqrt{2}}(4) + \frac{1}{\sqrt{2}}(0) + 0(6) = \frac{4}{\sqrt{2}} = 2\sqrt{2}$

$\hat{x}_2 = q_2^T b = 0(4) + 0(0) + 1(6) = 6$

Step 2 — Reconstruct the projection:

$p = \hat{x}_1 q_1 + \hat{x}_2 q_2 = 2\sqrt{2} \cdot \frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\\0\end{pmatrix} + 6\begin{pmatrix}0\\0\\1\end{pmatrix} = \begin{pmatrix}2\\2\\0\end{pmatrix} + \begin{pmatrix}0\\0\\6\end{pmatrix} = \begin{pmatrix}2\\2\\6\end{pmatrix}$

Interpretation: The projection averages the first two components of $b$ (since $q_1$ equally weights $x$ and $y$ ) and keeps the $z$ -component unchanged (since $q_2$ is the $z$ -axis).

Question 4

The projection formula simplifies from $P = A(A^TA)^{-1}A^T$ to $P = QQ^T$ when $A$ is replaced by an orthonormal matrix $Q$ . What is the key algebraic fact that makes this simplification — and also makes $P$ idempotent ( $P^2 = P$ )?

✅ Correct! The single fact $Q^TQ = I$ drives every simplification: the inverse disappears, and the middle factors collapse in the idempotence proof.

❌ Not quite. Think about what happens when you substitute $Q^TQ = I$ into the formula $P = Q(Q^TQ)^{-1}Q^T$ . What also happens when you compute $P^2 = (QQ^T)(QQ^T)$ ?

Show solution

Solution: The key fact is $Q^TQ = I$ .

For the formula simplification:

$P = Q\underbrace{(Q^TQ)^{-1}}_{= I^{-1} = I}Q^T = Q \cdot I \cdot Q^T = QQ^T$

For idempotence ( $P^2 = P$ ):

$P^2 = (QQ^T)(QQ^T) = Q\underbrace{(Q^TQ)}_{= I}Q^T = Q \cdot I \cdot Q^T = QQ^T = P$

The same fact — $Q^TQ$ collapsing to $I$ — is the engine behind both simplifications. In the idempotence proof, $Q^TQ$ appears sandwiched in the middle and collapses, which is the "algebraic magic moment."

Why the other options are wrong:

$Q$ doesn't need to be square (the $3 \times 2$ example works perfectly).
$Q^T = Q^{-1}$ only holds when $Q$ is square (orthogonal matrix), not for rectangular $Q$ .
General projection matrices do require inverses; it's specifically orthonormality that eliminates the inverse.

Solved: 0 / 4

No Inverses Needed: How Orthonormal Bases Turn Projections into Dot Products

IKey Formulas

IIVisualization 1 — When QTQ=IQ^TQ = IQTQ=I

IIIVisualization 2 — Projecting bbb onto Col(Q)(Q)(Q)

IVVisualization 3 — Every Coefficient Is a Dot Product

VQuiz Questions

IIVisualization 1 — When $Q^TQ = I$

IIIVisualization 2 — Projecting $b$ onto Col $(Q)$