Linear-Algebra-Done-Right · Unit 21 · Video 3 · Interactive Practice

Diagonal Matrices, Adjoints and Pseudoinverses via SVD

IKey Formulas

Formula	Name	What it gives
$Te_k = s_k f_k \ (k \le m), \quad Te_k = 0 \ (k > m)$	SVD on extended orthonormal bases	$\mathcal{M}(T)$ is $p \times n$ and diagonal, with $s_1, \dots, s_m$ from the top left
$T^*w = \sum_{k=1}^{m} s_k \langle w, f_k\rangle e_k$	Adjoint	Swap the roles of the $e$ 's and $f$ 's
$T^\dagger w = \sum_{k=1}^{m} \dfrac{\langle w, f_k\rangle}{s_k}\, e_k$	Pseudoinverse	Swap $e \leftrightarrow f$ and replace $s_k$ by $1/s_k$

Key Insight: $T(x, y) = (3y, 0)$ has a diagonal matrix with respect to no single basis, yet every linear map has one with respect to two suitable orthonormal bases — the $e$ 's for the inputs and the $f$ 's for the outputs.

IIVisualization 1 — Building the Diagonal Matrix

Column $k$ of $\mathcal{M}(T)$ holds the $f$ -coordinates of $Te_k$ , which is $s_k f_k$ or $0$ .

Map Columns filled 0

IIIVisualization 2 — One Basis or Two

No single basis makes the matrix of $T(x, y) = (3y, 0)$ diagonal; two orthonormal bases can.

Bases e₁ angle θ 30° f₁ angle φ 60°

IVVisualization 3 — Adjoint and Pseudoinverse

$T^\dagger w$ is the vector in $(\operatorname{null} T)^\perp$ that $T$ sends to $P_{\operatorname{range} T}\, w$ .

Map

💡 Challenge: find a nonzero $w$ with $T^\dagger w = 0$ , and name the subspace of $W$ it must lie in.

VQuiz Questions

Problem 1 · Shape of the Diagonal Matrix

Given: $T \in \mathcal{L}(\mathbf{F}^5, \mathbf{F}^3)$ has positive singular values $6$ and $2$ . Extend the orthonormal lists of its singular value decomposition to orthonormal bases $e_1, \dots, e_5$ of $\mathbf{F}^5$ and $f_1, f_2, f_3$ of $\mathbf{F}^3$ . Which is $\mathcal{M}\big(T, (e_1, \dots, e_5), (f_1, f_2, f_3)\big)$ ?

✅ Correct! One column per

e_k

(five), one row per

f_j

(three), with

Te_1 = 6f_1

Te_2 = 2f_2

and

Te_3 = Te_4 = Te_5 = 0

❌ Rows and columns are swapped. Column

k

holds the coordinates of

Te_k

in the basis of

W

, so there are

\dim W = 3

rows and

\dim V = 5

columns.

❌ Not quite. The matrix needs a column for every

e_k

, including the added

e_3, e_4, e_5

that

T

sends to

0

— five columns, not three.

❌ Not quite. The reciprocals

1/s_k

belong to the pseudoinverse;

Te_k = s_k f_k

puts

s_k

itself on the diagonal.

❌ Not quite. Column

k

lists the coordinates of

Te_k

f_1, f_2, f_3

Show solution

The singular value decomposition gives $Tv = 6\langle v, e_1\rangle f_1 + 2\langle v, e_2\rangle f_2$ , so $Te_1 = 6f_1$ and $Te_2 = 2f_2$ . The added vectors $e_3, e_4, e_5$ are orthogonal to $e_1, e_2$ , so every inner product in the formula vanishes and $Te_3 = Te_4 = Te_5 = 0$ .

Column $k$ of the matrix records the coordinates of $Te_k$ with respect to $f_1, f_2, f_3$ :

\mathcal{M}(T) = \begin{pmatrix} 6 & 0 & 0 & 0 & 0 \\ 0 & 2 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix},

a $3 \times 5$ diagonal matrix in the extended sense: every entry is $0$ except possibly those in positions $(k, k)$ for $k \le \min\{3, 5\}$ . The third row is zero because $f_3 \notin \operatorname{range} T$ .

Problem 2 · Adjoint or Pseudoinverse?

Given: $Tv = 4\langle v, e_1\rangle f_1 + 2\langle v, e_2\rangle f_2$ , where $e_1, e_2$ is orthonormal in $V$ and $f_1, f_2$ is orthonormal in $W$ . Which formula gives $T^\dagger w$ for $w \in W$ ?

✅ Correct! The

e

's and

f

's trade places and each

s_k

becomes

1/s_k

❌ That is $T^*w$ . Swapping

e \leftrightarrow f

alone gives the adjoint; the pseudoinverse also replaces each

s_k

1/s_k

❌ Not quite.

w

lies in

W

, where the orthonormal list is

f_1, f_2

, and

T^\dagger w

must land in

V

, spanned here by

e_1, e_2

— so the inner products are with the

f

's and the output uses the

e

's.

❌ Not quite. Each

s_k

is replaced by

1/s_k

, not

1/s_k^2

: applying

T

must cancel the

1/s_k

exactly, leaving

\langle w, f_k\rangle f_k

❌ Not quite. Swap the

e

's and

f

's and replace each

s_k

1/s_k

Show solution

Here $m = 2$ with $s_1 = 4$ , $s_2 = 2$ . The pseudoinverse formula $T^\dagger w = \sum_k \frac{\langle w, f_k\rangle}{s_k} e_k$ gives

T^\dagger w = \frac{\langle w, f_1\rangle}{4}\, e_1 + \frac{\langle w, f_2\rangle}{2}\, e_2.

Check: $T$ sends $e_k$ to $s_k f_k$ , so $T(T^\dagger w) = \langle w, f_1\rangle f_1 + \langle w, f_2\rangle f_2 = P_{\operatorname{range} T}\, w$ , and $T^\dagger w$ is a combination of $e_1, e_2$ , which are orthogonal to $\operatorname{null} T$ .

Option (a) is the adjoint $T^*w$ : it swaps the $e$ 's and $f$ 's but keeps $s_k$ .

Problem 3 · From the Map to the Pseudoinverse

Given: $T \in \mathcal{L}(\mathbf{R}^3, \mathbf{R}^2)$ , $T(x, y, z) = (3z, -2x)$ . Find a singular value decomposition of $T$ and then $T^\dagger(6, 4)$ .

Which is a singular value decomposition of T?

What is T†(6, 4)?

✅ Correct!

T^\dagger(w_1, w_2) = \frac{w_1}{3}(0, 0, 1) + \frac{-w_2}{2}(1, 0, 0) = \big(-\tfrac{w_2}{2}, 0, \tfrac{w_1}{3}\big)

, so

T^\dagger(6, 4) = (-2, 0, 2)

❌ Close, but… this formula does equal

Tv

, yet

-2

cannot be a singular value: singular values are positive. Move the sign into the output vector,

f_2 = (0, -1)

❌ Check it against $T$ . Expand the formula at

v = (x, y, z)

; it must give exactly

(3z, -2x)

, with positive coefficients.

❌ Check the sign.

\langle (6, 4), f_2\rangle = \langle (6, 4), (0, -1)\rangle = -4

, not

4

❌ That is $T^*(6, 4)$ . The pseudoinverse divides by each

s_k

instead of multiplying.

❌ Not quite. Each

\langle w, f_k\rangle

is divided by its own

s_k

\langle w, f_1\rangle = 6

goes with

s_1 = 3

, and

\langle w, f_2\rangle = -4

with

s_2 = 2

❌ Not quite. Use

T^\dagger w = \frac{\langle w, f_1\rangle}{s_1} e_1 + \frac{\langle w, f_2\rangle}{s_2} e_2

Show solution

Step 1: SVD. Write $T(x, y, z) = 3z\,(1, 0) + 2x\,(0, -1)$ . With $e_1 = (0, 0, 1)$ , $f_1 = (1, 0)$ , $e_2 = (1, 0, 0)$ , $f_2 = (0, -1)$ — orthonormal lists in $\mathbf{R}^3$ and $\mathbf{R}^2$ — this is

Tv = 3\langle v, e_1\rangle f_1 + 2\langle v, e_2\rangle f_2,

with positive coefficients $s_1 = 3$ , $s_2 = 2$ . (Check: $T^*T$ has eigenvalues $9, 4, 0$ , whose square roots are $3, 2, 0$ .) Option (a) writes the same map with coefficient $-2$ , which is not a singular value decomposition.

Step 2: pseudoinverse. With $w = (6, 4)$ : $\langle w, f_1\rangle = 6$ and $\langle w, f_2\rangle = -4$ , so

T^\dagger(6, 4) = \frac{6}{3}(0, 0, 1) + \frac{-4}{2}(1, 0, 0) = (-2,\ 0,\ 2).

Check: $T(-2, 0, 2) = (6, 4)$ . Here $\operatorname{range} T = \mathbf{R}^2$ , so $P_{\operatorname{range} T}\, w = w$ , and $(-2, 0, 2)$ is orthogonal to $\operatorname{null} T = \operatorname{span}\big((0, 1, 0)\big)$ ✓.

Problem 4 · Equal Singular Values

Given: every positive singular value of $T \in \mathcal{L}(V, W)$ equals the same number $s$ . Which identity holds?

✅ Correct!

T^\dagger w = \sum_k \frac{\langle w, f_k\rangle}{s} e_k = \frac{1}{s^2}\sum_k s\langle w, f_k\rangle e_k = \frac{1}{s^2} T^*w

❌ Not quite. Going from

T^*

T^\dagger

trades a factor

s

for

1/s

, which divides by

s^2

; it does not multiply.

❌ Close, but…

T^*

carries a factor

s

and

T^\dagger

carries

1/s

, so the ratio is

1/s^2

, not

1/s

❌ Not quite. That holds only when

s = 1

. For

T(x, y) = (3y, 0)

T^*(w_1, w_2) = (0, 3w_1)

but

T^\dagger(w_1, w_2) = (0, w_1/3)

❌ Not quite. Compare the two formulas term by term.

Show solution

With $s_1 = \cdots = s_m = s$ the two formulas are

T^*w = \sum_{k=1}^{m} s\langle w, f_k\rangle e_k, \qquad T^\dagger w = \sum_{k=1}^{m} \frac{\langle w, f_k\rangle}{s}\, e_k.

Each term of $T^\dagger w$ is the matching term of $T^*w$ divided by $s^2$ , so $T^\dagger = \frac{1}{s^2} T^*$ .

Check with the video's operator ( $s = 3$ ): $\frac{1}{9}T^*(w_1, w_2) = \frac{1}{9}(0, 3w_1) = (0, w_1/3) = T^\dagger(w_1, w_2)$ ✓. In particular, if $s = 1$ — for instance when $T$ is an isometry — then $T^\dagger = T^*$ .

Solved: 0 / 4