LINEAR-ALGEBRA · Unit 1 · Video 3 · Interactive Practice

Can We Always Solve Ax = b?

IKey Formulas

Formula	Name	When It Applies
$A\mathbf{x} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n$	Column picture of $A\mathbf{x}$	Always — $A\mathbf{x}$ is a linear combination of columns
$\mathbf{x} = A^{-1}\mathbf{b}$	Inverse formula	Only when $A$ is non-singular (invertible)
Independent columns $\Rightarrow$ spans all of $\mathbb{R}^n$	Non-singular	$A\mathbf{x} = \mathbf{b}$ solvable for every $\mathbf{b}$
Dependent columns $\Rightarrow$ spans only a subspace	Singular	$A\mathbf{x} = \mathbf{b}$ solvable for only some $\mathbf{b}$

IIVisualization 1 — Columns Combine to Reach a Target

$A\mathbf{x}$ stacks the scaled columns tip-to-tail; pick weights $x_1,x_2$ to land on $\mathbf{b}=(5,4)$ .

x₁ (weight on a₁) 1 x₂ (weight on a₂) 1

💡 Challenge: land exactly on $\mathbf{b}$ (the weights are $x_1=2,\ x_2=1$ ). Since $\mathbf{a}_1=(2,1)$ and $\mathbf{a}_2=(1,2)$ point in independent directions, some choice of weights reaches every point of $\mathbb{R}^2$ .

IIIVisualization 2 — Determinant and Singularity

The parallelogram on $\mathbf{a}_1,\mathbf{a}_2$ has area $|\det A|$ ; it collapses to zero exactly when the columns become dependent.

a₂ x 1 a₂ y 3

💡 At $\det A = 0$ the reachable set collapses from the whole plane to the single line through $\mathbf{a}_1$ : the matrix is singular.

IVVisualization 3 — Is b in the Column Space?

This singular matrix's column space is the plane $b_3=b_1+b_2$ ; $A\mathbf{x}=\mathbf{b}$ is solvable only when $\mathbf{b}$ lands on the diagonal.

b₁ 1 b₂ 1 b₃ 2

💡 Because $\mathbf{a}_3=\mathbf{a}_1+\mathbf{a}_2$ , all three columns lie in one plane, so every $\mathbf{b}$ off that plane is permanently unreachable.

VQuiz Questions

Question 1

Given $A = \begin{pmatrix} 1 & 3 \\ 2 & 4 \end{pmatrix}$ and $\mathbf{x} = \begin{pmatrix} 2 \\ 1 \end{pmatrix}$ , compute $A\mathbf{x}$ using the column picture.

Hint: $A\mathbf{x} = x_1 \cdot (\text{column 1}) + x_2 \cdot (\text{column 2})$

✅ Correct! Scale column 1 by $x_1 = 2$ and column 2 by $x_2 = 1$ , then add.

❌ Not quite. Remember: $A\mathbf{x} = x_1 \cdot (\text{column 1}) + x_2 \cdot (\text{column 2})$ . Scale each column by its corresponding weight, then add.

Show solution

Solution:

The columns of $A$ are $\mathbf{a}_1 = \begin{pmatrix}1\\2\end{pmatrix}$ and $\mathbf{a}_2 = \begin{pmatrix}3\\4\end{pmatrix}$ .

Using the column picture:

$A\mathbf{x} = x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 = 2\begin{pmatrix}1\\2\end{pmatrix} + 1\begin{pmatrix}3\\4\end{pmatrix} = \begin{pmatrix}2\\4\end{pmatrix} + \begin{pmatrix}3\\4\end{pmatrix} = \begin{pmatrix}5\\8\end{pmatrix}$

The answer is (5, 8).

Common mistakes:

(7, 10) comes from swapping the weights: $1 \cdot \mathbf{a}_1 + 2 \cdot \mathbf{a}_2$
(4, 10) comes from dotting each column with $\mathbf{x}$ instead of scaling columns by weights

Question 2

True or False: If a $3 \times 3$ matrix has $\mathbf{a}_3 = 2\mathbf{a}_1 - \mathbf{a}_2$ , then the matrix is non-singular (invertible).

✅ Correct! When one column can be written as a combination of others, the columns are dependent and the matrix is singular.

❌ Not quite. If $\mathbf{a}_3 = 2\mathbf{a}_1 - \mathbf{a}_2$ , then column 3 adds nothing new — it's a combination of the other columns, making them dependent.

Show solution

Solution:

If $\mathbf{a}_3 = 2\mathbf{a}_1 - \mathbf{a}_2$ , then column 3 is a linear combination of columns 1 and 2.

This means the columns are dependent.

$\text{Dependent columns} \Rightarrow \text{Singular (NOT invertible)} \Rightarrow A^{-1} \text{ does not exist}$

The statement is False.

Question 3

A $4 \times 4$ matrix $A$ is known to be singular. Which statement is true about the equation $A\mathbf{x} = \mathbf{b}$ ?

✅ Correct! Singular means the column space doesn't fill all of R⁴ — some targets are reachable, others are permanently out of reach.

❌ Not quite. Remember from the video: singular does not mean no solutions. It means the columns don't span the full space, so only some right-hand sides are reachable.

Show solution

Solution:

A singular matrix has dependent columns, so:

The column space is a proper subspace of $\mathbb{R}^4$ (not all of it)
When $\mathbf{b}$ is in the column space → $A\mathbf{x} = \mathbf{b}$ has solutions
When $\mathbf{b}$ is outside the column space → $A\mathbf{x} = \mathbf{b}$ has no solution

Key insight from the video: Singular $\neq$ "no solutions ever." Singular means "not ALL $\mathbf{b}$ are reachable." Some targets lie in the column space (solvable), while others lie outside it (unsolvable).

The answer is: Ax = b is solvable for some b but not all b.

Question 4

You generate a $100 \times 100$ matrix by choosing each entry randomly. What is most likely true about this matrix?

✅ Correct! Singular matrices are 'measure zero' — column dependence requires an astronomically unlikely coincidence. A random matrix is almost surely invertible.

❌ Not quite. Think about the dart analogy from the video: for columns to be dependent, a random vector must land exactly on a lower-dimensional subspace, which happens with probability zero.

Show solution

Solution:

For a random matrix to be singular, its columns must satisfy a very precise relationship — one column must lie exactly in the subspace spanned by the others.

Think of the video's analogy: in 100-dimensional space, the "bad" directions for the 100th column form a 99-dimensional hyperplane inside the 100-dimensional space. A randomly chosen vector lands on that hyperplane with probability zero — just like a randomly thrown dart hitting an infinitely thin surface.

A random square matrix is almost certainly non-singular (invertible), regardless of its size.

The answer is: It is almost certainly non-singular.

Solved: 0 / 4