LINEAR-ALGEBRA ยท Interactive Practice โ€” Unit 26 ยท Video 3

From Bowls to Saddles: How the Hessian Decides Where Functions Bottom Out

IKey Formulas

Concept Formula Meaning
Gradient zero โˆ‡f=0\nabla f = 0 Locates a critical point
Hessian (2D) H=[fxxfxyfxyfyy]H = \begin{bmatrix} f_{xx} & f_{xy} \\ f_{xy} & f_{yy} \end{bmatrix} Matrix of second partials
Minimum test fxx>0f_{xx} > 0 AND fxxfyyโˆ’fxy2>0f_{xx}f_{yy} - f_{xy}^2 > 0 HH is positive definite
Saddle test fxxfyyโˆ’fxy2<0f_{xx}f_{yy} - f_{xy}^2 < 0 HH is indefinite
Eigenvalue test All eigenvalues of H>0H > 0 Equivalent to positive definite

For a quadratic f=ax2+2bโ€‰xy+cy2f = a x^2 + 2b\, xy + c y^2, the Hessian is constant: H=[2a2b2b2c]H = \begin{bmatrix} 2a & 2b \\ 2b & 2c \end{bmatrix}

IIVisualization 1 โ€” Bowl, Saddle, or Peak

The pivot 2a2a and the determinant of the Hessian decide whether the origin is a minimum, a maximum, or a saddle.

๐Ÿ’ก Challenge: hold a=2a = 2 and c=6c = 6 fixed โ€” how large must bb grow before the cross term overpowers the diagonal and the bowl collapses into a saddle?

IIIVisualization 2 โ€” Eigenvectors as Principal Axes

The Hessian's eigenvectors are the principal axes of the level curves; each eigenvalue's sign is the curvature along its axis.

๐Ÿ’ก Every entry of H=[2667]H = \begin{bmatrix} 2 & 6 \\ 6 & 7 \end{bmatrix} is positive, yet one eigenvalue is negative โ€” the determinant, not the sign of the entries, decides definiteness.

IVQuiz Questions

Question 1

Consider f(x,y)=2x2+12xy+20y2f(x, y) = 2x^2 + 12xy + 20y^2. The Hessian at the origin is H=[4121240].H = \begin{bmatrix} 4 & 12 \\ 12 & 40 \end{bmatrix}.

Is the origin a minimum, a maximum, or a saddle?

โœ… Correct! Pivot 4>04 > 0 and detโก=16>0\det = 16 > 0, so HH is positive definite.

โŒ Not quite. Check the pivot fxxf_{xx} and the determinant fxxfyyโˆ’fxy2f_{xx}f_{yy} - f_{xy}^2 โ€” both need to be positive for a minimum.

Show solution

Solution:

Apply both tests:

  • Upper-left pivot: fxx=4>0f_{xx} = 4 > 0.
  • Determinant: detโกH=4โ‹…40โˆ’122=160โˆ’144=16>0\det H = 4 \cdot 40 - 12^2 = 160 - 144 = 16 > 0.

Both conditions are satisfied, so HH is positive definite. The origin is a minimum, and the surface is a bowl.

Question 2

A student claims: "The matrix H=[2667]H = \begin{bmatrix} 2 & 6 \\ 6 & 7 \end{bmatrix} must be positive definite because every entry is positive."

True or False?

โœ… Correct! detโกH=14โˆ’36=โˆ’22<0\det H = 14 - 36 = -22 < 0, so HH is indefinite (a saddle).

โŒ Not quite. Compute detโกH=2(7)โˆ’62\det H = 2(7) - 6^2. Is it really positive? Positive entries do not guarantee positive definiteness.

Show solution

Solution:

Positive entries do not imply positive definite. Compute the determinant: detโกH=2โ‹…7โˆ’62=14โˆ’36=โˆ’22<0.\det H = 2 \cdot 7 - 6^2 = 14 - 36 = -22 < 0.

Since the determinant is negative, HH is indefinite โ€” one eigenvalue is positive, the other negative. The corresponding surface is a saddle, not a bowl.

The student is confusing entries with pivots/eigenvalues. Always check the determinant condition.

Question 3

For the function f(x,y)=2x2+12xy+7y2f(x, y) = 2x^2 + 12xy + 7y^2, compute the Hessian at the origin and classify the critical point.

What is the determinant of HH, and what type of critical point does the origin represent?

โœ… Correct! detโกH=56โˆ’144=โˆ’88<0\det H = 56 - 144 = -88 < 0, so it's a saddle.

โŒ Not quite. Be careful with the sign: detโกH=fxxfyyโˆ’fxy2=4(14)โˆ’122\det H = f_{xx}f_{yy} - f_{xy}^2 = 4(14) - 12^2. Is the cross term big enough to flip the sign?

Show solution

Solution:

Compute the second partials:

  • fxx=4f_{xx} = 4
  • fyy=14f_{yy} = 14
  • fxy=12f_{xy} = 12

So H=[4121214].H = \begin{bmatrix} 4 & 12 \\ 12 & 14 \end{bmatrix}.

Determinant: detโกH=4โ‹…14โˆ’122=56โˆ’144=โˆ’88.\det H = 4 \cdot 14 - 12^2 = 56 - 144 = -88.

Since detโกH<0\det H < 0, HH is indefinite and the origin is a saddle point. The surface curves up along one eigenvector direction and down along another.

Question 4

You slice the graph z=f(x,y)z = f(x, y) of a quadratic with positive definite Hessian by the horizontal plane z=1z = 1. What shape is the resulting curve in the xyxy-plane?

โœ… Correct! Positive definite โ†’ closed, bounded level sets โ†’ ellipses with axes along the eigenvectors of HH.

โŒ Not quite. Positive definite means the surface curves up in every direction. What kind of horizontal cross-section does a bowl have?

Show solution

Solution:

A positive definite Hessian means the surface is a bowl curving upward in every direction. Slicing a bowl horizontally gives a closed curve โ€” specifically, an ellipse.

  • The axes of the ellipse line up with the eigenvectors of HH.
  • The semi-axis lengths are 1/ฮปi1/\sqrt{\lambda_i} where ฮปi\lambda_i are the eigenvalues.

By contrast:

  • An indefinite Hessian (saddle) slices to a hyperbola.
  • A degenerate Hessian can slice to parallel lines or a parabola.

The shape of the level set is the geometric signature of the matrix.

Solved: 0 / 4