Calculus

The Hessian

A surface does not have one second derivative. It bends by a different amount in every direction you could walk, and the four numbers of the Hessian hold all of them at once. Its eigenvectors are the two directions that bend the most and the least, and their signs decide whether you are standing in a bowl, on a dome or on a saddle.

a direction that bends upa direction that bends downthe gradient, which knows nothing about bendingthe quadratic approximation

Drag the white point along the surface, or click the surface to put it there. Drag anywhere else to turn the view, scroll to zoom.

Standing on—

—

Surface

The eigen-directions of the HessianWhat the quadratic approximation saysThe gradient

Values

f_xx—

f_xy—

f_yy—

λ₁, the larger—

λ₂, the smaller—

strongest ÷ weakest bending—

What to observe

At the centre of the Saddle the gradient vanishes, so there is no arrow, but the eigen-directions still disagree: the teal curve bends up and the red one bends down.
On the Bowl both curves bend up, so both eigenvalues are positive. The violet patch lies on the surface everywhere, because a quadratic surface is its own quadratic approximation.
On Ripples the patch hugs the surface only near the point and peels away further out. The Hessian is a new matrix at every point: drag the point and the two curves turn with it.
In the Narrow valley the eigenvalues differ thirtyfold: one curve runs almost flat along the floor while the other climbs the walls, which is what makes gradient descent zigzag.

Shortcuts:f fullscreen