Linear Algebra

Projections & Gram–Schmidt

The closest point of a line or a plane to a vector is where the error meets it at a right angle. The same picture fits a line to data, and done one direction at a time it straightens any basis into an orthonormal one.

ba₁, q₁a₂, q₂a₃, q₃p, the projectione = b − p, the errorc, a candidate point

Drag to orbit · scroll to zoom · drag the arrow tips, or the candidate ring inside the subspace

Distance from b—

—

View

Project b onto

Vector b

x 1.00

y 1.20

z 2.60

a₁

x 2.20

y 0.60

z 0.00

a₂

x 0.40

y 2.00

z 0.50

Spin the scene

Values

x̂ = (AᵀA)⁻¹Aᵀb—

p = Ax̂—

e = b − p—

Aᵀe—

|e|² + |p − c|²—

|b − c|²—

What to observe

Press Run: the candidate c sweeps through the plane and its distance to b dips to its lowest exactly as it passes p. There b, p, c form a right triangle, so |b − c|² = |e|² + |p − c|².
Least squares is the same picture. With columns (1, 1, 1) and the times t, the plane holds every straight line, b holds the data, p the fitted values and e the residuals; Aᵀe = 0 are the normal equations. Thelinear models page fits lines this way.
Run Gram–Schmidt: each vector loses its shadow on the q's already made, and what is left is normalised. Every shadow length and leftover length lands in R, so A = QR.
Put a₃ in the plane of a₁ and a₂: once its two shadows are taken away nothing is left, r₃₃ = 0, and there is no third direction to find.

Shortcuts:space run/pause · s step · r reset · f fullscreen