Machine Learning

Principal Component Analysis

Keep one number per point instead of two: its shadow on a line u. Turn u and watch how much of the spread the shadows keep. The best line keeps the most and throws away the least, and it is the top eigenvector of the covariance matrix.

the datau and the shadows on itresiduals: what u throws awayPC1PC2

Drag anywhere to turn u · drag a point to move it · any control takes over the tour

Spread kept by u—

—

Chapter

Data

Direction of u 100°

Squash onto u 0%

two numbers eachone number each

Before PCA

Standardize each axis (divide by its spread)

Show

Residuals to the line

Values

covariance C—

λ₁, λ₂ (variance along PC1, PC2)—

direction of PC1—

explained by PC1—

kept / lost by u—

What to observe

Turn u and watch the amber shadows spread out and bunch up. They are most spread along PC1 and least along PC2, always 90° apart. Those two are the eigenvectorsof the covariance matrix C, and the peak height is λ₁.
Kept plus lost is always the same total, by Pythagoras on every point. So the line that keeps the most spread is also the one with thesmallest residuals. Project onto it and each point becomes a single number.
On Round every direction keeps about half: λ₁ ≈ λ₂ and there is no best line to pick, so one number per point loses a lot. On Almost a line PC1 keeps nearly all of it.
On Mixed units PC1 lies along x only because x is written in bigger numbers. Standardize and it swings round to the real trend in the data.
Variance squares distances, so one far point pulls hard. On Roundthere is no trend to resist it, and a single outlier decides where PC1 points. Drag one out and watch.

Shortcuts:space run/pause · r reset · f fullscreen