Machine Learning
Perplexity of a Neighbourhood
Give a point a Gaussian and its neighbours become probabilities. Perplexity, two to the entropy, counts how many neighbours that really is, and t-SNE picks a width for every point so the count comes out the same: narrow in a crowd, wide out on its own.
the chosen point i and its σᵢneighbours, sized by p(j | i)a flat spread over Perp neighboursthe binary search
Click a point to choose it · drag points to move them
Width of the Gaussian—
—
Points
few neighbourseveryone
Show
Values
σᵢ—
distance to the nearest point—
entropy H(Pᵢ)—
perplexity 2^H—
points holding 90% of p—
search steps—
What to observe
- Perplexity is a smooth count of neighbours. At 5 the sorted bars hold about five big ones and a tail; the teal box is a flat spread over exactly 5 that has the same entropy.
- Turn on σ for every point. Inside the dense group the circles are small, in the sparse one they are wide: the same perplexity means the same number of neighbours, not the same distance. This is how t-SNE copes with clusters of different density.
- The outlier has no near neighbours, so its σ grows until it takes in about Perp points from the group, however far away they are. Every point gets its share, even the lonely ones.
- Wider σ always means a flatter spread and a higher perplexity, so σ can be found by halving an interval: too few neighbours, widen; too many, narrow. Each red dot is one guess.
- Perplexity can never pass N − 1, every other point equally likely. Push it to the top and σ shoots off: the neighbourhood stops being local, and the structure t-SNE would keep is gone.
Shortcuts:space animate · r reset · f fullscreen