Statistics
Wasserstein Distance
How far apart are two distributions? The Wasserstein (earth mover's) distance measures the least work needed to reshape one pile of probability into the other: mass times distance moved. Unlike overlap, it still knows two distant bumps are far apart.
Wasserstein W₁—
Overlap—
Top: the two distributions. Bottom: their running totals (CDFs). The shaded area between the CDFs is exactly W₁. Drag the top panel to move Q.
What to observe
- Slide Q's mean away from P. The Wasserstein distance grows steadily with the gap, because it counts how far the mass has to travel, the shaded area between the CDFs stretches with it.
- Watch Overlap instead: once the two bumps stop touching it hitszero and stays there, blind to whether they are near or a mile apart. That is why overlap-based losses fail to train generative models.
- In 1D the distance is simply the area between the two CDFs. Reshape P or widen Q and see how spreading mass out, not just moving it, also costs.
Shortcuts: space run/pause · s step · r reset · f fullscreen