Generative AI

The Data Manifold

A 28 × 28 image is one point in a space of 784 dimensions, yet every image of one kind sits on a thin curved sheet inside it. Walk the sheet, average two images, add noise and pull it back: generative models live on that sheet.

the sheet of real imagesyour image xA → B by averaging pixelsA → B along the sheetnoise

Drag on the sheet to move x · drag A or B · drag empty space to orbit · scroll to zoom · click a frame in a strip

Distance to the sheet—

—

Image family

Rotation 0°

Stroke width 2.7 px

Go from A to B by

Position along A → B 0.50

Noise σ 0.00

cleannoisy

Show

Density fogKnob gridThumbnailsDenoising field

Values

pixel-space dimensions—

knobs that make the images—

variance in PC1 · PC2 · PC3—

knobs of x—

knobs of the nearest sheet image—

distance to the sheet—

x in PC1, PC2, PC3—

width of the sheet—

random pixels to the sheet—

render + PCA time—

What to observe

Drag along the sheet. The digit turns and thickens smoothly and its distance to the sheet stays 0: 784 pixel values, all fixed by justtwo knobs.
Go from A to B by averaging pixels and the path is a straight chord that leaves the sheet: the midpoint is a ghostly double exposure. Walk the sheet instead and every frame is a real image.
Add noise and the image lifts off the sheet, while its PCA dot hardly moves. Denoise pulls it back to the nearest real image, which is what a diffusion model learns: a field pointing back to the sheet.
Random pixels are static, about as far from the nearest real image as the whole sheet is wide. Yet in PC1-PC3 the dot lands somewhere ordinary: three axes hide the other 781.

Shortcuts:r reset · f fullscreen