Deep Learning simulators
Optimization, neural networks and the math that trains them.
How Layers Bend Space
Every hidden layer is two units wide, so the space it makes can be drawn. Step through the layers and watch the grid fold until one line separates the classes.
Gradient Descent
Tune the learning rate and the starting point, then watch optimization roll downhill into a minimum, or overshoot it and diverge.
Optimizers
Race SGD, Momentum, NAG, RMSProp and Adam down the same loss, drawn either as a surface you can click on or as a volumetric cloud in three parameters.
Neuron Activations
A network trains itself in front of you, then you draw a shape and watch it go through: every wire lit by what it carries, every neuron opened up to show the picture it looks for.
Embeddings & Cosine Similarity
Every word is an arrow and meaning is the direction it points. Rank the same words by cosine, by raw dot product and by straight-line distance, and watch the level sets turn from rays into rings.
Attention as a Soft Lookup
The mechanism on its own: a dictionary returns one value, attention returns a blend of all of them, weighted by how well each key matches the query. Sharpen it, flatten it, mask a key out.
Self-Attention
Where the queries and the keys come from: three projections of one X, so the sentence looks itself up. The full matrix, one head at a time, down to every multiplication.
Image Convolution
Slide a kernel over an image to detect edges, blur or sharpen, and see how stride, padding and dilation reshape the output feature map.