Deep Learning simulators
Optimization, neural networks and the math that trains them.
How Layers Bend Space
Every hidden layer is two units wide, so the space it makes can be drawn. Step through the layers and watch the grid fold until one line separates the classes.
Backpropagation
One tiny network, one example, every number on screen. Push the input forward, send the error back down the same wires, then see why the activation's slope decides whether a gradient survives many layers.
Optimizers
Six optimizers descend the same loss: a 1D slice where too big a learning rate overshoots and diverges, a surface you can click on, or a density cloud in three parameters.
Batch Size & Gradient Noise
Nobody computes the real gradient. Watch which examples get drawn, watch each one pull the step its own way, and watch their average swing into line with the truth as the batch grows.
Initialisation & Normalisation
Every layer multiplies the spread of the signal by a factor, and depth compounds it. Pick a weight scale and watch the activations survive, vanish or overflow, then normalise each layer and see the scale stop compounding.
Softmax & Temperature
Raw scores become probabilities, and one number decides how strong an opinion that is. Drag the temperature from a single confident answer all the way to a five-way shrug.
Inside a Digit Classifier
Draw a digit and watch a pretrained MNIST classifier read it, as a plain stack of layers or as a convolutional network. Every wire is lit by what it carries, and clicking a neuron opens it up to show the picture it looks for.
Contrastive Learning
Nobody says where anything goes. Things of a kind are pulled together and everything else pushed apart, and the embedding arranges itself from that. Four losses say it four different ways, and you can watch each one push.
Embeddings & Cosine Similarity
Every word is an arrow and meaning is the direction it points. Rank the same words by cosine, by raw dot product and by straight-line distance, and watch the level sets turn from rays into rings.
Positional Encoding
Attention reads a set, so order has to be written into the vectors. Added sinusoids make a score depend on where two words sit; rotary makes it depend on their offset alone.
Convolution
A window of weights slides over a grid and each position gives one number. Padding, stride and dilation decide how many positions there are, and with several channels every filter reads all of them at once and sums the slices.
Receptive Field
Trace the cone of pixels one output cell can see. Depth adds to the window, stride multiplies it and costs resolution, doubling dilation grows it exponentially for free, and a repeated dilation leaves holes.
Autoencoder & Latent Space
Squeeze a picture through a gap of two numbers and force the network to rebuild it. Nothing says what the numbers should mean, and they come out as a map you can drag around.