Machine Learning simulators
Clustering, model fitting and the ideas that turn data into predictions.
k-Means
Assign points to the nearest center, move each center to its mean, repeat. Watch clusters snap into place, and sometimes into the wrong place.
Gaussian Mixtures
k-means hands every point to one cluster and draws only circles. A mixture gives every point a share of each cluster instead, and lets a cluster stretch and tilt, so a point between two of them can finally say so.
DBSCAN
Density-based clustering that finds arbitrary shapes and flags outliers as noise, with no need to choose the number of clusters up front. A second view works out reachability and connectivity on small cases, one point at a time.
Hierarchical Clustering
Merge the two closest groups over and over until everything is one, watch the dendrogram grow branch by branch, and drag a cut across it to choose the clusters.
Clustering Metrics
Silhouette, Davies–Bouldin and DBCV judge a clustering by its geometry alone; purity, the Rand index and ARI hold it up against the true classes. Split, scramble and hand-edit clusters and watch where the six scores part ways.
Linear Models
Linear and logistic regression score the input with the same line, z = wx + b. Draw each residual as a literal square, snap to the least-squares optimum, and watch the same descent fit both models.
Overfitting
Raise a polynomial’s degree until it threads every training point yet fails on new data, then tame it with regularization.
The Curse of Dimensionality
Add axes and space stops behaving: data thins out, a ball keeps its volume in a paper-thin crust, and near and far collapse into the same distance.
Principal Component Analysis
Keep one number per point: its shadow on a line. Turn the line and watch the spread it keeps rise and fall; the best line keeps the most, loses the least, and is the top eigenvector of the covariance.
Perplexity of a Neighbourhood
Put a Gaussian on one point and its neighbours become probabilities. Perplexity counts how many of them really count, and a binary search finds the width that makes the count come out right: narrow in a crowd, wide for a loner.
t-SNE
Tie each point of a 3D cloud to its neighbours with threads, throw everything on a flat page, and let the threads pull while all points push apart. The map that settles keeps who was next to whom.