A diamond has corners. A circle doesn't.
Why lasso (L1) can zero out a weight entirely while ridge (L2) never quite gets there, shown first as the geometry of the constraint region in weight space, then as a live curve fit on real, noisy data.
Regularization
Six optimizers walk into a bowl
SGD, Momentum, Nesterov, RMSProp, Adam and Prodigy, dropped from the same point on a real loss surface and raced to the bottom, live in your browser. First a clean bowl, then rougher terrain with false valleys.
OptimizersA layer without a bend does nothing. One bend changes everything.
Stack as many purely linear layers as you like and you still only get a straight decision boundary — the algebra collapses them into one. Watch a tiny network train live on a dataset only a nonlinearity can solve, architecture switchable mid-training.
Neural networksNothing learns on a flat line.
tanh, sigmoid, ReLU and SiLU on one axis, with their slopes underneath. Drag across and watch three of the four gradients fade toward nothing on the left — and see which one goes to zero permanently rather than just quietly.
Activation functions