📐Backpropagation

Untrained, the network's 13,002 parameters are random and its guesses are garbage. Training fixes that with three mechanisms. First, a loss function turns each wrong answer into a single number: cross-entropy, the standard choice for classification, is minus the log of the probability the network gave the correct answer, so confident wrongness is punished brutally.

Gradient descent

Second, gradient descent treats the loss as a landscape over all 13,002 parameters and repeatedly steps against the local slope: parameters minus learning rate times gradient. In practice the slope is estimated on small random mini-batches of examples rather than the whole dataset, trading noise for enormous speed. The math ancestry runs back to Robbins and Monro's stochastic approximation in 1951.

Backpropagation: the price collapse

Third, computing the slope for every parameter separately would cost 13,002 forward passes. Backpropagation runs the chain rule backward through the network and delivers all of them for roughly the cost of two or three forward passes, total, regardless of parameter count. That cheap-gradient property is the economic fact that makes deep learning possible at all. Credit history, kept honest: Linnainmaa published the underlying reverse-mode differentiation in 1970; Rumelhart, Hinton, and Williams rediscovered and popularized it for networks in 1986.

See every piece run

The 3D explainer stages each mechanism: the loss meter fills with real cross-entropy values, the descent balls roll on an actual computed landscape, the backward amber wave traces backprop through every drawn fiber, and Train mode runs the whole assembly on real digits until held-out accuracy passes ninety percent in front of you.