Architectures · intermediate · concept 61 of 176

🏗️Normalization Layers (BatchNorm to RMSNorm)

Layers that rescale activations so training stays stable as networks get deep. BatchNorm made deep CNNs trainable, LayerNorm made transformers possible, and RMSNorm is the leaner variant inside most current LLMs. Unglamorous, and load-bearing for everything.