Architectures · intermediate · concept 87 of 311 · Updated Nov 4, 2026
Residual Connections & ResNet
A shortcut that adds a layer's input to its output, letting gradients flow through very deep networks instead of vanishing. ResNet used it to jump from 20-layer to 150-layer networks in 2015, and every transformer block since is built around the same skip connection.
Top AI contributors on this
- Geoffrey HintonUniversity of Toronto (emeritus)
- Yoshua BengioLawZero / Université de Montréal
- Yann LeCunAMI Labs / NYU
- Andrew NgDeepLearning.AI / AI Fund
- Ilya SutskeverSafe Superintelligence Inc. (SSI)
- Andrej KarpathyAnthropic
Key terms
Skip connectionIdentity mappingVanishing gradientsDeep networksResNet
Learn these first
Where you meet it in the real world
Inside every modern architecture: CNNs, transformers, diffusion U-Nets, all of them
Videos
▶ C4W2L03 Resnets ↗
DeepLearningAI · YouTube
▶ [Classic] Deep Residual Learning for Image Recognition (Paper Explained) ↗
Yannic Kilcher · YouTube
Guides and articles
8.6. Residual Networks (ResNet) and ResNeXt: Dive into Deep Learning 1.0.3 documentation ↗
Dive into Deep Learning
[1512.03385] Deep Residual Learning for Image Recognition ↗
arXiv (original paper)