Topic

Gradient Descent

Gradient descent is how a model improves: measure the error, find which direction lowers it, take a small step and repeat. The size of that step, the learning rate, decides whether training settles, crawls or blows up. This one idea, repeated billions of times with backpropagation supplying the direction, is what training a neural network amounts to.

1 guide on Gradient Descent.

Reference Guide · 20 min read

How Neural Networks Work

The canonical 784-16-16-10 MNIST network as an explorable 3D machine: 16 stations across 4 acts, a 16-step guided journey from pixels to the brain question, all 12,960 weight connections really drawn, and a training row where gradient descent is really computed on an illustrative loss terrain, and a Train mode where the network genuinely learns 10,000 real MNIST digits in your browser.

Neural Networks Deep Learning Backpropagation Gradient Descent MNIST 3D Interactive AI Explainer

Keep learning Gradient Descent

Gradient descent, how neural networks learn | Deep Learning Chapter 2 video · 3Blue1Brown
Gradient Descent, Step-by-Step video · StatQuest with Josh Starmer
Stanford CS229: Machine Learning - Linear Regression and Gradient Descent | Lecture 2 (Autumn 2018) video · Stanford Online
A note on why gradient descent is even needed in the first place post · r/MachineLearning
An overview of gradient descent optimization algorithms guide · Sebastian Ruder
Batch gradient descent versus stochastic gradient descent post · Cross Validated
Every Model Learned by Gradient Descent Is Approximately a Kernel Machine post · Hacker News
Gradient descent (Wikipedia) guide · Wikipedia

From the AI Roadmap. Every link was checked.

3D Interactive AI Explainer Backpropagation Deep Learning MNIST Neural Networks

All Guides

Gradient Descent

How Neural Networks Work

Keep learning Gradient Descent

Related topics