Intermediate · About 92 hours · 4 projects · Free · Updated Nov 4, 2026
AI Roadmap for Developers
In about 92 hours you ship four things you can show: a classic ML model with a demo, a deep learning model on a public URL, a RAG system over your own documents, and an agent running in production with evals and monitoring.
0of 92 hours · 0%
Saved in this browser. No account, no email.
What you'll build
Go at your own pace. Stop at any milestone: each one leaves you with something to show.
- 1A classic ML model, shippedhour 28
Take one tabular dataset from raw file to a tuned model with honest metrics, behind a demo anyone can run.
- 2A deep learning model you trained, tracked and deployedhour 58
Fine-tune a pretrained model, track your runs, and deploy it on a public URL.
- 3A RAG system over your own documentshour 75
Build a retrieval app over your own documents that answers with citations, score it, and put it on a public URL.
- 4An agent in productionhour 92
Turn your app into a monitored agent: tools, human approval, an injection guard, a container and CI.
Skills you'll learn
- Python
- pandas
- scikit-learn
- Cross-validation
- Streamlit
- MLflow
- PyTorch
- Backpropagation
- Transfer learning
- Experiment tracking
- Gradio
- Hugging Face Spaces
- LLM APIs
- Embeddings
- Vector search
- Chunking
- Citations
- Evals
- Agents
- Tool calling
- Human approval
- Prompt injection
- Docker
- CI
- Observability
Prerequisites
- You write code for a living, in any language.
- You are comfortable with git and the command line.
- You have a laptop that runs Python 3.11 or later. A free Colab account covers the GPU steps.
Program outline
4 milestones · 16 sections · 80 lessons · 4 projects · Every course and source is free
Milestone 1 · Hours 1 to 28 · 5 sections · 25 lessons
Python, data and your first model
Learn Python, the math you need and data work, then train and ship your first model.
Hours 1 to 50/14
By the end you have a working Python setup, a map of the field, and a script that loads, transforms and saves real data.
In the Encyclopedia: What is AI?, Python Programming for AI
The map of the field in one hour55 min
- WatchWhat Is Artificial Intelligence? · Crash Course AI #1 · CrashCourse
- CodeAI-Expert-Roadmap · Visual roadmap · AMAI-GmbH/AI-Expert-Roadmap
- ReadGoogle AI Education · Google
- DoWrite a 10-line note: what narrow AI, machine learning, deep learning and LLMs each mean, and which one your current product already uses.
Python for AI: setup and the parts you actually use65 min
- ReadPython.org Official Tutorial · docs.python.org
- Code30-Days-Of-Python · Asabeneh/30-Days-Of-Python
- DoInstall Python 3.12 with uv or venv, create a project repo, and write a script that uses list comprehensions, dicts and a dataclass. Run it and commit.
Functions, classes, files and errors65 min
- ReadAutomate the Boring Stuff · Free Book · automatetheboringstuff.com
- DoWrite a script that reads a CSV you have at work (or any public one), counts one column by category, and writes a JSON summary with error handling.
CS50P as a drill set70 min
- CourseCS50P: Introduction to Programming with Python, Harvard (free) · cs50.harvard.edu
- DoPick one CS50P problem (the CSV cleaning one is a good fit) and solve it end to end with a pytest test for it.
The Python idioms ML code is written in65 min
- WatchPython Tutorial · Corey Schafer Playlist · Corey Schafer
- Codepython-guide · Hitchhiker's Guide · realpython/python-guide
- DoWatch one Corey Schafer video on generators or decorators, then set up Jupyter or VS Code notebooks, rerun your scripts so far in a notebook, and commit.
Hours 6 to 110/14
By the end you can read a matrix multiplication, a gradient and a probability distribution in code, and you have written each one in plain Python.
In the Encyclopedia: Math for AI, Statistics & Probability for ML
Vectors and matrices, seen65 min
- WatchEssence of Linear Algebra · 3Blue1Brown (Full Series) · 3Blue1Brown
- DoWatch the first four chapters. Write dot product and matrix multiply as plain Python functions and test them against hand-computed 2x2 cases.
Matrix shapes, transformations, and why ML cares65 min
- WatchMathematics for Machine Learning: Linear Algebra, Imperial College London (free lectures) · Digital Media - Imperial College London
- Codemath-as-code · Math symbols explained in code · Jam3/math-as-code
- DoRewrite five formulas from math-as-code in Python, and write the shape rule (m by n times n by p) as a test that fails on a bad shape.
Derivatives and gradients: the one calculus idea you need70 min
- WatchEssence of Calculus · 3Blue1Brown · 3Blue1Brown
- Read3Blue1Brown · Interactive Lessons · 3blue1brown.com
- DoImplement numerical gradient descent on f(x, y) = x squared plus 3 y squared in 20 lines and print the path to the minimum.
Distributions, mean, variance and p-values65 min
- WatchStatistics Fundamentals · StatQuest (Full Playlist) · StatQuest with Josh Starmer
- ReadSeeing Theory · Visual intro to probability · seeing-theory.brown.edu
- DoWatch three StatQuest videos. Simulate 10,000 coin flips and 10,000 normal samples with the random module; compute mean, standard deviation and a 95 percent interval; print a text histogram.
Bayes, conditional probability and the stats behind ML70 min
- WatchBayesian Statistics · 3Blue1Brown · 3Blue1Brown
- ReadOpenIntro Statistics (free PDF) · openintro.org
- DoRead the OpenIntro probability chapter. Write a tiny naive Bayes spam filter on 20 messages you type yourself and print its predictions.
Hours 12 to 170/15
By the end you can load, clean, reshape and chart a real dataset with NumPy and pandas, and you have one SQL query and one cleaning script you will reuse.
In the Encyclopedia: NumPy, Pandas & Data Visualization, Data Engineering & Data Pipelines for AI
NumPy: arrays, broadcasting, vectorized code70 min
- ReadReal Python · NumPy Tutorial · realpython.com
- Codenumpy-100 · 100 NumPy exercises · rougier/numpy-100
- DoDo the first 25 exercises of numpy-100. Rewrite your earlier matrix multiply and gradient descent in NumPy and time both against pure Python at n = 500.
pandas: load, filter, group, join65 min
- Course
- DoDo two Kaggle Pandas lessons. Load a CSV from work (or UCI), answer three questions with groupby, merge and pivot, and save the result.
Charts that tell you something70 min
- WatchMatplotlib Tutorial · Corey Schafer · Corey Schafer
- ReadPython Data Science Handbook · Free Online · jakevdp.github.io
- DoMake three charts of the dataset from the last step: a histogram, a time series, and a scatter with a trend line. Save them as PNGs and commit.
Cleaning real data: missing values, scaling, encoding70 min
- Course
- WatchFeature Scaling & Encoding · Krish Naik · Krish Naik
- DoWrite clean.py: a function that takes a DataFrame, fills or drops missing values, scales numeric columns, one-hot encodes categoricals, and returns features plus a report dict.
SQL and a pipeline you can rerun75 min
- CourseMode Analytics · SQL Tutorial · mode.com
- WatchData Pipeline Overview · ByteByteGo
- Codeydata-profiling · Auto EDA · ydataai/ydata-profiling
- DoLoad the clean CSV into a local Postgres (or DuckDB), write the query that reproduces your earlier groupby, and a run.py that does clean, load, query end to end.
Hours 18 to 230/14
By the end you have trained a regression and a classification model with scikit-learn, and you can say with numbers whether either one is any good.
In the Encyclopedia: Intro to Machine Learning, Model Evaluation & Experiment Tracking
Machine learning in code before theory65 min
- CourseKaggle Learn · Intro to ML (Free) · Kaggle
- DoDo lessons 1 to 4 of Kaggle Intro to ML. Train a DecisionTreeRegressor on the housing data and print MAE on a held-out split.
Supervised learning: regression, classification, overfitting70 min
- CourseGoogle ML Crash Course (Free) · Google
- DoDo the Linear Regression and Logistic Regression modules. Fit both on your cleaned dataset and plot a learning curve: training against validation error as training size grows.
The whole workflow, Geron's way70 min
- CodeHands-On Machine Learning notebooks, Aurelien Geron (free code) · ageron/handson-ml3
- DoRun the chapter 2 end-to-end notebook top to bottom. Change one feature engineering step and record how the RMSE moved.
Metrics: confusion matrix, precision, recall, ROC70 min
- WatchConfusion Matrix · StatQuest · StatQuest with Josh Starmer
- WatchROC & AUC Explained · StatQuest (MUST WATCH) · StatQuest with Josh Starmer
- Readscikit-learn · Model Evaluation Guide · scikit-learn
- DoWrite evaluate.py that prints a confusion matrix, precision, recall, F1 and AUC for your classifier from earlier. Move the threshold and watch precision and recall trade.
Cross-validation and experiment tracking70 min
- WatchCross-Validation · StatQuest · StatQuest with Josh Starmer
- ReadMLflow · Getting Started Guide · mlflow.org
- Read
- DoRead rules 1 to 10 of Rules of ML. Wrap training in 5-fold cross-validation, log every run's params and metrics to MLflow, open the UI and compare two runs.
Hours 24 to 280/15
By the end you have shipped a complete ML project: data, features, a tuned model, honest metrics and a demo someone else can click.
In the Encyclopedia: Scikit-Learn & Hands-On ML Projects, 🧪 PRACTICE MILESTONE: Build Your First ML Project
Feature engineering and pipelines70 min
- CourseKaggle Learn · Feature Engineering · Kaggle
- ReadWhat's the difference between fit and fit_transform in scikit-learn models? · Data Science Stack Exchange
- DoDo two Kaggle Feature Engineering lessons. Build a scikit-learn Pipeline (ColumnTransformer plus model) for your dataset, add two engineered features and compare CV scores.
Gradient boosting and tuning75 min
- Course
- CodeApproaching (Almost) Any ML Problem · Thakur (free PDF) · abhishekkrthakur/approachingalmost
- DoDo the XGBoost and pipelines lessons, read Thakur's chapter on hyperparameter tuning. Swap in gradient boosting, run RandomizedSearchCV, log the runs.
Project part 1: pick the problem, frame it, baseline it70 min
- Read
- ReadKaggle · House Prices Competition · Kaggle
- CodeHands-On ML end-to-end project notebook, Aurelien Geron (free code) · ageron/handson-ml3
- DoPick one: Titanic, House Prices, or a dataset from your work. Write the README: the problem, the metric, and the score of a dumb baseline model.
Project part 2: train, evaluate, iterate65 min
- Watch
- DoTrain three models, pick one by cross-validation, put the metrics table in the README, and save the model with joblib.
Project part 3: a demo someone can click65 min
- WatchBuilding ML Apps with Streamlit · Patrick Loeber · Patrick Loeber
- ReadStreamlit · Build ML Web Apps · streamlit.io
- DoBuild app.py: a Streamlit form that takes the features and returns the prediction. Run it, record a 30-second screen capture, link it from the README.
Milestone 1 project · about 8 hours
A classic ML model, shipped
Take one tabular dataset from raw file to a tuned model with honest metrics, behind a demo anyone can run.
- Python
- pandas
- scikit-learn
- Cross-validation
- Streamlit
- MLflow
Courses that teach this
- Intro to Machine Learning
Kaggle · about 3 hours
Build and tune your first models in Python, with exercises in the browser.
- Intermediate Machine Learning
Kaggle, Alexis Cook · 4 hours
Handle missing and categorical data, build pipelines, cross-validate, and avoid data leakage.
- Machine Learning Crash Course
Google · about 15 hours
Core machine learning with interactive exercises, from regression to production.
Build it
Pick one tabular dataset: Titanic, House Prices, or one from your work.
Clean it in a script and split it, keeping a test set you never train on.
Build a scikit-learn Pipeline with preprocessing and a baseline model, then beat the baseline.
Tune it and report cross-validated metrics, including where it fails.
Save the model and put it behind a Streamlit form someone else can run with one command.
You finish with
A GitHub repo with clean.py, a Pipeline, a metrics table from cross-validation, a saved model, an MLflow run log, and a demo someone else can run with one command.
Check your work
The metrics come from cross-validation, and the test set was never used to tune.
The model beats a simple baseline, and the README shows by how much.
Someone else can run the demo with one command.
The README says where the model fails.
See how others did it
- Chapter 2: End-to-end Machine Learning project
ageron/handson-ml3
Tabular project end to end: pipelines, cross-validation, grid search, joblib model saving.
- Column Transformer with Mixed Types
scikit-learn.org
Build a mixed-type preprocessing pipeline on Titanic, then tune it with randomized search.
- Create an app
docs.streamlit.io
Build a Streamlit app with widgets and caching, then deploy it on Community Cloud.
- Caching overview
docs.streamlit.io
Load a trained model once with st.cache_resource so each session stays fast.
Milestone 2 · Hours 29 to 58 · 5 sections · 25 lessons
Deep learning with PyTorch
Write a neural network from scratch, learn PyTorch, build a small GPT and deploy a model.
Hours 29 to 340/12
By the end you have written a neural network from scratch in NumPy that learns handwritten digits, and you can explain backpropagation line by line.
In the Encyclopedia: Neural Networks & Deep Learning Fundamentals
What a neural network is60 min
- Watch
- DoWatch chapters 1 and 2. Draw the forward pass of a two-layer network for a 4-pixel image by hand, then write it as 15 lines of NumPy.
Backpropagation65 min
- ReadNeural Networks & Deep Learning · Free Online Book by Michael Nielsen · neuralnetworksanddeeplearning.com
- DoRead Nielsen's chapter 2. Implement the gradient of a single sigmoid neuron by hand and check it against a numerical gradient.
From scratch, with code75 min
- WatchNeural Networks from Scratch · Sentdex · sentdex
- Codeneural-networks-and-deep-learning · Nielsen's book code · mnielsen/neural-networks-and-deep-learning
- DoWatch two Sentdex videos. Write Layer, forward, backward and an SGD loop in NumPy; train on a 2D toy dataset and plot the decision boundary.
Training on MNIST in NumPy75 min
- Codeneural-networks-and-deep-learning · Nielsen's book code · mnielsen/neural-networks-and-deep-learning
- Read
- DoRead d2l's multilayer perceptron chapter. Train your NumPy network on MNIST (load it with scikit-learn or torchvision), pass 90 percent test accuracy, and save the loss curve.
Loss, activations, optimizers: the vocabulary75 min
- WatchMIT 6.S191 · Intro to Deep Learning (2026) · Alexander Amini
- DoWatch MIT 6.S191 lecture 1. Swap sigmoid for ReLU and plain SGD for momentum in your network, rerun, and write a 10-line note on what changed and why.
Hours 35 to 400/12
By the end you can build, train, evaluate and save a PyTorch model with a proper training loop, data loaders and GPU support, and your NumPy network is retired.
In the Encyclopedia: PyTorch
Tensors and autograd70 min
- CourseLearn PyTorch · learnpytorch.io (Free) · learnpytorch.io
- DoDo chapter 00 of Learn PyTorch. Re-implement your plain Python gradient descent with torch tensors and autograd and compare the gradients with your numerical ones.
The training loop70 min
- ReadPyTorch Official Tutorials · PyTorch
- DoDo Learn the Basics from the official tutorials. Write train.py with a model, loss, optimizer, train and eval loops, and a saved checkpoint.
nn.Module, datasets and data loaders70 min
- CourseLearn PyTorch · learnpytorch.io (Free) · learnpytorch.io
- DoDo chapter 02 (classification). Port your NumPy MNIST network to nn.Module with a DataLoader and match or beat its accuracy.
micrograd: the engine under autograd75 min
- WatchAndrej Karpathy · Neural Networks: Zero to Hero (LEGENDARY) · Andrej Karpathy
- DoWatch the first hour of lecture 1 and code along: build Value with backward() and verify it against torch on one expression.
GPU, checkpoints, reproducibility70 min
- Codepytorch/examples · Official examples · pytorch/examples
- ReadOfficial PyTorch Tutorials (free) · docs.pytorch.org
- ReadWhat is the use of torch.no_grad in pytorch? · Data Science Stack Exchange
- DoAdd seed control, device selection (CUDA, MPS or CPU), and checkpoint save and resume to train.py. Run it twice and confirm the loss is identical.
Hours 41 to 460/16
By the end you have fine-tuned a pretrained CNN on your own images and trained a character-level RNN, and you know where each one lost to transformers.
In the Encyclopedia: CNNs & Computer Vision, RNNs, LSTMs & Sequence Models
Convolutions and pooling, and why they work60 min
- ReadCNN Explainer · Interactive Visualization · poloclub.github.io
- WatchNeural Networks Part 8: Image Classification with CNNs · StatQuest · StatQuest with Josh Starmer
- DoImplement a 2D convolution in NumPy and run it with an edge-detection kernel on one image; show the input and output side by side.
Train a CNN on images in PyTorch75 min
- Course
- CodeHands-On ML CNN notebook, Aurelien Geron (free code) · ageron/handson-ml3
- DoDo Kaggle Computer Vision lessons 1 to 3. Train a small CNN on Fashion-MNIST or CIFAR-10 for 3 epochs, log accuracy, and try one augmentation.
Transfer learning: fine-tune a pretrained ResNet75 min
- WatchTransfer Learning · Andrew Ng (C3W2L07) · DeepLearningAI
- Paper
- Codepytorch/vision (torchvision) · Official CV library · pytorch/vision
- DoRead sections 1 and 3 of the ResNet paper. Fine-tune torchvision's resnet18 on 100 of your own photos in two classes and report accuracy.
Sequences: RNNs, LSTMs and the vanishing gradient65 min
- WatchRNN & LSTM Explained · StatQuest · StatQuest with Josh Starmer
- Read
- DoWrite a vanilla RNN cell's forward pass in NumPy and print how the gradient shrinks over 50 steps.
char-RNN: generate text70 min
- ReadThe Unreasonable Effectiveness of RNNs · Karpathy Blog · Andrej Karpathy
- Codepytorch/examples/word_language_model · pytorch/examples
- DoTrain a character-level LSTM in PyTorch on a text file of your choosing (your repo's commit log works) for 10 minutes and sample 500 characters.
Hours 47 to 520/13
By the end you have built a small GPT from scratch in PyTorch, trained it on text, and you can draw attention from memory.
In the Encyclopedia: Transformers & Attention Mechanism
Attention, illustrated70 min
- Read3Blue1Brown · But what is a GPT? + Attention, step by step · 3blue1brown.com
- DoWatch both 3Blue1Brown chapters. Write scaled dot-product attention in 10 lines of NumPy and check that every softmax row sums to one.
The Illustrated Transformer and the paper75 min
- ReadThe Illustrated Transformer · Jay Alammar (MUST READ) · Jay Alammar
- Paper
- DoRead section 3 of Attention Is All You Need and annotate it: one sentence of your own per equation.
Build GPT from scratch, part 175 min
- WatchAndrej Karpathy · Let's build GPT from scratch · Andrej Karpathy
- DoWatch the first hour and code along: the bigram model and the data pipeline. Train it.
Build GPT from scratch, part 275 min
- WatchAndrej Karpathy · Let's build GPT from scratch · Andrej Karpathy
- Codekarpathy/nanoGPT · Build GPT from scratch (38K+ stars) · Andrej Karpathy
- DoFinish the attention head, multi-head attention and the block; train on Tiny Shakespeare; sample text. Compare your code with nanoGPT's model.py.
Encoder, decoder, BERT, GPT and the Hugging Face library75 min
- ReadDecoder-Only Transformers: The Workhorse of Generative LLMs · Cameron R. Wolfe
- CourseHugging Face LLM Course (Free) · Hugging Face
- DoDo chapter 1 of the Hugging Face LLM Course. Run three pipelines (sentiment, fill-mask, generation) and note which architecture each model uses.
Hours 53 to 580/17
By the end your deep learning project is deployed on a public URL with a demo, and you have fine-tuned a text classifier with Hugging Face.
In the Encyclopedia: 🏗️ PRACTICE MILESTONE: Deep Learning Projects, NLP & Text Processing Fundamentals
Project part 1: pick, scope, baseline60 min
- ReadKaggle · Classification Competitions · Kaggle
- ReadPapers With Code · Reproduce a Paper · paperswithcode.com
- Codepytorch/examples · Official examples · pytorch/examples
- DoChoose: fine-tune a CNN on your own image set, or train a text classifier. Write the README with the metric and a baseline score.
Project part 2: train, track, improve70 min
- Read
- Codehuggingface/transformers · THE library (140K+ stars) · huggingface/transformers
- DoTrain three runs with different learning rates, log them to MLflow, and keep the best checkpoint.
Project part 3: deploy on Hugging Face Spaces75 min
- ReadGradio · Build ML Demos in Minutes · gradio.app
- ReadHugging Face Spaces · Deploy ML Demos for Free · Hugging Face
- WatchDeploy ML Models with FastAPI · AssemblyAI · AssemblyAI
- DoWrap the model in a Gradio app, push it to a Space, send the URL to a colleague and ask them to break it. Log what broke.
Tokens, embeddings, word2vec60 min
- WatchWord2Vec Explained · StatQuest · StatQuest with Josh Starmer
- ReadspaCy 101 · Official Tutorial · spacy.io
- DoTokenize 1,000 sentences with spaCy, count the tokens, and print the five nearest neighbors of five words using spaCy's vectors.
Text classification with Hugging Face75 min
- CourseHugging Face LLM Course (Free) · Hugging Face
- WatchHugging Face NLP Course · Walkthrough · Hugging Face
- DoDo chapter 3 of the LLM Course. Fine-tune distilbert on a small labeled set (an SST-2 subset, or your own support tickets) and report accuracy.
Milestone 2 project · about 10 hours
A deep learning model you trained, tracked and deployed
Fine-tune a pretrained model, track your runs, and deploy it on a public URL.
- PyTorch
- Backpropagation
- Transfer learning
- Experiment tracking
- Gradio
- Hugging Face Spaces
Courses that teach this
- Practical Deep Learning for Coders
fast.ai, Jeremy Howard · 9 lessons of about 90 min
Build and deploy your own deep learning model on data you collect.
- Neural Networks: Zero to Hero
Andrej Karpathy · 8 lectures, up to 2h25m each
Code neural networks from scratch, from backpropagation up to a GPT and tokenizer.
- Learn PyTorch for Deep Learning
Daniel Bourke · about 40 hours
Write PyTorch from tensors to transfer learning, paper replication and deployment.
- LLM Course
Hugging Face · 6 to 8 hours per chapter
Use, fine-tune and share language models with the Transformers library.
Build it
Fine-tune a pretrained CNN on your own images, or a text classifier with Hugging Face.
Measure a simple baseline first, so the model has something to beat.
Run at least three tracked experiments and record what each one changed.
Pick a checkpoint and say why, using the metric on held-out data.
Deploy a Gradio demo on a Hugging Face Space and ask a colleague to break it.
You finish with
A public Gradio demo on a Space, the training script with seed and checkpoint control, the run log, and a README with the metric, the baseline, and what a colleague broke when they tried it.
Check your work
The model beats the baseline on held-out data.
Three runs are logged, and you can say why you chose the checkpoint.
The training script reproduces the result from a fixed seed.
A stranger can use the public demo.
See how others did it
- 09. PyTorch Model Deployment
learnpytorch.io
Fine-tune EffNetB2, wrap it in a Gradio demo, deploy it to Hugging Face Spaces.
- 07. PyTorch Experiment Tracking
learnpytorch.io
Run a series of modelling experiments and compare them in TensorBoard.
- Text classification
huggingface.co
Fine-tune DistilBERT on IMDb with Trainer, then push the model to the Hub.
- gradio-app/trackio
gradio-app/trackio
Log runs with a wandb-style API and host the dashboard on a Space.
Milestone 3 · Hours 59 to 75 · 3 sections · 15 lessons
LLMs and RAG
Call models from code, learn prompting and structured output, then build retrieval over your own documents.
Hours 59 to 640/16
By the end you call a model from code with a system prompt, structured JSON output and a tool call, and you can explain how it was trained.
In the Encyclopedia: Prompt Engineering & Using LLMs Effectively, Tool Use & Function Calling, Large Language Models (LLMs)
Prompting as an engineering discipline65 min
- ReadAnthropic · Prompt Engineering Guide (MUST READ) · platform.claude.com
- ReadOpenAI · Prompt Engineering Guide · OpenAI
- DoWrite a prompt file for one task from your job (classifying tickets, say). Run it on 10 examples through a free tier or a local model and score the answers by hand.
The interactive tutorial, few-shot and chain of thought75 min
- Codeanthropics/prompt-eng-interactive-tutorial · anthropics/prompt-eng-interactive-tutorial
- DoDo chapters 1 to 5 of the tutorial. Add few-shot examples and a reasoning step to your first prompt, re-score the 10 examples, and record the delta.
Tool calling and structured outputs75 min
- ReadTool use with Claude · Anthropic
- ReadFunction calling · OpenAI
- CodeClaude Cookbooks · Anthropic
- DoWrite a 60-line script in which the model must return JSON that matches a schema and call one function you define (look up a record). Validate with pydantic and reject bad output.
How LLMs work, from Karpathy75 min
- WatchAndrej Karpathy · Intro to Large Language Models (1hr) · Andrej Karpathy
- DoWrite a one-page explainer for your team: pretraining, fine-tuning, RLHF and in-context learning, in your words.
Run one locally, read the scaling law75 min
- Codeollama/ollama · Run LLMs locally (100K+ stars) · ollama/ollama
- PaperScaling Laws for Neural Language Models · Kaplan et al. · Jared Kaplan
- ReadUnderstanding Large Language Models · Sebastian Raschka
- DoRun a small open model with Ollama, point your first script at it, and compare answers and latency with the hosted model.
Hours 65 to 690/15
By the end you have a working RAG pipeline over your own documents with a 20-question eval set and a score, and you know when to fine-tune instead.
In the Encyclopedia: Fine-Tuning & RAG, LLM Evaluation & Benchmarks
RAG, embeddings and vector search65 min
- WatchRAG Explained in 5 Minutes · IBM · IBM Technology
- WatchVector Databases Explained · Fireship · Fireship
- ReadPinecone Learning Center · Vector DBs & RAG · Pinecone
- DoEmbed 50 chunks of your own docs with a local embedding model, find the three nearest chunks for five questions, and print them.
Build the RAG pipeline75 min
- WatchBuilding RAG Applications · LangChain · pixegami
- Codechroma-core/chroma · Vector database · chroma-core/chroma
- DoChunk, embed, store in Chroma, retrieve, and answer with a model. Answer five questions about your docs with citations.
Fine-tuning: LoRA and PEFT65 min
- Watch
- ReadHugging Face · PEFT (Parameter-Efficient Fine-Tuning) · Hugging Face
- DoWrite the decision note for your use case: RAG or fine-tune, and why. List the data a fine-tune would need.
Fine-tune a small model yourself70 min
- CourseHugging Face smol course v2 · fine-tuning on consumer GPUs · Hugging Face
- DoDo the first unit of the smol course. Run a LoRA fine-tune of a small model on 200 examples on a free Colab GPU and compare before and after on 10 prompts.
Evals: your RAG needs a score65 min
- ReadYour AI product needs evals · Hamel Husain
- ReadDefine success criteria and build evaluations · Anthropic
- DoWrite a 20-question golden set with expected answers for your docs. Score the pipeline with exact match plus an LLM judge and put the score in the README.
Hours 70 to 750/15
By the end your LLM app is live on a public URL with its eval score in the README, and you have generated an image from code.
In the Encyclopedia: 🚀 PRACTICE MILESTONE: Build & Deploy an LLM App, Generative AI
Project part 1: the app skeleton70 min
- ReadLangChain · Build LLM Apps Tutorial · LangChain
- CodeHands-On Large Language Models notebooks, Alammar and Grootendorst (free code) · HandsOnLLM/Hands-On-Large-Language-Models
- DoScaffold the app: FastAPI or Streamlit, an ask endpoint over your RAG pipeline, and a config that picks the model provider.
Project part 2: chat UI with streaming65 min
- ReadVercel AI SDK · Build AI-powered web apps · sdk.vercel.ai
- Codestreamlit/streamlit · Build demos fast · streamlit/streamlit
- DoAdd a chat UI that streams tokens and a show-sources panel that lists the retrieved chunks.
Project part 3: deploy75 min
- ReadHugging Face Spaces · Deploy ML Demos for Free · Hugging Face
- WatchEnd-to-End Gen AI Project · LangChain & OpenAI · codebasics
- DoDeploy to a Space or any free host. Add a rate limit and an API key read from the environment. Put the URL and the eval score in the README.
Project part 4: workflows before agents60 min
- ReadAnthropic's Building Effective Agents guide · Anthropic
- WatchHugging Face + LangChain in 5 Minutes · AI Jason
- DoRefactor into one workflow function that chains retrieve, answer, verify. Write three tests that use recorded model responses.
Generative media from code70 min
- WatchHow AI Image Generation Works · Computerphile · Computerphile
- CourseHugging Face · Diffusion Models Course (Free) · Hugging Face
- DoDo unit 1 of the diffusion course. Generate four images from one prompt with the diffusers library on Colab, use one as the app's logo, and note the seed.
Milestone 3 project · about 8 hours
A RAG system over your own documents
Build a retrieval app over your own documents that answers with citations, score it, and put it on a public URL.
- LLM APIs
- Embeddings
- Vector search
- Chunking
- Citations
- Evals
Courses that teach this
- LLM Zoomcamp: Free Course on Building LLM Applications with RAG, Agents & Vector Search
DataTalks.Club, Alexey Grigorev
Build a RAG knowledge base with evaluation, monitoring, UI and a capstone.
- Building with the Claude API
Anthropic · 9 hr
Call the Claude API, then build tool use, RAG, evals, MCP and agents.
- LLM Course
Hugging Face · 6 to 8 hours per chapter
Use, fine-tune and share language models with the Transformers library.
Build it
Collect the documents you want to ask questions about, your own or a public set.
Split them into chunks, create embeddings, and store them in a vector index.
Retrieve the top passages for a question and answer from them, with citations.
Write 20 golden questions with expected answers and score retrieval and answers.
Change one thing, such as chunk size or reranking, and show the score move.
Put it on a public URL with the eval score in the README.
You finish with
A live URL, the repo, the golden set, and the eval score in the README.
Check your work
Every answer cites the passage it came from.
The 20-question golden set has a score for retrieval and for answers.
One change moved the score, and the README shows before and after.
It says it does not know when the documents do not answer.
See how others did it
- RAG Evaluation
huggingface.co
Build a filtered synthetic question set, then score RAG answers with a judge.
- Retrieval Augmented Generation
anthropics/claude-cookbooks
Compare retrieval levels on a labeled eval set using precision, recall, MRR, accuracy.
- Citations
anthropics/claude-cookbooks
Return answers that cite exact passages from plain text and PDF documents.
- Cinnamon/kotaemon
Cinnamon/kotaemon
Working document chat app with PDF citation highlights, Docker images, hosted live demos.
Milestone 4 · Hours 76 to 92 · 3 sections · 15 lessons
Agents and shipping
Add agents, security, containers, logging and monitoring, then write it all up.
Hours 76 to 810/14
By the end your app has an agent loop that plans, calls two tools and stops, plus a written prompt-injection threat model and one test that proves a guard.
In the Encyclopedia: AI Agents & Agentic AI, Prompt Injection & Jailbreaking
What an agent is, from first principles75 min
- ReadAI Agents from First Principles · Cameron R. Wolfe
- WatchReAct Pattern Explained · Sam Witteveen · Sam Witteveen
- DoWrite a 40-line ReAct loop by hand: the model proposes an action, your code runs the tool, repeat for at most five steps, then stop.
Hugging Face Agents Course, unit 170 min
- CourseHugging Face AI Agents Course (free, certified) · Hugging Face
- DoBuild the unit's first agent with smolagents and give it your RAG retriever as a tool.
LangGraph: state, nodes, edges70 min
- ReadLangGraph Documentation · langchain-ai.github.io
- Codelangchain-ai/langgraph · Agent framework · langchain-ai/langgraph
- DoRebuild the agent as a LangGraph graph with a human-approval node before any action that writes.
MCP: tools as a protocol65 min
- CourseHugging Face MCP Course · Model Context Protocol · Hugging Face
- DoDo unit 1 of the MCP course. Expose your retrieve function as an MCP server and call it from an MCP client.
Prompt injection: the threat model70 min
- ReadThe lethal trifecta for AI agents · Simon Willison
- ReadOWASP GenAI Top 10: Prompt Injection · genai.owasp.org
- WatchWhat Is a Prompt Injection Attack? · IBM Technology
- DoWrite the threat note for your agent: which of private data, untrusted content and an exfiltration channel it has. Add one guard (allowlisted tools, no outbound fetch) and a test that an injected document cannot trigger a tool.
Hours 82 to 870/14
By the end the app runs in a container with a CI test, request logs and a cost counter, and you have trained an RL agent on a toy environment to understand RLHF.
In the Encyclopedia: MLOps & Deploying AI Models, Reinforcement Learning Deep Dive
Serving: containers and an API70 min
- WatchDocker for Data Scientists · Abhishek Thakur · Abhishek Thakur
- DoWrite the Dockerfile for the app, build it, run it, and curl the endpoint from outside the container.
Testing and CI for ML code70 min
- CourseMade With ML · End-to-end ML projects · madewithml.com
- DoDo the testing and CI/CD lessons of Made With ML. Add pytest tests for the retriever and for the eval score threshold, and a GitHub Actions workflow that runs them.
Logging, tracing and cost65 min
- ReadMLflow Documentation · mlflow.org
- Read
- DoRead rules 11 to 25 of Rules of ML. Log every request with tokens in and out, latency, model and an estimated cost, and write a daily summary script.
Versioning data and models, and the runbook65 min
- Codeiterative/dvc · Data version control · iterative/dvc
- Read
- DoVersion the eval set and the index with DVC. Write the runbook: how to rebuild the index, rerun the evals, and roll back.
RL in one hour, and what RLHF is75 min
- WatchRLHF Explained · Hugging Face · Hugging Face
- CourseHugging Face Deep RL Course (Free) · Hugging Face
- DoDo unit 1 of the Deep RL course. Train a PPO agent on CartPole with stable-baselines3 for five minutes, plot the reward, and write five lines on how RLHF differs.
Hours 88 to 920/18
By the end your project has a responsible-use note and a public write-up, you have a weekly habit for staying current, and you have reproduced one number from a paper.
In the Encyclopedia: AI Safety, Ethics & Responsible AI, Build Your AI Portfolio & Get Hired, Reading Research Papers & Staying Current
Safety and fairness for your app70 min
- ReadGoogle · Responsible AI Practices · Google
- Code
- WatchWhy AI models hallucinate, and why everyone should know · Harper Carroll AI
- DoWrite the responsible-use note: what the app must not answer, how uncertain answers are flagged, who to contact. Add SHAP explanations to your first shipped model.
Make it a portfolio75 min
- ReadMy 2025 AI Engineer Roadmap List · CyprianTinasheAarons
- Codeeugeneyan/applied-ml · Real-world ML papers by company · eugeneyan/applied-ml
- ReadChip Huyen · ML Interviews Book (Free) · huyenchip.com
- DoWrite a 600-word write-up of the three projects with links, metrics and what broke. Publish it on your site or GitHub profile.
How to read a paper65 min
- ReadHow to Read a Paper by S. Keshav · ccr.sigcomm.org
- WatchHow to Read ML Papers Effectively · Andrew Ng · youtube.com
- DoDo a three-pass read of a paper your app depends on (the RAG paper or the LoRA paper) and write a one-paragraph summary.
A system for staying current70 min
- Codedair-ai/ML-Papers-of-the-Week · dair-ai/ML-Papers-of-the-Week
- ReadThe Batch · Andrew Ng's Weekly AI Newsletter · Andrew Ng
- WatchYannic Kilcher · Paper Explanations (Best Channel) · youtube.com
- DoSet up a weekly 30-minute ritual: one paper summary, one repo tried, one line in a log. Write the first entry.
Reproduce one number70 min
- ReadPapers With Code · Reproduce a Paper · paperswithcode.com
- Codelabml-nn · Annotated PyTorch implementations · labmlai/annotated_deep_learning_paper_implementations
- DoPick one annotated implementation (attention or LoRA), run it, and reproduce one plot or table number from its paper within tolerance. Note the difference and why.
Milestone 4 project · about 10 hours
An agent in production
Turn your app into a monitored agent: tools, human approval, an injection guard, a container and CI.
- Agents
- Tool calling
- Human approval
- Prompt injection
- Docker
- CI
- Observability
Courses that teach this
- AI Agents Course
Hugging Face · about 15 hours
Build agents with tools and frameworks, with a free certificate at the end.
- Introduction to LangGraph - Python
LangChain Academy · 6 hours of video
Build agents with state, memory, human-in-the-loop and deployment in LangGraph.
- 5-Day AI Agents Intensive Course with Google
Google, on Kaggle
Build ADK agents with tools, memory, evaluation, and multi-agent deployment using A2A.
- MLOps Zoomcamp: A Free 9-Week Course on Productionizing ML Services
DataTalks.Club
Build an end-to-end MLOps pipeline: tracking, orchestration, deployment, monitoring.
Build it
Add an agent loop to your app that plans, calls two tools and stops by itself.
Put a human approval step in front of any action that changes something.
Write a prompt-injection threat model and one test that proves your guard works.
Run the app in a container, with a CI test on every commit.
Log each request with its tokens and cost, and write a runbook for when it breaks.
Publish a responsible-use note and a 600-word write-up of your projects.
You finish with
The repo with tests passing in CI, the threat note, the responsible-use note, and a 600-word public write-up of your projects.
Check your work
The agent stops by itself and cannot loop forever.
Any action that changes something waits for approval.
A test proves the injection guard works, and CI runs it.
Every request logs its tokens and cost.
See how others did it
- Human-in-the-loop
openai/openai-agents-python
Mark a tool as needing approval, pause the run, resume after a decision.
- wassim249/fastapi-langgraph-agent-production-ready-template
wassim249/fastapi-langgraph-agent-production-ready-template
Agent service with Docker, CI workflows, Langfuse tracing, structured logs, human-in-the-loop, evals.
- LlamaFirewall
meta-llama/PurpleLlama
Open-source guard: PromptGuard 2 detects injection, with examples and a tests folder.
- Model Usage & Cost Tracking
langfuse.com
Log tokens and cost for every LLM call, inferred from model prices.
Check yourself at the end
Train, evaluate and ship a classic ML model with scikit-learn, with cross-validated metrics you can defend in a code review.
Write and debug PyTorch training code: data loaders, training loops, checkpoints, transfer learning, and a small GPT you built from scratch.
Call hosted and local LLMs from your own code with system prompts, structured JSON output and tool calls, with a golden-set eval that gates every prompt change.
Build and deploy a RAG application over your own documents, then turn it into an agent with tools, a human-approval step and a prompt-injection threat model.
Run an LLM app in production: containerized, tested in CI, logged with tokens and cost per request, with a runbook to rebuild the index and roll back.
Read a research paper in three passes, reproduce one of its numbers, and keep a weekly habit that keeps your stack current.