Intermediate · About 57 hours · 3 projects · Free

AI Roadmap for QA and testing

In about 57 hours you can hand a release manager an eval set, a red-team report and a regression suite for one AI system, and defend every number in them.

0of 57 hours · 0%

Saved in this browser. No account, no email.

What you'll build

Go at your own pace. Stop at any milestone: each one leaves you with something to show.

  1. 1
    Eval set and rubric for one chatbot or RAG system

    Build the eval set and rubric a colleague could rerun, for one chatbot or RAG system you are allowed to test.

    hour 23
  2. 2
    Red-team report

    Attack the same system by hand, log every result, and report what broke.

    hour 40
  3. 3
    Regression suite with a judge model and a human sample

    Build a suite that runs on every prompt or model change and fails when quality drops.

    hour 57

Skills you'll learn

  • Golden datasets
  • Failure taxonomies
  • Scoring rubrics
  • Inter-rater agreement
  • Test splits
  • Prompt injection
  • Jailbreaks
  • RAG failures
  • Bias probes
  • OWASP LLM Top 10
  • NIST AI RMF
  • Bug reports
  • Regression testing
  • LLM judges
  • Calibration
  • Drift detection
  • CI gates
  • Human sampling

Prerequisites

  • You write test plans and automation today.
  • You can read and edit a Python script.
  • You have one chatbot or RAG system you are allowed to test. A free chat UI will do for the first ten hours.

Program outline

3 milestones · 10 sections · 50 lessons · 3 projects · Every course and source is free

Learn as a feed

Milestone 1 · Hours 1 to 23 · 4 sections · 20 lessons

Know the system under test

Learn how language models fail, then build the test set and rubric.

Hours 1 to 60/16

By the end you can explain in a test plan why one input gives different outputs, and you have 25 black-box test cases with expected behaviors for your chatbot.

In the Encyclopedia: Large Language Models (LLMs), Text Generation & Sampling, Context Window & Long Context

  1. What you are testing: a next-token machine, not a lookup table65 min

  2. How a model is trained and tuned, told by someone who built one75 min

    • Watch
      Andrej Karpathy · Intro to Large Language Models (1hr) · Andrej Karpathy
    • Do
      Turn 5 claims from the talk into test cases: knowledge has a cutoff, the model cannot do arithmetic reliably without a tool, it will invent a citation when pushed, it follows the system prompt over the user most of the time, it can be talked into contradicting itself. Each row: claim, input you will send, expected behavior, what a fail looks like.

  3. Sampling and temperature: why the same prompt never gives the same answer60 min

    • Read
      What is text generation? | IBM · IBM
    • Read
      Hugging Face: Text Generation · Hugging Face
    • Do
      Script: send one golden question to the chatbot 10 times at the default temperature (by hand in the UI if you have no API) and store all 10 answers. Write the pass rule as a rubric row: which key facts must appear for a run to pass. Report the pass rate as k of 10. This k of n format is the unit of every result from now on.

  4. Context window: what the model can see, and what falls off the edge70 min

  5. Known open problems are your first risk list60 min

    • Read
      Open challenges in LLM research · Chip Huyen
    • Read
      State of AI: a 100 trillion token usage study · OpenRouter
    • Do
      From the open-problems list, pick the 5 that apply to your system (hallucination, long context, multilingual, latency, cost are usual) and write one test case per problem with input, expected behavior and the measurement (pass rate, seconds, tokens). You now have 25 cases; tag each with the problem it probes.

Hours 7 to 110/14

By the end you have a failure taxonomy v1, the system prompt rewritten as checkable requirements, 20 golden questions with expected behaviors, and a schema check that rejects malformed answers.

In the Encyclopedia: AI Hallucinations, Prompt Engineering, Structured Outputs

  1. Hallucination: confident, plausible, wrong70 min

    • Watch
      Why Large Language Models Hallucinate · IBM Technology
    • Read
      Why language models hallucinate · OpenAI
    • Read
      Reduce hallucinations · Anthropic
    • Do
      Start taxonomy.md with 10 rubric rows, one per failure type: fabricated fact, fabricated citation or URL, wrong number or date, answer to a different question, unsupported certainty, refusal when it should answer, answer when it should refuse, stale fact, contradiction within one answer, made-up tool result. Each row: name, one-line definition, one example from your system, severity 1 to 3, how you detect it.

  2. The system prompt is the requirements document65 min

    • Watch
      Prompting 101 | Code w/ Claude · Anthropic
    • Read
      Prompt engineering overview · Anthropic
    • Do
      Get the system prompt of your chatbot (or write the one you believe it has). Turn every instruction in it into a numbered requirement and one test case that would fail if the instruction were ignored: tone, allowed topics, refusal rules, format, language, citation rules. Expect 8 to 15 rows.

  3. Prompting techniques are promises you can test70 min

    • Course
      DeepLearning.AI · ChatGPT Prompt Engineering (Free) · DeepLearning.AI · free, certificate extra
    • Do
      Take three techniques from the course (delimiters around user text, few-shot examples, asking for a structured answer) and write one test case per technique that checks the promise holds: user text containing fake delimiters, a few-shot example that the model should not copy verbatim, an answer that must still parse when the question is hostile. Add the 3 cases to golden_set.csv.

  4. Clear, direct, and checkable: writing golden questions65 min

    • Code
      anthropics/prompt-eng-interactive-tutorial · anthropics/prompt-eng-interactive-tutorial
    • Do
      Read chapters 1 to 4 of the tutorial (run the exercises if you have an API key). Then write 10 golden questions for your system with an expected behavior each, mixing the 5 question types you will meet in production: fact lookup, procedure, comparison, out-of-scope request, and ambiguous request that should prompt a clarifying question. Add them to golden_set.csv.

  5. Structured outputs: the first automatic check65 min

    • Read
      Structured outputs · Anthropic
    • Read
      Structured model outputs · OpenAI
    • Do
      Write a JSON schema for the answer record your system should return (answer, citations list, confidence or abstain flag, refusal reason). Then write check_schema.py that validates a file of answers against it, and 5 test cases it must catch: missing field, wrong type, extra field, empty citations on a factual answer, free text instead of JSON.

Hours 12 to 170/14

By the end every taxonomy row has a check type and a metric, and you have a 5 to 7 criterion rubric with scoring anchors that a second person can apply.

In the Encyclopedia: LLM Evaluation & Benchmarks, Evaluation Metrics & Cross-Validation, NLP Metrics (BLEU, ROUGE)

  1. What an eval is, and why your product needs one before it ships65 min

    • Read
      Your AI product needs evals · Hamel Husain
    • Read
      Define success criteria and build evaluations · Anthropic
    • Do
      Add a check_type column to taxonomy.md and fill it for every row with one of: assertion (code can decide), judge (a model scores against a rubric), human (a person must look). Add a how_often column: every run, nightly, weekly sample. Rows with no viable check are bugs in the taxonomy; mark them.

  2. Benchmarks, contamination, and why a leaderboard is not your regression suite75 min

  3. Precision, recall, and the cost of each kind of wrong75 min

  4. Overlap metrics: fast, repeatable, and easy to fool60 min

    • Read
      Task-Specific LLM Evals that Do & Don't Work (Eugene Yan) · eugeneyan.com
    • Read
      BLEU metric · Hugging Face
    • Do
      Add a metric column to golden_set.csv: exact match for extraction and IDs, ROUGE-L for summaries, judge for open answers, assertion for format. Script: run ROUGE-L on 5 summary cases against a reference answer and record the scores; write one case where ROUGE is high and the answer is wrong, to remind the team what the number cannot see.

  5. Rubric v1: criteria with anchors a stranger could apply70 min

    • Read
      LLM evals: frequently asked questions · Hamel Husain
    • Read
      LLM evaluation: a beginner's guide (Evidently AI) · evidentlyai.com
    • Do
      Write rubric.md: 5 to 7 criteria (for example correctness, grounding in sources, completeness, refusal behavior, format, tone) each scored 1 to 5 with a one-line anchor per score and a real example for 1, 3 and 5. Each criterion maps back to taxonomy rows. Give it to a colleague to score 3 answers blind and note where they asked you a question.

Hours 18 to 230/15

By the end your golden set is labeled by two people with agreement measured, split so the prompt author never sees the held-out half, and the eval set ships.

In the Encyclopedia: Data Quality for AI, Data Labeling, Data Leakage

  1. Label errors: your expected answers are data, and data has bugs70 min

    • Course
      Introduction to Data-Centric AI · MIT
    • Do
      Watch lecture 2 (Label Errors) from the course page and skim its lab. Then audit 20 expected answers in golden_set.csv the way the lecture audits test sets: is the label right, is it unambiguous, could two reasonable people disagree. Log every bad label as a bug report row in label_bugs.csv (id, what is wrong, fix, who decided) and fix the set.

  2. Labeling guides and expert review: how golden answers get written well75 min

    • Read
      What Is Data Labeling? | IBM · IBM
    • Listen
      AI's Unsung Hero: Data Labeling and Expert Evals · AI + a16z
    • Do
      Write labeling_guide.md in 10 rules (what counts as correct, how to handle partial answers, when to mark abstain as right). Have a second rater score 20 items with rubric.md. Script: agreement.py that prints percent agreement per criterion. Below 80 percent on a criterion means the anchor text needs work, not the rater.

  3. Leakage: when the test set leaks into the prompt, every score is a lie70 min

  4. Data checks for the corpus and the golden set65 min

  5. Ship the eval set: first full run, first bug reports65 min

    • Read
      Working with evals · OpenAI
    • Code
      Hugging Face Evaluation Guidebook · huggingface/evaluation-guidebook
    • Do
      Run every golden question through the system 5 times, score with check_schema.py and the assertions you have, and record k of 5 per case. File one bug report per case with a pass rate under your threshold (steps, input, expected, actual, taxonomy row, severity). Write README.md for the eval set folder. The first milestone project is done when a colleague can rerun it from the README.

Milestone 1 project · about 8 hours

Eval set and rubric for one chatbot or RAG system

Build the eval set and rubric a colleague could rerun, for one chatbot or RAG system you are allowed to test.

  • Golden datasets
  • Failure taxonomies
  • Scoring rubrics
  • Inter-rater agreement
  • Test splits

Courses that teach this

Build it

  1. Pick one chatbot or RAG system you are allowed to test.

  2. Write 50 golden questions with expected behaviors, and a failure taxonomy with a check type per row.

  3. Write a scoring rubric with anchors a second person can apply.

  4. Have two people label the set and measure their agreement.

  5. Split it into a dev set and a held-out set the prompt author never sees.

  6. Run every case 5 times and file a bug for every case under your threshold.

You finish with

A folder with golden_set.csv, taxonomy.md, rubric.md, labeling_guide.md, check_schema.py, agreement.py, leak_check.py, label_bugs.csv, the first run's k of 5 results, and a README a colleague can rerun from.

Check your work

  • Two raters agree at a measured rate, and you report it.

  • The prompt author never sees the held-out set.

  • Each failure type has a check.

  • A colleague can rerun it from the README.

See how others did it

Milestone 2 · Hours 24 to 40 · 3 sections · 15 lessons

Attack it

Learn how AI systems get attacked, and attack one yourself.

Hours 24 to 280/16

By the end you have run 20 injection and jailbreak cases against your system, logged each as pass or bug, and classified each by its OWASP category.

In the Encyclopedia: Prompt Injection & Jailbreaking, Red Teaming AI Systems

  1. Prompt injection: instructions hiding inside data75 min

  2. Indirect injection: the attack arrives in a document, a page or a tool result75 min

  3. Jailbreaks: role play, many-shot, encoding, split payloads65 min

    • Read
      Many-shot jailbreaking · Anthropic
    • Read
      Adversarial Attacks on LLMs · Lilian Weng
    • Do
      Write 5 jailbreak cases: a role-play frame, a many-shot prefix of fake dialogue, a base64 or leetspeak encoding of a banned request, a request split across two turns, a hypothetical or fiction frame. Record the exact payload so a developer can reproduce it, and the expected refusal text or behavior.

  4. The lethal trifecta: private data, untrusted content, a way out75 min

  5. Run the campaign like a tester, log it like a tester60 min

    • Read
      Challenges in red teaming AI systems · Anthropic
    • Read
      What is Red Teaming? | IBM · IBM
    • Do
      Run all 20 cases, 3 times each. Fill actual and severity in red_team_log.csv and tag each with its OWASP GenAI Top 10 category. Write the first 3 bug reports for the worst findings, each with steps, payload, observed output, expected output, and the guardrail that should have caught it.

Hours 29 to 340/18

By the end you have a RAG failure checklist with cases for retrieval misses, stale and poisoned documents, and a guardrail matrix that shows which check blocks which attack.

In the Encyclopedia: Retrieval-Augmented Generation (RAG), AI Guardrails, Data Poisoning

  1. RAG as a pipeline: a failure point at every stage75 min

  2. What goes wrong in RAG when the truth keeps changing70 min

  3. Grounding and citations: does the source actually say that65 min

    • Read
      Contextual retrieval in AI systems · Anthropic
    • Read
      A Practitioner's Guide to Retrieval Augmented Generation · Cameron R. Wolfe
    • Do
      Add a citation faithfulness row to rubric.md: for each cited chunk, does it support the sentence attached to it (yes, partly, no). Write 5 cases built to fail it: a correct answer with the wrong citation, a citation to a chunk about a different product, a sentence with no citation where one is required, a quote altered by one word, a citation to a document that was removed from the corpus.

  4. Poisoning: who can write to the corpus can write to the answers70 min

  5. Guardrails: which check catches which attack75 min

    • Read
      What Are AI Guardrails? | IBM · IBM
    • Read
      How to implement LLM guardrails · OpenAI
    • Read
      NeMo Guardrails documentation · NVIDIA
    • Do
      Build guardrail_matrix.csv: one row per red-team and RAG case so far, one column per guardrail in your system (input filter, output filter, schema check, tool permission, human approval, rate limit). Fill each cell with blocked, passed, or not applicable from your runs. Every passed cell under a guardrail that should have blocked it becomes a bug report.

Hours 35 to 400/14

By the end the red-team report is written: method, cases, results by category, severity, fixes requested, and what you could not test.

In the Encyclopedia: Adversarial Machine Learning, AI Bias & Fairness, Red Teaming AI Systems

  1. The attack taxonomy: evasion, poisoning, privacy, misuse65 min

  2. ATLAS: adversary techniques written down like a tester's playbook60 min

  3. Bias probes: the same question, a different name60 min

    • Watch
      Algorithmic Bias in AI: What It Is and How to Fix It · IBM Technology
    • Read
      What Is AI Bias? | IBM · IBM
    • Do
      Write 10 paired prompts that differ only in a name, gender, location, age or dialect, run each pair 3 times, and score the answers on tone, refusal, length and quality with rubric.md. Any pair with a consistent difference becomes a bug report with both transcripts attached.

  4. How a lab red teams its own model, and what your campaign is missing75 min

    • Paper
      Anthropic: Red Teaming · arXiv
    • Do
      Compare your case list to the categories in the paper. Write 5 cases for categories you have not covered that apply to your system, and write a one-line out-of-scope note for each category that does not apply, so the report shows the decision, not an omission.

  5. Write the red-team report60 min

    • Read
      What is AI Agent Security? | IBM · IBM
    • Read
      Safety best practices · OpenAI
    • Do
      Write red_team_report.md: one-paragraph summary, method (cases, runs, environment, model and prompt version), results by category with counts, severity scale, top 5 findings with repro steps, fixes requested with the guardrail each belongs to, and what you could not test. The log, the matrix and the bug reports are the appendix. The second milestone project is done when a developer can reproduce finding 1 without asking you.

Milestone 2 project · about 8 hours

Red-team report

Attack the same system by hand, log every result, and report what broke.

  • Prompt injection
  • Jailbreaks
  • RAG failures
  • Bias probes
  • OWASP LLM Top 10
  • NIST AI RMF
  • Bug reports

Courses that teach this

Build it

  1. Write at least 40 cases: direct and indirect prompt injection, jailbreaks and data exfiltration.

  2. Add RAG cases: retrieval misses, citation failures and poisoned documents, plus paired bias probes.

  3. Run each case 3 times and log every result.

  4. Map each case to its OWASP and NIST class and to the guardrail that should have caught it.

  5. Write the report: method, results by category, severity, top 5 findings and what was out of scope.

You finish with

red_team_log.csv, guardrail_matrix.csv, bug reports with reproduction steps, and a 4 to 6 page red_team_report.md with method, results by category, severity, top 5 findings, fixes requested and what was out of scope.

Check your work

  • At least 40 cases, each run 3 times and logged.

  • Each case maps to an OWASP and a NIST class.

  • A developer can reproduce each bug from the report.

  • It says what was out of scope.

See how others did it

Milestone 3 · Hours 41 to 57 · 3 sections · 15 lessons

Regression and release

Learn to judge model output automatically, and build a suite that guards every release.

Hours 41 to 450/16

By the end you have a judge prompt calibrated against a human sample, 8 judge-gaming cases, an agent trajectory check, and 3 vendor claims turned into tests.

In the Encyclopedia: Reward Models and Process Reward Models, Agent Evaluation, ARC-AGI and Reasoning Benchmarks

  1. What a judge model is: a model that scores, and that can be wrong60 min

  2. Reward hacking: how a scorer gets gamed, and how to catch it75 min

    • Read
      Reward Hacking in Reinforcement Learning | Lil'Log · Lil'Log (OpenAI researcher)
    • Read
      RLHF Book, chapter 7: Reward Modeling · rlhfbook.com
    • Do
      Write 8 judge-gaming test cases and run them through judge_prompt_v1: a longer answer with the same content, a confident tone on a wrong answer, the same two answers in swapped order, an answer that praises the question, an answer that restates the rubric words, a correct answer in plain style, an answer that says it is correct, a wrong answer with a fake citation. Record which ones moved the score.

  3. Agent evaluation: many steps, many ways to be right75 min

    • Read
      Demystifying evals for AI agents · Anthropic
    • Read
      Agent Evaluation: A Detailed Guide · Cameron R. Wolfe
    • Watch
      Agent Evaluation walkthrough · Google Cloud Tech
    • Do
      Pick one agent task in your system (or a tool-using flow). Write its eval as test cases on three levels: end state (was the task done), trajectory (which tools were called, in what order, with what arguments), and budget (steps, tokens, seconds, cost). Expected values for each, and which level is a hard fail.

  4. Calibrate the judge against people, then keep the record70 min

    • Course
      Agentic AI, Andrew Ng (free, certificate extra) · DeepLearning.AI · free, certificate extra
    • Do
      Take the evaluation and error-analysis lessons of the course, then write calibrate.py: it takes 30 outputs scored by the judge and by a human (your second rater), prints agreement per criterion and lists every item where they differ by 2 or more. Fix the judge prompt or the anchors for the worst disagreements, rerun, and save calibration_record.md with both agreement numbers. Below 80 percent agreement means the judge cannot gate a release yet.

  5. Benchmarks and their limits: a vendor score is not your release gate65 min

    • Read
      ARC Prize: What is ARC-AGI? · arcprize.org
    • Read
      AI benchmarks & capabilities · Epoch AI
    • Read
      What Are LLM Benchmarks? · IBM
    • Do
      Take 3 claims from your vendor's model page or launch post and turn each into a test case against your own golden set (for example a long-context claim into your buried-fact case, a reasoning claim into your multi-step case, a safety claim into 5 red-team cases). Record the vendor number next to your own.

Hours 46 to 510/16

By the end the regression suite has an offline tier that runs on every prompt change and an online tier that samples production, with drift signals and a trace spec that makes every failure reproducible.

In the Encyclopedia: Offline and Online Evaluation, ML Monitoring & Observability, Model & Data Drift

  1. Offline and online: two tiers, two questions70 min

    • Read
      What is Model Evaluation? | IBM · IBM
    • Watch
      LLM observability in production: tracing and online evals · Evidently AI
    • Do
      Write run_spec.md and the stub of regress.py: offline tier runs on every prompt or model change over the golden set and the red-team log with schema checks, assertions and the judge; online tier samples N production conversations a week for human review. State the trigger, inputs, checks, thresholds and the owner for each tier.

  2. A/B tests and why offline wins do not always show up online75 min

    • Read
      How Not To Run an A/B Test · Evan Miller
    • Course
      Production ML systems (Machine Learning Crash Course) · Google
    • Do
      Write the online release criteria as rubric rows: the one metric that decides, 3 guardrail metrics that must not get worse (refusal rate, latency, cost per answer), the minimum sample before anyone reads the result, and the stop rule. Record the common mistakes from the reading as checks a reviewer must sign.

  3. Drift: the world changes and no error is raised70 min

    • Read
      What Is Model Drift? | IBM · IBM
    • Read
      What is data drift? · Evidently AI
    • Read
      What is concept drift? · Evidently AI
    • Do
      List the drift signals for your system as rubric rows with a threshold each: question topic mix, answer length, refusal rate, retrieval hit rate, judge score trend, share of answers with no citation. Script: drift.py that computes refusal rate and median answer length per week from a log export and prints the weeks that cross a threshold.

  4. Traces: what every request must record so a bug can be reproduced65 min

    • Read
      Data distribution shifts and monitoring · Chip Huyen
    • Read
      What is AI Observability? | IBM · IBM
    • Do
      Write trace_spec.md: the fields every request log must carry (prompt version, model and version, temperature, retrieved document ids and scores, tool calls with arguments, tokens in and out, latency, judge score, user feedback, trace id). Then write 8 test cases that fail when a field is missing or wrong, and run them against a real log line.

  5. The regression runner v165 min

    • Read
      Evidently AI · ML Monitoring · evidentlyai.com
    • Watch
      Observability vs. APM vs. Monitoring · IBM Technology
    • Do
      Build regress.py: loads the golden set and the red-team log, calls the system n times per case, writes every output with the trace fields, applies check_schema.py and the assertions, calls the judge with judge_prompt_v1, and prints pass rate per taxonomy row next to the stored baseline. Exit non-zero when any threshold is crossed.

Hours 52 to 570/16

By the end the suite runs on a real prompt change with a judge and a human sample, a coding agent has drafted and you have reviewed new cases, and the release note states what passed, what failed and what is unknown.

In the Encyclopedia: LLMOps, Human in the Loop, Agentic Coding

  1. LLMOps: version the prompt, evaluate every change, trace what happened60 min

  2. Patterns from production: evals, guardrails, caching, and their tests75 min

  3. Human in the loop: who looks, how often, and what happens when they disagree with the judge75 min

  4. AI for test automation: let the agent draft, keep the review75 min

  5. Run it for real and write the release note65 min

    • Read
      What is AgentOps? | IBM · IBM
    • Read
      Mastering LLM techniques: LLMOps · NVIDIA
    • Do
      Run regress.py against a real prompt change (or a model version bump), review the human sample, and write release_note.md: pass rate per taxonomy row versus baseline, judge and human agreement, red-team cases that regressed, open bug reports by severity, and what the suite cannot see. The third milestone project is done when the note is read by someone who can block the release.

Milestone 3 project · about 10 hours

Regression suite with a judge model and a human sample

Build a suite that runs on every prompt or model change and fails when quality drops.

  • Regression testing
  • LLM judges
  • Calibration
  • Drift detection
  • CI gates
  • Human sampling

Courses that teach this

Build it

  1. Build a runner that executes the golden set and red-team cases on every prompt or model change.

  2. Validate the output schema and fail the run on your thresholds.

  3. Write a judge prompt and calibrate it against 30 human-scored items.

  4. Compare each run to a stored baseline, and track drift signals from production logs.

  5. Add a routine human sample of production conversations.

  6. Run it on one real prompt change and write the release note.

You finish with

regress.py, judge_prompt_v1.md, calibrate.py with calibration_record.md, trace_spec.md, drift.py, prompt_change_checklist.md, human_sample.md, and one release_note.md from a real prompt change.

Check your work

  • It runs by itself on a prompt or model change.

  • The judge agrees with humans at a measured rate.

  • A failed threshold fails the run.

  • One real release note came out of it.

See how others did it

Check yourself at the end

  • You can write a test plan for a non-deterministic system that states what pass means for each failure type, how many runs it takes to say so, and what the threshold is.

  • You own a golden set with expected behaviors, a failure taxonomy with a check type per row, and a scoring rubric that two raters apply with measured agreement.

  • You can run a prompt-injection, jailbreak and RAG-poisoning campaign against a chatbot or RAG system and report each finding with severity, reproduction steps, its OWASP and NIST class, and the guardrail that should have caught it.

  • You can build a regression suite that runs on every prompt or model change, scores outputs with a judge model you calibrated against a human sample, compares to a baseline, and blocks a release on a threshold.

  • You can read a trace, name the drift signals worth alerting on, and tell a vendor benchmark claim from evidence about your own system.

  • You can use a coding agent to draft test cases and automation while keeping the review in human hands and reporting how often the agent was wrong.