Solution overview

Large language models generate text one token at a time, and every token waits for the model's weights to stream in from GPU memory. Speculative decoding gets more out of each wait: a small drafter model guesses several tokens ahead, and the large model checks all of them in a single pass. Accepted guesses come almost for free, and the output stays exactly what the large model would have written on its own.

In this on-demand lab you explore a real run recorded on an NVIDIA L40S, with Gemma 4 31B running with and without Google's drafter on code, JSON and prose prompts. You place bets on the results, play with interactive simulations built from the recorded numbers, race the two engines side by side, and discover when speculative decoding pays off and when it doesn't.

When you're ready to measure it on your own GPU, continue with the scheduled lab The Token Race: Exploring Speculative Decoding.

Loading