Google releases multi-token prediction drafters for Gemma 4, promising up to 3x faster inference without quality loss

Google is adding Multi-Token Prediction drafters to the Gemma 4 open model family, using speculative decoding to reduce inference latency. The company says the approach can deliver up to a 3x speedup across LiteRT-LM, MLX, Hugging Face Transformers, and vLLM without degrading output quality or reasoning.

The release reflects a broader push to make capable open models practical on developer workstations, mobile devices, and lower-cost cloud infrastructure. By targeting the memory-bandwidth bottleneck in LLM inference, speculative decoding could become a key optimization for making larger models feel more responsive without retraining or shrinking them.

Accelerating Gemma 4: faster inference with multi-token prediction drafters