Speculative Decoding Triples LLM Text Generation Speed

Researchers have developed speculative decoding, a technique enabling large language models to generate text three times faster by using smaller models to predict tokens before verification. The method powers faster AI search results and could accelerate enterprise AI deployments, with broader adoption expected as the technique matures.