NVIDIA’s Sigma Drafts Text in Parallel Blocks and Runs About 3.5 Times Faster Than Standard Decoding, Making Diffusion a Real Contender for Cheaper AI Inference
NVIDIA, Cornell and Georgia Tech researchers detail Nemotron-Labs-Sigma, a 3B/8B continuous diffusion language model that drafts text in parallel blocks, reporting up to 8x faster math generation after distillation and about 3.5x throughput over standard decoding, with clear caveats on scale and coding accuracy.
Read more