[Paper] Fast-Decoding Diffusion Language Models via Progress-Aware Confidence Schedules

Source: arXiv - 2512.02892v1

Overview

Diffusion‑based large language models (dLLMs) promise higher quality text generation than classic autoregressive models, but their iterative sampling makes them painfully slow for real‑world use. The paper introduces SchED – a training‑free, model‑agnostic early‑exit strategy that stops the diffusion decoding process as soon as the model’s confidence crosses a smooth, progress‑aware threshold. Across multiple dLLM families and ten downstream benchmarks, SchED slashes inference time by up to 4× while preserving virtually all of the original quality.

Key Contributions

• SchED algorithm: a simple, training‑free early‑exit method that aggregates full‑sentence logit margins and applies a dynamic confidence schedule tied to decoding progress.
• Model‑agnostic design: works out‑of‑the‑box on any diffusion language model (tested on Dream and LLaDA, both base and instruction‑tuned).
• Strong empirical gains: 3.8–4.0× speed‑up on instruction‑tuned models with 99.8–100 % of baseline scores; 2.34× speed‑up in aggressive settings with >99 % performance retention.
• Robustness analysis: beats prior confidence‑based early‑exit techniques, especially on long‑form generation where earlier methods collapse.
• Entropy insight: shows that instruction tuning accelerates the decay of predictive entropy, making confidence thresholds easier to hit earlier in the diffusion chain.

Methodology

1. Full‑span logit margin computation – For each diffusion step, SchED collects the difference between the top‑1 and top‑2 token logits across the entire generated sequence (the “margin”).
2. Progress‑aware confidence schedule – Instead of a static cutoff, the algorithm uses a smooth function of decoding progress (e.g., a sigmoid that rises as more diffusion steps are completed). This reflects the intuition that early steps are noisy, while later steps should be more certain.
3. Early‑exit decision – When the aggregated margin exceeds the schedule’s threshold, decoding stops and the current token sequence is emitted. No extra training or fine‑tuning is required; the schedule can be tuned once per model family.
4. Evaluation pipeline – The authors plug SchED into two dLLM families (Dream & LLaDA) and run it on ten diverse tasks: multiple‑choice QA, math problems, long‑form QA, summarization, and translation, covering both base and instruction‑tuned variants.

Results & Findings

• Quality‑penalized metric (QPS, γ=4): SchED consistently outperforms earlier confidence‑based early‑exit methods, which either stall on long texts or cause noticeable quality loss.
• Entropy decay: Instruction‑tuned models show a faster drop in token‑level predictive entropy, meaning they become “confident” earlier—exactly what SchED exploits.
• Stability: Across all ten benchmarks, the speed‑up is stable; there are no catastrophic failures on any particular task.

Practical Implications

• Faster inference for production services – Deploying dLLMs for chatbots, code assistants, or summarization pipelines can now meet latency budgets without sacrificing the quality advantage of diffusion sampling.
• Cost savings – Reducing the number of diffusion steps directly cuts GPU compute time and energy consumption, which is especially valuable for large‑scale API providers.
• Plug‑and‑play integration – Since SchED requires no retraining, existing diffusion models can be retrofitted with a few lines of code (margin aggregation + schedule check).
• Better UX for long‑form generation – Applications like document drafting or multi‑turn reasoning benefit from the robust early‑exit behavior, avoiding the “stall” problem seen in prior methods.
• Guidance for model developers – The entropy analysis suggests that instruction tuning not only improves downstream performance but also makes models more amenable to early‑exit strategies, informing future training pipelines.

Limitations & Future Work

• Schedule tuning – While SchED is training‑free, selecting the optimal confidence schedule still requires a small validation sweep per model family.
• Edge cases – Extremely creative or highly ambiguous prompts may retain high entropy throughout decoding, limiting early‑exit benefits.
• Generality beyond diffusion LLMs – The method is tailored to diffusion‑based generation; applying a similar confidence schedule to other non‑autoregressive paradigms remains open.
• Future directions proposed by the authors include:
  1. Learning adaptive schedules that adjust on‑the‑fly per input.
  2. Extending SchED to multimodal diffusion models (e.g., text‑to‑image).
  3. Combining early‑exit with other acceleration tricks like quantization or distillation for even larger speed gains.

Authors

• Amr Mohamed
• Yang Zhang
• Michalis Vazirgiannis
• Guokan Shang

Paper Information

• arXiv ID: 2512.02892v1
• Categories: cs.CL
• Published: December 2, 2025
• PDF: Download PDF