




















Abstract:We present CadLLM, a training-free method to accelerate the inference throughput of diffusion-based LLMs (dLLMs). We first investigate the dynamic nature of token unmasking confidence across blocks and steps. Based on this observation, we present a lightweight adaptive approach that controls the generation block size, step size, and threshold based on the average confidence of unmasked tokens. We further reduce softmax overhead by dynamically leveraging a subset of the vocabulary to regulate sampling breadth. CadLLM is a plug-and-play, model-agnostic method compatible with KV-cache-based dLLMs. Extensive experiments on four popular tasks demonstrate that CadLLM yields up to 1.1-2.28x throughput improvement over the state-of-the-art baseline with competitive accuracy.
| Comments: | 12 pages, 3 figures. Accepted to Findings of ACL 2026 |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.07173 [cs.LG] |
| (or arXiv:2512.07173v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2512.07173 arXiv-issued DOI via DataCite |
From: Jucheng Shen [view email]
[v1]
Mon, 8 Dec 2025 05:15:41 UTC (1,198 KB)
[v2]
Wed, 14 Jan 2026 20:22:57 UTC (1,202 KB)
[v3]
Tue, 27 Jan 2026 22:50:35 UTC (1,201 KB)
[v4]
Thu, 16 Apr 2026 21:26:25 UTC (1,206 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。