← 返回时间线

paper

What Makes a Good Unmasking Policy? A Controlled Study of Learned Unmasking in Diffusion Language Models

arXiv ↗
ID
2610.31576
分类
—
首次捕获
2026-10-23
状态
unread
作者
Jinya Sakurai, Xun Xu, Jaehong Yoon
信号
🔥 12

信号历史

  • 2026-10-23HF Daily Papers · 🔥3
  • 2026-10-24HF Daily Papers · 🔥5
  • 2026-10-25HF Daily Papers · 🔥11
  • 2026-10-26HF Daily Papers · 🔥12
暂无信号数据

摘要

Masked diffusion language models generate tokens in parallel, but their efficiency depends on which predictions are committed. Learned unmasking policies improve the speed--accuracy trade-off, yet differ in supervision, inputs, and capacity, obscuring which choices matter. We view supervised policies as distilling a decoding procedure through a teacher and examine independently what the teacher labels as committable, on which states the policy imitates it, what the policy observes, and how much the number of function evaluations (NFE) must decrease to offset policy overhead. We find that labeling predictions matching the final output is overly permissive at low NFE, and hidden states are weak alone but can complement confidence. About a 10% NFE reduction suffices to offset policy overhead for policies up to 7.5M parameters across tested settings. We therefore train a policy on trajectory-contraction targets, which accept only commitments staying on a reference trajectory, using confidence and hidden states. On LLaDA-8B-Instruct, the resulting 856K-parameter policy achieves the best accuracy--NFE frontier and highest throughput on GSM8K, is competitive at moderate targets on MATH-500 and HumanEval, and largely transfers to Dream-7B-Instruct.

我的笔记

还没有笔记。

在 GitHub 上写笔记 ↗(新建 content/notes/2610.31576.md,PR 合并后本页自动更新)