70

T^\star: Progressive Block Scaling for MDM Through Trajectory Aware RL

Hanchen Xia
Baoyou Chen
Yutang Ge
Guojiang Zhao
Siyu Zhu
Main:4 Pages
4 Figures
Bibliography:2 Pages
2 Tables
Appendix:2 Pages
Abstract

We present T*, a simple TraceRL-based training curriculum for progressive block-size scaling in masked diffusion language models (MDMs). Starting from an AR-initialized small-block MDM, T* transitions smoothly to larger blocks, enabling higher-parallelism decoding with minimal performance degradation on math reasoning benchmarks. Moreover, further analysis suggests that T* can converge to an alternative decoding schedule that achieves comparable performance.

View on arXiv
Comments on this paper