Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines

22 July 2024

Sanjiv Kumar

Andrej Risteski

Papers citing "Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines"

4 / 4 papers shown

Title
Do Transformers Parse while Predicting the Masked Word? Haoyu Zhao A. Panigrahi Rong Ge Sanjeev Arora 74 29 0 14 Mar 2023
How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding Yuchen Li Yuan-Fang Li Andrej Risteski 98 61 0 07 Mar 2023
Diffusion-LM Improves Controllable Text Generation Xiang Lisa Li John Thickstun Ishaan Gulrajani Percy Liang Tatsunori B. Hashimoto AI4CE 163 768 0 27 May 2022
Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions Emiel Hoogeboom Didrik Nielsen P. Jaini Patrick Forré Max Welling DiffM 199 392 0 10 Feb 2021