Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL

16 May 2025

Papers citing "Learning When to Think: Shaping Adaptive Reasoning in R1-Style Models via Multi-Stage RL"

31 / 31 papers shown

Title
Pangu Embedded: An Efficient Dual-system LLM Reasoner with Metacognition Hanting Chen Yasheng Wang Kai Han Dong Li Lin Li ... Hailin Hu Yehui Tang Dacheng Tao Xinghao Chen Yunhe Wang LRM 31 0 0 28 May 2025
Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models Bin Yu Hang Yuan Haotian Li X. Xu Yuliang Wei Bailing Wang Weizhen Qi Kai Chen LRM 63 2 0 06 May 2025
ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning Jingyang Yi Jiazheng Wang Sida Li ReLM OODD LRM 367 4 0 30 Apr 2025
Efficient Reasoning for LLMs through Speculative Chain-of-Thought Jikai Wang Junlin Li Jianye Hou Hao Fei Lijun Wu Min Zhang LLMAG LRM 86 3 0 27 Apr 2025
Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning Jie Cheng Ruixi Qiao Lijun Li Chao Guo Jianmin Wang Gang Xiong Yisheng Lv Fei-Yue Wang LRM 311 3 0 21 Apr 2025
Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Models Yule Liu Jingyi Zheng Zhen Sun Zifan Peng Wenhan Dong Zeyang Sha Shiwen Cui Weiqiang Wang Xinlei He OffRL LRM 88 7 0 18 Apr 2025
Reasoning Models Can Be Effective Without Thinking Wenjie Ma Jingxuan He Charlie Snell Tyler Griggs Sewon Min Matei A. Zaharia ReLM LRM 92 36 1 14 Apr 2025
GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models Jixiao Zhang Chunsheng Zuo LRM 67 14 0 13 Apr 2025
Concise Reasoning via Reinforcement Learning Mehdi Fatemi Banafsheh Rafiee Mingjie Tang Kartik Talamadupula ReLM OffRL LRM 91 11 0 07 Apr 2025
VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks Yu Yue Yufeng Yuan Qiying Yu Xiaochen Zuo Ruofei Zhu ... Ru Zhang Xin Liu Mingxuan Wang Yonghui Wu Lin Yan OffRL LRM 91 20 0 07 Apr 2025
ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning Bairu Hou Yang Zhang Jiabao Ji Yujian Liu Kaizhi Qian Jacob Andreas Shiyu Chang OffRL LRM 85 20 0 02 Apr 2025
SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild Weihao Zeng Yuzhen Huang Qian Liu Wei Liu Keqing He Zejun Ma Junxian He OffRL ReLM LRM 118 85 0 24 Mar 2025
FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models Mingyang Song Mao Zheng Zheng Li Wenjie Yang Xuan Luo Yue Pan Feng Zhang ReLM LRM 118 7 0 21 Mar 2025
Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't Quy-Anh Dang Chris Ngo OffRL LRM 86 13 0 20 Mar 2025
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models Yang Sui Yu-Neng Chuang Guanchu Wang Jiamu Zhang Tianyi Zhang ... Hongyi Liu Andrew Wen Shaochen Zhong Hanjie Chen OffRL ReLM LRM 163 71 0 20 Mar 2025
DAPO: An Open-Source LLM Reinforcement Learning System at Scale Qiying Yu Zheng Zhang Ruofei Zhu Yufeng Yuan Xiaochen Zuo ... Ya Zhang Lin Yan Mu Qiao Yonghui Wu Mingxuan Wang OffRL LRM 125 131 0 18 Mar 2025
Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation Songjun Tu Jiahao Lin Xiangyu Tian Qichao Zhang Linjing Li ... Nan Xu Wei He Xiangyuan Lan D. Jiang Dongbin Zhao LRM 98 4 0 17 Mar 2025
Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning Yuxiao Qu Matthew Y. R. Yang Amrith Rajagopal Setlur Lewis Tunstall E. Beeching Ruslan Salakhutdinov Aviral Kumar OffRL 112 28 0 10 Mar 2025
What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret Yufeng Yuan Yu Yue Ruofei Zhu Tiantian Fan Lin Yan OffRL 90 17 0 03 Mar 2025
Chain of Draft: Thinking Faster by Writing Less Silei Xu Wenhao Xie Lingxiao Zhao Pengcheng He AI4TS LRM 107 62 0 25 Feb 2025
CoT-Valve: Length-Compressible Chain-of-Thought Tuning Xinyin Ma Guangnian Wan Runpeng Yu Gongfan Fang Xinchao Wang LRM 142 38 0 13 Feb 2025
Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search Maohao Shen Guangtao Zeng Zhenting Qi Zhang-Wei Hong Zhenfang Chen Wei Lu G. Wornell Subhro Das David D. Cox Chuang Gan LRM LLMAG 423 13 0 04 Feb 2025
OverThink: Slowdown Attacks on Reasoning LLMs A. Kumar Jaechul Roh A. Naseh Marzena Karpinska Mohit Iyyer Amir Houmansadr Eugene Bagdasarian LRM 101 19 0 04 Feb 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 258 1,503 0 22 Jan 2025
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning Haotian Luo Li Shen Haiying He Yun Wang Shiwei Liu Wei Li Naiqiang Tan Xiaochun Cao Dacheng Tao VLM LRM 128 77 0 22 Jan 2025
Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs Xingyu Chen Jiahao Xu Tian Liang Zhiwei He Jianhui Pang ... Zizhuo Zhang Rui Wang Zhaopeng Tu Haitao Mi Dong Yu LRM ReLM 132 158 0 30 Dec 2024
HybridFlow: A Flexible and Efficient RLHF Framework Guangming Sheng Chi Zhang Zilingfeng Ye Xibin Wu Wang Zhang Ru Zhang Size Zheng Haibin Lin Chuan Wu AI4CE 94 171 0 28 Sep 2024
Self-Reflection in LLM Agents: Effects on Problem-Solving Performance Matthew Renze Erhan Guven LRM LLMAG 70 41 0 05 May 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 91 953 0 05 Feb 2024
ReFT: Reasoning with Reinforced Fine-Tuning Trung Quoc Luong Xinbo Zhang Zhanming Jie Peng Sun Xiaoran Jin Hang Li OffRL LRM ReLM 55 108 0 17 Jan 2024
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 601 9,009 0 28 Jan 2022