Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking

18 February 2025

Papers citing "Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking"

42 / 42 papers shown

Title
Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models Makesh Narsimhan Sreedhar Traian Rebedea Christopher Parisien LRM 31 0 0 26 May 2025
Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers Viet-Anh Nguyen Shiqian Zhao Gia Dao Runyi Hu Yi Xie Luu Anh Tuan AAML LRM 48 1 0 22 May 2025
Trust Me, I Can Handle It: Self-Generated Adversarial Scenario Extrapolation for Robust Language Models Md Rafi Ur Rashid Vishnu Asutosh Dasu Ye Wang Gang Tan Shagufta Mehnaz AAML ELM 56 0 0 20 May 2025
Safety in Large Reasoning Models: A Survey Cheng Wang Yang Liu Yangqiu Song Duzhen Zhang Zechao Li ... Shengju Yu Xinfeng Li Junfeng Fang Jiaheng Zhang Bryan Hooi LRM 339 7 0 24 Apr 2025
X-Guard: Multilingual Guard Agent for Content Moderation Bibek Upadhayay Vahid Behzadan Ph.D 57 2 0 11 Apr 2025
GuardReasoner: Towards Reasoning-based LLM Safeguards Yue Liu Hongcheng Gao Shengfang Zhai Jun Xia Tianyi Wu Zhiwei Xue Yuxiao Chen Kenji Kawaguchi Jiaheng Zhang Bryan Hooi AI4TS LRM 153 20 0 30 Jan 2025
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zihao Huang Ziyao Xu Zhiyong Yang Zonghan Yang Zongyu Lin OffRL ALM AI4TS VLM LRM 181 250 0 22 Jan 2025
Rule Based Rewards for Language Model Safety Tong Mu Alec Helyar Johannes Heidecke Joshua Achiam Andrea Vallone Ian Kivlichan Molly Lin Alex Beutel John Schulman Lilian Weng ALM 73 42 0 02 Nov 2024
Does Refusal Training in LLMs Generalize to the Past Tense? Maksym Andriushchenko Nicolas Flammarion 90 31 0 16 Jul 2024
$R^2$ -Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning Mintong Kang Yue Liu LRM 83 13 0 08 Jul 2024
Safety Arithmetic: A Framework for Test-time Safety Alignment of Language Models by Steering Parameters and Activations Rima Hazra Sayan Layek Somnath Banerjee Soujanya Poria KELM LLMSV 58 10 0 17 Jun 2024
Safety Alignment Should Be Made More Than Just a Few Tokens Deep Xiangyu Qi Ashwinee Panda Kaifeng Lyu Xiao Ma Subhrajit Roy Ahmad Beirami Prateek Mittal Peter Henderson 78 99 0 10 Jun 2024
Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge Weikai Lu Huiping Zhuang Jianwei Wang Zhengdong Lu Zelin Chen Huiping Zhuang Cen Chen MU AAML KELM 47 28 0 08 Apr 2024
SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding Zhangchen Xu Fengqing Jiang Luyao Niu Jinyuan Jia Bill Yuchen Lin Radha Poovendran AAML 145 101 0 14 Feb 2024
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal Mantas Mazeika Long Phan Xuwang Yin Andy Zou Zifan Wang ... Nathaniel Li Steven Basart Bo Li David A. Forsyth Dan Hendrycks AAML 55 369 0 06 Feb 2024
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks Andy Zhou Bo Li Haohan Wang AAML 63 81 0 30 Jan 2024
ReFT: Reasoning with Reinforced Fine-Tuning Trung Quoc Luong Xinbo Zhang Zhanming Jie Peng Sun Xiaoran Jin Hang Li OffRL LRM ReLM 51 108 0 17 Jan 2024
Intention Analysis Makes LLMs A Good Jailbreak Defender Yuqi Zhang Liang Ding Lefei Zhang Dacheng Tao LLMSV 45 23 0 12 Jan 2024
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations Hakan Inan Kartikeya Upasani Jianfeng Chi Rashi Rungta Krithika Iyer ... Michael Tontchev Qing Hu Brian Fuller Davide Testuggine Madian Khabsa AI4MH 66 423 0 07 Dec 2023
Safe RLHF: Safe Reinforcement Learning from Human Feedback Josef Dai Xuehai Pan Ruiyang Sun Jiaming Ji Xinbo Xu Mickel Liu Yizhou Wang Yaodong Yang 76 330 0 19 Oct 2023
Jailbreaking Black Box Large Language Models in Twenty Queries Patrick Chao Alexander Robey Yan Sun Hamed Hassani George J. Pappas Eric Wong AAML 72 642 0 12 Oct 2023
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks Alexander Robey Eric Wong Hamed Hassani George J. Pappas AAML 73 243 0 05 Oct 2023
Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions Federico Bianchi Mirac Suzgun Giuseppe Attanasio Paul Röttger Dan Jurafsky Tatsunori Hashimoto James Zou ALM LM&MA LRM 50 196 0 14 Sep 2023
Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon Zhuohan Li Siyuan Zhuang Ying Sheng Lianmin Zheng Cody Hao Yu Joseph E. Gonzalez Haotong Zhang Ion Stoica VLM 118 2,049 0 12 Sep 2023
Certifying LLM Safety against Adversarial Prompting Aounon Kumar Chirag Agarwal Suraj Srinivas Aaron Jiaxun Li Soheil Feizi Himabindu Lakkaraju AAML 47 186 0 06 Sep 2023
Baseline Defenses for Adversarial Attacks Against Aligned Language Models Neel Jain Avi Schwarzschild Yuxin Wen Gowthami Somepalli John Kirchenbauer Ping Yeh-Chiang Micah Goldblum Aniruddha Saha Jonas Geiping Tom Goldstein AAML 94 373 0 01 Sep 2023
Detecting Language Model Attacks with Perplexity Gabriel Alon Michael Kamfonas AAML 67 213 0 27 Aug 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 163 1,376 0 27 Jul 2023
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning Tri Dao LRM 73 1,221 0 17 Jul 2023
Jailbroken: How Does LLM Safety Training Fail? Alexander Wei Nika Haghtalab Jacob Steinhardt 158 928 0 05 Jul 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 242 4,186 0 09 Jun 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 100 1,044 0 31 May 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 288 3,712 0 29 May 2023
Self-Refine: Iterative Refinement with Self-Feedback Aman Madaan Niket Tandon Prakhar Gupta Skyler Hallinan Luyu Gao ... Bodhisattwa Prasad Majumder Katherine Hermann Sean Welleck Amir Yazdanbakhsh Peter Clark ReLM LRM DiffM 106 1,577 0 30 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 653 13,788 0 15 Mar 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 152 1,583 0 15 Dec 2022
Distilling Reasoning Capabilities into Smaller Language Models Kumar Shridhar Alessandro Stolfo Mrinmaya Sachan LRM ReLM 55 165 0 01 Dec 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra Christopher Ré VLM 166 2,131 0 27 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 694 12,525 0 04 Mar 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 191 4,175 0 27 Oct 2021
Measuring Mathematical Problem Solving With the MATH Dataset Dan Hendrycks Collin Burns Saurav Kadavath Akul Arora Steven Basart Eric Tang D. Song Jacob Steinhardt ReLM FaML 122 2,109 0 05 Mar 2021
A Study of BFLOAT16 for Deep Learning Training Dhiraj D. Kalamkar Dheevatsa Mudigere Naveen Mellempudi Dipankar Das K. Banerjee ... Sudarshan Srinivasan Abhisek Kundu M. Smelyanskiy Bharat Kaul Pradeep Dubey MQ 67 340 0 29 May 2019