Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models

4 October 2023

Dahua Lin

Papers citing "Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models"

33 / 33 papers shown

Title
Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety Zihan Guan Mengxuan Hu Ronghang Zhu Sheng Li Anil Vullikanti AAML 31 0 0 11 May 2025
Teaching Models to Understand (but not Generate) High-risk Data Ryan Yixiang Wang Matthew Finlayson Luca Soldaini Swabha Swayamdipta Robin Jia 121 0 0 05 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Jen-tse Huang Joey Tianyi Zhou AAML MU 83 3 0 01 May 2025
Alleviating the Fear of Losing Alignment in LLM Fine-tuning Kang Yang Guanhong Tao X. Chen Jun Xu 36 0 0 13 Apr 2025
Single-pass Detection of Jailbreaking Input in Large Language Models Leyla Naz Candogan Yongtao Wu Elias Abad Rocamora Grigorios G. Chrysos V. Cevher AAML 51 0 0 24 Feb 2025
Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities Zora Che Stephen Casper Robert Kirk Anirudh Satheesh Stewart Slocum ... Zikui Cai Bilal Chughtai Y. Gal Furong Huang Dylan Hadfield-Menell MU AAML ELM 85 3 0 03 Feb 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 82 44 0 20 Jan 2025
Focus On This, Not That! Steering LLMs With Adaptive Feature Specification Tom A. Lamb Adam Davies Alasdair Paren Philip H. S. Torr Francesco Pinto 47 0 0 30 Oct 2024
Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense Rui Min Zeyu Qin Nevin L. Zhang Li Shen Minhao Cheng AAML 36 4 0 13 Oct 2024
Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation Guozhi Liu Weiwei Lin Tiansheng Huang Ruichao Mo Qi Mu Li Shen AAML 63 10 0 13 Oct 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min-Bin Lin 41 8 0 09 Oct 2024
An Adversarial Perspective on Machine Unlearning for AI Safety Jakub Łucki Boyi Wei Yangsibo Huang Peter Henderson F. Tramèr Javier Rando MU AAML 73 32 0 26 Sep 2024
Recent Advances in Attack and Defense Approaches of Large Language Models Jing Cui Yishi Xu Zhewei Huang Shuchang Zhou Jianbin Jiao Junge Zhang PILM AAML 54 1 0 05 Sep 2024
Learning to Ask: When LLM Agents Meet Unclear Instruction Wenxuan Wang Juluan Shi Chaozheng Wang Cheryl Lee Chaozheng Wang Cheryl Lee Youliang Yuan Jen-tse Huang Wenxiang Jiao Michael R. Lyu LLMAG 34 8 0 31 Aug 2024
Acceptable Use Policies for Foundation Models Kevin Klyman 31 14 0 29 Aug 2024
Measuring Human Contribution in AI-Assisted Content Generation Yueqi Xie Tao Qi Jingwei Yi Ryan Whalen Junming Huang Qian Ding Yu Xie Xing Xie Fangzhao Wu Fangzhao Wu 39 1 0 27 Aug 2024
Tamper-Resistant Safeguards for Open-Weight LLMs Rishub Tamirisa Bhrugu Bharathi Long Phan Andy Zhou Alice Gatti ... Andy Zou Dawn Song Bo Li Dan Hendrycks Mantas Mazeika AAML MU 51 41 0 01 Aug 2024
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models Seungone Kim Juyoung Suk Ji Yong Cho Shayne Longpre Chaeeun Kim ... Sean Welleck Graham Neubig Moontae Lee Kyungjae Lee Minjoon Seo ELM ALM LM&MA 97 31 0 09 Jun 2024
Teams of LLM Agents can Exploit Zero-Day Vulnerabilities Richard Fang Antony Kellermann Akul Gupta Qiusi Zhan Richard Fang R. Bindu Daniel Kang LLMAG 40 29 0 02 Jun 2024
Stress-Testing Capability Elicitation With Password-Locked Models Ryan Greenblatt Fabien Roger Dmitrii Krasheninnikov David M. Krueger 38 14 0 29 May 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 46 32 0 27 May 2024
Societal Adaptation to Advanced AI Jamie Bernardi Gabriel Mukobi Hilary Greaves Lennart Heim Markus Anderljung 42 5 0 16 May 2024
When LLMs Meet Cybersecurity: A Systematic Literature Review Jie Zhang Haoyu Bu Hui Wen Yu Chen Lun Li Hongsong Zhu 42 36 0 06 May 2024
Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment Jiong Wang Jiazhao Li Yiquan Li Xiangyu Qi Junjie Hu Yixuan Li P. McDaniel Muhao Chen Bo Li Chaowei Xiao AAML SILM 40 17 0 22 Feb 2024
Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content Federico Bianchi James Zou 32 4 0 21 Feb 2024
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning Zhaorui Yang Tianyu Pang H. Feng Han Wang Wei Chen Minfeng Zhu Qian Liu ALM 32 35 0 21 Feb 2024
LLM Agents can Autonomously Hack Websites Richard Fang R. Bindu Akul Gupta Qiusi Zhan Daniel Kang LLMAG 21 49 0 06 Feb 2024
Navigating the OverKill in Large Language Models Chenyu Shi Xiao Wang Qiming Ge Songyang Gao Xianjun Yang Tao Gui Qi Zhang Xuanjing Huang Xun Zhao Dahua Lin 19 11 0 31 Jan 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 34 78 0 25 Jan 2024
Exploiting Novel GPT-4 APIs Kellin Pelrine Mohammad Taufeeque Michal Zajkac Euan McLean Adam Gleave SILM 20 20 0 21 Dec 2023
LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B Simon Lermen Charlie Rogers-Smith Jeffrey Ladish ALM 20 82 0 31 Oct 2023
A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT Jules White Quchen Fu Sam Hays Michael Sandborn Carlos Olea Henry Gilbert Ashraf Elnashar Jesse Spencer-Smith Douglas C. Schmidt LLMAG 51 1,053 0 21 Feb 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022