There Is More to Refusal in Large Language Models than a Single Direction
Faaiz Joad
Majd Hawasly
Sabri Boughorbel
Nadir Durrani
Husrev Taha Sencar
- LLMSV
Main:7 Pages
1 Figures
Bibliography:3 Pages
19 Tables
Appendix:11 Pages
Abstract
Prior work argues that refusal in large language models is mediated by a single activation-space direction, enabling effective steering and ablation. We show that this account is incomplete. Across eleven categories of refusal and non-compliance, including safety, incomplete or unsupported requests, anthropomorphization, and over-refusal, we find that these refusal behaviors correspond to geometrically distinct directions in activation space. Yet despite this diversity, linear steering along any refusal-related direction produces nearly identical refusal to over-refusal trade-offs, acting as a shared one-dimensional control knob. The primary effect of different directions is not whether the model refuses, but how it refuses.
View on arXivComments on this paper
