Bypassing the Safety Training of Open-Source LLMs with Priming Attacks

Bypassing the Safety Training of Open-Source LLMs with Priming Attacks

19 December 2023

Gagandeep Singh

Papers citing "Bypassing the Safety Training of Open-Source LLMs with Priming Attacks"

14 / 14 papers shown

Title
One Trigger Token Is Enough: A Defense Strategy for Balancing Safety and Usability in Large Language Models Haoran Gu Handing Wang Yi Mei Mengjie Zhang Yaochu Jin 11 0 0 12 May 2025
Representation Bending for Large Language Model Safety Ashkan Yousefpour Taeheon Kim Ryan S. Kwon Seungbeen Lee Wonje Jeung Seungju Han Alvin Wan Harrison Ngan Youngjae Yu Jonghyun Choi AAML ALM KELM 52 0 0 02 Apr 2025
SafeArena: Evaluating the Safety of Autonomous Web Agents Ada Defne Tur Nicholas Meade Xing Han Lù Alejandra Zambrano Arkil Patel Esin Durmus Spandana Gella Karolina Stañczak Siva Reddy LLMAG ELM 77 2 0 06 Mar 2025
Investigating Generalization of One-shot LLM Steering Vectors Jacob Dunefsky Arman Cohan LLMSV 29 0 0 26 Feb 2025
A generative approach to LLM harmfulness detection with special red flag tokens Sophie Xhonneux David Dobre Mehrnaz Mohfakhami Leo Schwinn Gauthier Gidel 43 1 0 22 Feb 2025
Fast Proxies for LLM Robustness Evaluation Tim Beyer Jan Schuchardt Leo Schwinn Stephan Günnemann AAML 39 0 0 14 Feb 2025
Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignment Jason Vega Junsheng Huang Gaokai Zhang Hangoo Kang Minjia Zhang Gagandeep Singh 29 0 0 05 Nov 2024
HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Router Lingrui Mei Shenghua Liu Yiwei Wang Baolong Bi Ruibin Yuan Xueqi Cheng 33 4 0 03 Oct 2024
Backtracking Improves Generation Safety Yiming Zhang Jianfeng Chi Hailey Nguyen Kartikeya Upasani Daniel M. Bikel Jason Weston Eric Michael Smith SILM 26 5 0 22 Sep 2024
Improving Alignment and Robustness with Circuit Breakers Andy Zou Long Phan Justin Wang Derek Duenas Maxwell Lin Maksym Andriushchenko Rowan Wang Zico Kolter Matt Fredrikson Dan Hendrycks AAML 36 70 0 06 Jun 2024
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks Maksym Andriushchenko Francesco Croce Nicolas Flammarion AAML 79 155 0 02 Apr 2024
Building Guardrails for Large Language Models Yizhen Dong Ronghui Mu Gao Jin Yi Qi Jinwei Hu Xingyu Zhao Jie Meng Wenjie Ruan Xiaowei Huang OffRL 57 23 0 02 Feb 2024
On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused? Hangfan Zhang Zhimeng Guo Huaisheng Zhu Bochuan Cao Lu Lin Jinyuan Jia Jinghui Chen Di Wu 60 23 0 02 Oct 2023
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 264 1,798 0 14 Dec 2020