Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs

25 August 2023

Papers citing "Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs"

50 / 82 papers shown

Title
What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practices Sander Noels Guillaume Bied Maarten Buyl Alexander Rogiers Yousra Fettach Jefrey Lijffijt Tijl De Bie 32 0 0 04 Apr 2025
Improving LLM-as-a-Judge Inference with the Judgment Distribution Victor Wang Michael J.Q. Zhang Eunsol Choi 53 0 0 04 Mar 2025
JailBench: A Comprehensive Chinese Security Assessment Benchmark for Large Language Models Shuyi Liu Simiao Cui Haoran Bu Yuming Shang Xi Zhang ELM 62 0 0 26 Feb 2025
CHBench: A Chinese Dataset for Evaluating Health in Large Language Models Chenlu Guo Nuo Xu Yi-Ju Chang Yuan Wu AI4MH LM&MA 55 1 0 24 Feb 2025
SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention Jiaqi Wu Chen Chen Chunyan Hou Xiaojie Yuan AAML 54 0 0 24 Feb 2025
Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMs Giulio Zizzo Giandomenico Cornacchia Kieran Fraser Muhammad Zaid Hameed Ambrish Rawat Beat Buesser Mark Purcell Pin-Yu Chen P. Sattigeri Kush R. Varshney AAML 43 1 0 24 Feb 2025
Leveraging Reasoning with Guidelines to Elicit and Utilize Knowledge for Enhancing Safety Alignment Haoyu Wang Zeyu Qin Li Shen Xueqian Wang Minhao Cheng Dacheng Tao 89 1 0 06 Feb 2025
STAIR: Improving Safety Alignment with Introspective Reasoning Y. Zhang Siyuan Zhang Yao Huang Zeyu Xia Zhengwei Fang Xiao Yang Ranjie Duan Dong Yan Yinpeng Dong Jun Zhu LRM LLMSV 56 3 0 04 Feb 2025
Evaluating the Propensity of Generative AI for Producing Harmful Disinformation During an Election Cycle Erik J Schlicht 109 0 0 20 Jan 2025
LLM Content Moderation and User Satisfaction: Evidence from Response Refusals in Chatbot Arena Stefan Pasch 29 0 0 08 Jan 2025
Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMs Alexander von Recum Christoph Schnabl Gabor Hollbeck Silas Alberti Philip Blinde Marvin von Hagen 90 2 0 22 Dec 2024
Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari AAML 36 3 0 31 Oct 2024
Effective and Efficient Adversarial Detection for Vision-Language Models via A Single Vector Youcheng Huang Fengbin Zhu Jingkun Tang Pan Zhou Wenqiang Lei Jiancheng Lv Tat-Seng Chua AAML 31 4 0 30 Oct 2024
SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types Yutao Mou Shikun Zhang Wei Ye ELM 38 6 0 29 Oct 2024
Quantifying Risk Propensities of Large Language Models: Ethical Focus and Bias Detection through Role-Play Yifan Zeng Liang Kairong Fangzhou Dong Peijia Zheng 46 0 0 26 Oct 2024
ChineseSafe: A Chinese Benchmark for Evaluating Safety in Large Language Models H. Zhang Hongfu Gao Qiang Hu Guanhua Chen L. Yang Bingyi Jing Hongxin Wei Bing Wang Haifeng Bai Lei Yang AILaw ELM 49 1 0 24 Oct 2024
Large Language Models Still Exhibit Bias in Long Text Wonje Jeung Dongjae Jeon Ashkan Yousefpour Jonghyun Choi ALM 29 2 0 23 Oct 2024
SafetyAnalyst: Interpretable, transparent, and steerable safety moderation for AI behavior Jing-Jing Li Valentina Pyatkin Max Kleiman-Weiner Liwei Jiang Nouha Dziri Anne Collins Jana Schaich Borg Maarten Sap Yejin Choi Sydney Levine 19 1 0 22 Oct 2024
Insights and Current Gaps in Open-Source LLM Vulnerability Scanners: A Comparative Analysis Jonathan Brokman Omer Hofman Oren Rachmil Inderjeet Singh Vikas Pahuja Rathina Sabapathy Aishvariya Priya Amit Giloni Roman Vainshtein Hisashi Kojima 31 1 0 21 Oct 2024
RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style Yantao Liu Zijun Yao Rui Min Yixin Cao Lei Hou Juanzi Li OffRL ALM 18 25 0 21 Oct 2024
JudgeBench: A Benchmark for Evaluating LLM-based Judges Sijun Tan Siyuan Zhuang Kyle Montgomery William Y. Tang Alejandro Cuadron Chenguang Wang Raluca A. Popa Ion Stoica ELM ALM 51 36 0 16 Oct 2024
Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Models Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari AAML 34 1 0 15 Oct 2024
JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework Fan Liu Yue Feng Zhao Xu Lixin Su Xinyu Ma Dawei Yin Hao Liu ELM 27 7 0 11 Oct 2024
Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond Shanshan Han 73 1 0 09 Oct 2024
RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking Yifan Jiang Kriti Aggarwal Tanmay Laud Kashif Munir Jay Pujara Subhabrata Mukherjee AAML 46 10 0 26 Sep 2024
XTRUST: On the Multilingual Trustworthiness of Large Language Models Yahan Li Yi Wang Yi-Ju Chang Yuan Wu HILM LRM 24 0 0 24 Sep 2024
Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI Ambrish Rawat Stefan Schoepf Giulio Zizzo Giandomenico Cornacchia Muhammad Zaid Hameed ... Elizabeth M. Daly Mark Purcell P. Sattigeri Pin-Yu Chen Kush R. Varshney AAML 40 7 0 23 Sep 2024
Programming Refusal with Conditional Activation Steering Bruce W. Lee Inkit Padhi K. Ramamurthy Erik Miehling Pierre L. Dognin Manish Nagireddy Amit Dhurandhar LLMSV 91 13 0 06 Sep 2024
Large Language Models for Automatic Detection of Sensitive Topics Ruoyu Wen Stephanie Elena Crowe Kunal Gupta Xinyue Li Mark Billinghurst S. Hoermann Dwain Allan Alaeddin Nassani Thammathip Piumsomboon AI4MH 21 0 0 02 Sep 2024
FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench Aman Priyanshu Supriti Vijay AAML 23 1 0 28 Aug 2024
Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks Kexin Chen Yi Liu Dongxia Wang Jiaying Chen Wenhai Wang 44 1 0 18 Aug 2024
SAGE-RT: Synthetic Alignment data Generation for Safety Evaluation and Red Teaming Anurakt Kumar Divyanshu Kumar Jatan Loya Nitin Aravind Birur Tanay Baswa Sahil Agarwal P. Harshangi SyDa 37 5 0 14 Aug 2024
PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing Blazej Manczak Eliott Zemour Eric Lin Vaikkunth Mugunthan 26 2 0 23 Jul 2024
How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studies Alina Leidinger Richard Rogers 32 5 0 16 Jul 2024
CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses Jing Yao Xiaoyuan Yi Xing Xie ELM ALM 36 7 0 15 Jul 2024
Safe-Embed: Unveiling the Safety-Critical Knowledge of Sentence Encoders Jinseok Kim Jaewon Jung Sangyeop Kim S. Park Sungzoon Cho 51 0 0 09 Jul 2024
$R^2$ -Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning Mintong Kang Bo-wen Li LRM 32 12 0 08 Jul 2024
Jailbreak Attacks and Defenses Against Large Language Models: A Survey Sibo Yi Yule Liu Zhen Sun Tianshuo Cong Xinlei He Jiaxing Song Ke Xu Qi Li AAML 34 79 0 05 Jul 2024
Single Character Perturbations Break LLM Alignment Leon Lin Hannah Brown Kenji Kawaguchi Michael Shieh AAML 70 2 0 03 Jul 2024
The Art of Saying No: Contextual Noncompliance in Language Models Faeze Brahman Sachin Kumar Vidhisha Balachandran Pradeep Dasigi Valentina Pyatkin ... Jack Hessel Yulia Tsvetkov Noah A. Smith Yejin Choi Hannaneh Hajishirzi 65 20 0 02 Jul 2024
WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs Seungju Han Kavel Rao Allyson Ettinger Liwei Jiang Bill Yuchen Lin Nathan Lambert Yejin Choi Nouha Dziri 37 62 0 26 Jun 2024
CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference Erxin Yu Jing Li Ming Liao Siqi Wang Zuchen Gao Fei Mi Lanqing Hong ELM LRM 23 9 0 25 Jun 2024
Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari 28 0 0 25 Jun 2024
From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking Siyuan Wang Zhuohan Long Zhihao Fan Zhongyu Wei 42 6 0 21 Jun 2024
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Tinghao Xie Xiangyu Qi Yi Zeng Yangsibo Huang Udari Madhushani Sehwag ... Bo Li Kai Li Danqi Chen Peter Henderson Prateek Mittal ALM ELM 50 51 0 20 Jun 2024
Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing Han Jiang Xiaoyuan Yi Zhihua Wei Shu Wang Xing Xie Xing Xie ALM ELM 50 5 0 20 Jun 2024
Knowledge-to-Jailbreak: One Knowledge Point Worth One Attack Shangqing Tu Zhuoran Pan Wenxuan Wang Zhexin Zhang Yuliang Sun Jifan Yu Hongning Wang Lei Hou Juanzi Li ALM 42 1 0 17 Jun 2024
$$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts$ $\texttt{MoE-RBench}$ : Towards Building Reliable Language Models with Sparse Mixture-of-Experts Guanjie Chen Xinyu Zhao Tianlong Chen Yu Cheng MoE 62 5 0 17 Jun 2024
Exploring Safety-Utility Trade-Offs in Personalized Language Models Anvesh Rao Vijjini Somnath Basu Roy Chowdhury Snigdha Chaturvedi 35 6 0 17 Jun 2024
MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases Rithesh Murthy Liangwei Yang Juntao Tan Tulika Awalgaonkar Yilun Zhou ... Zuxin Liu Ming Zhu Huan Wang Caiming Xiong Silvio Savarese 57 5 0 12 Jun 2024