SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety

8 April 2024

Paul Röttger

Papers citing "SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety"

28 / 28 papers shown

Title
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
The Structural Safety Generalization Problem Julius Broomfield Tom Gibbs Ethan Kosak-Hine George Ingebretsen Tia Nasir Jason Zhang Reihaneh Iranmanesh Sara Pieri Reihaneh Rabbany Kellin Pelrine AAML 20 0 0 13 Apr 2025
Societal Impacts Research Requires Benchmarks for Creative Composition Tasks Judy Hanwen Shen Carlos Guestrin 25 0 0 09 Apr 2025
Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings Zonghao Ying Guangyi Zheng Yongxin Huang Deyue Zhang Wenxin Zhang Quanchen Zou Aishan Liu X. Liu Dacheng Tao ELM 60 3 0 19 Mar 2025
Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMs Giulio Zizzo Giandomenico Cornacchia Kieran Fraser Muhammad Zaid Hameed Ambrish Rawat Beat Buesser Mark Purcell Pin-Yu Chen P. Sattigeri Kush R. Varshney AAML 37 0 0 24 Feb 2025
A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety Rakeen Rouf Trupti Bavalatti Osama Ahmed Dhaval Potdar Faraz Jawed EGVM 56 1 0 23 Feb 2025
Comprehensive Analysis of Transparency and Accessibility of ChatGPT, DeepSeek, And other SoTA Large Language Models Ranjan Sapkota Shaina Raza Manoj Karkee 29 4 0 21 Feb 2025
Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluation Maria Eriksson Erasmo Purificato Arman Noroozian Joao Vinagre Guillaume Chaslot Emilia Gomez David Fernandez Llorca ELM 117 1 0 10 Feb 2025
Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs Angelina Wang Michelle Phan Daniel E. Ho Sanmi Koyejo 40 2 0 04 Feb 2025
ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain Haochen Zhao Xiangru Tang Ziran Yang Xiao Han Xuanzhi Feng ... Senhao Cheng Di Jin Yilun Zhao Arman Cohan Mark B. Gerstein ELM 68 0 0 23 Nov 2024
A Systematic Review of NeurIPS Dataset Management Practices Yiwei Wu Leah Ajmani Shayne Longpre Hanlin Li 29 0 0 31 Oct 2024
Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari AAML 36 3 0 31 Oct 2024
SoK: Towards Security and Safety of Edge AI Tatjana Wingarz Anne Lauscher Janick Edinger Dominik Kaaser Stefan Schulte Mathias Fischer 22 0 0 07 Oct 2024
Decoding Hate: Exploring Language Models' Reactions to Hate Speech Paloma Piot Javier Parapar 32 0 0 01 Oct 2024
Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI Ambrish Rawat Stefan Schoepf Giulio Zizzo Giandomenico Cornacchia Muhammad Zaid Hameed ... Elizabeth M. Daly Mark Purcell P. Sattigeri Pin-Yu Chen Kush R. Varshney AAML 29 6 0 23 Sep 2024
Compromesso! Italian Many-Shot Jailbreaks Undermine the Safety of Large Language Models Fabio Pernisi Dirk Hovy Paul Röttger 36 0 0 08 Aug 2024
SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models Muxi Diao Rumei Li Shiyang Liu Guogang Liao Jingang Wang Xunliang Cai Weiran Xu AAML 41 1 0 05 Aug 2024
Blockchain for Large Language Model Security and Safety: A Holistic Survey Caleb Geren Amanda Board Gaby G. Dagher Tim Andersen Jun Zhuang 32 3 0 26 Jul 2024
Know Your Limits: A Survey of Abstention in Large Language Models Bingbing Wen Jihan Yao Shangbin Feng Chenjun Xu Yulia Tsvetkov Bill Howe Lucy Lu Wang 44 11 0 25 Jul 2024
How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studies Alina Leidinger Richard Rogers 23 2 0 16 Jul 2024
The Art of Saying No: Contextual Noncompliance in Language Models Faeze Brahman Sachin Kumar Vidhisha Balachandran Pradeep Dasigi Valentina Pyatkin ... Jack Hessel Yulia Tsvetkov Noah A. Smith Yejin Choi Hannaneh Hajishirzi 42 15 0 02 Jul 2024
Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing Han Jiang Xiaoyuan Yi Zhihua Wei Shu Wang Xing Xie Xing Xie ALM ELM 44 5 0 20 Jun 2024
Knowledge-to-Jailbreak: One Knowledge Point Worth One Attack Shangqing Tu Zhuoran Pan Wenxuan Wang Zhexin Zhang Yuliang Sun Jifan Yu Hongning Wang Lei Hou Juanzi Li ALM 33 1 0 17 Jun 2024
Towards Understanding Sycophancy in Language Models Mrinank Sharma Meg Tong Tomasz Korbak D. Duvenaud Amanda Askell ... Oliver Rausch Nicholas Schiefer Da Yan Miranda Zhang Ethan Perez 205 178 0 20 Oct 2023
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 211 327 0 23 Aug 2022
"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset Eric Michael Smith Melissa Hall Melanie Kambadur Eleonora Presani Adina Williams 53 128 0 18 May 2022
BBQ: A Hand-Built Bias Benchmark for Question Answering Alicia Parrish Angelica Chen Nikita Nangia Vishakh Padmakumar Jason Phang Jana Thompson Phu Mon Htut Sam Bowman 200 235 0 15 Oct 2021
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 190 607 0 03 Sep 2019