ALERT: A Comprehensive Benchmark for Assessing Large Language Models'
Safety through Red Teaming

ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming

6 April 2024

Simone Tedeschi

Felix Friedrich

Kristian Kersting

Roberto Navigli

Papers citing "ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming"

14 / 14 papers shown

Title
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models Bang An Shiyue Zhang Mark Dredze 54 0 0 25 Apr 2025
Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond Shanshan Han 64 1 0 09 Oct 2024
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Wenxuan Zhang Philip H. S. Torr Mohamed Elhoseiny Adel Bibi 38 9 0 27 Aug 2024
Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation Riccardo Cantini Giada Cosenza A. Orsino Domenico Talia AAML 37 5 0 11 Jul 2024
Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents Avital Shafran R. Schuster Vitaly Shmatikov 34 27 0 09 Jun 2024
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety Paul Röttger Fabio Pernisi Bertie Vidgen Dirk Hovy ELM KELM 51 30 0 08 Apr 2024
A Safe Harbor for AI Evaluation and Red Teaming Shayne Longpre Sayash Kapoor Kevin Klyman Ashwin Ramaswami Rishi Bommasani ... Daniel Kang Sandy Pentland Arvind Narayanan Percy Liang Peter Henderson 49 10 0 07 Mar 2024
OLMo: Accelerating the Science of Language Models Dirk Groeneveld Iz Beltagy Pete Walsh Akshita Bhagia Rodney Michael Kinney ... Jesse Dodge Kyle Lo Luca Soldaini Noah A. Smith Hanna Hajishirzi OSLM 124 349 0 01 Feb 2024
The Falcon Series of Open Language Models Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra-Aimée Cojocaru ... Quentin Malartic Daniele Mazzotta Badreddine Noune B. Pannier Guilherme Penedo AI4TS ALM 104 389 0 28 Nov 2023
Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs Shashank Gupta Vaishnavi Shrivastava A. Deshpande A. Kalyan Peter Clark Ashish Sabharwal Tushar Khot 108 49 0 08 Nov 2023
GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts Jiahao Yu Xingwei Lin Zheng Yu Xinyu Xing SILM 110 292 0 19 Sep 2023
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech Mai Elsherief Caleb Ziems D. Muchlinski Vaishnavi Anupindi Jordyn Seybolt M. D. Choudhury Diyi Yang 85 233 0 11 Sep 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 264 1,798 0 14 Dec 2020