Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models

4 November 2021

Ahmed Hassan Awadallah

Papers citing "Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models"

50 / 57 papers shown

Title
IM-BERT: Enhancing Robustness of BERT through the Implicit Euler Method Mihyeon Kim Juhyoung Park Youngbin Kim 31 0 0 11 May 2025
Consistency in Language Models: Current Landscape, Challenges, and Future Directions Jekaterina Novikova Carol Anderson Borhane Blili-Hamelin Subhabrata Majumdar HILM 69 0 0 01 May 2025
aiXamine: Simplified LLM Safety and Security Fatih Deniz Dorde Popovic Yazan Boshmaf Euisuh Jeong M. Ahmad Sanjay Chawla Issa M. Khalil ELM 80 0 0 21 Apr 2025
Adversarial Training of Reward Models Alexander Bukharin Haifeng Qian Shengyang Sun Adithya Renduchintala Soumye Singhal Z. Wang Oleksii Kuchaiev Olivier Delalleau T. Zhao AAML 32 0 0 08 Apr 2025
None of the Others: a General Technique to Distinguish Reasoning from Memorization in Multiple-Choice LLM Evaluation Benchmarks Eva Sánchez Salido Julio Gonzalo Guillermo Marco ELM 60 2 0 18 Feb 2025
SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models Jiawen Zhang Kejia Chen Zunlei Feng Jian Lou Mingli Song J. Liu X. J. Yang AAML SILM FedML 43 1 0 02 Feb 2025
On Memorization of Large Language Models in Logical Reasoning Chulin Xie Yangsibo Huang Chiyuan Zhang Da Yu Xinyun Chen Bill Yuchen Lin Bo Li Badih Ghazi Ravi Kumar LRM 53 20 0 30 Oct 2024
Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation Saiful Islam Salim Rubin Yuchan Yang Alexander Cooper Suryashree Ray Saumya Debray Sazzadur Rahaman AAML 44 0 0 12 Oct 2024
Towards Building a Robust Knowledge Intensive Question Answering Model with Large Language Models Xingyun Hong Yan Shao Zhilin Wang Manni Duan Jin Xiongnan 42 0 0 09 Sep 2024
Towards Resilient and Efficient LLMs: A Comparative Study of Efficiency, Performance, and Adversarial Robustness Xiaojing Fan Chunliang Tao AAML 37 28 0 08 Aug 2024
Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation Riccardo Cantini Giada Cosenza A. Orsino Domenico Talia AAML 50 5 0 11 Jul 2024
$$\texttt{MoE-RBench}$: Towards Building Reliable Language Models with Sparse Mixture-of-Experts$ $\texttt{MoE-RBench}$ : Towards Building Reliable Language Models with Sparse Mixture-of-Experts Guanjie Chen Xinyu Zhao Tianlong Chen Yu Cheng MoE 71 5 0 17 Jun 2024
Is On-Device AI Broken and Exploitable? Assessing the Trust and Ethics in Small Language Models Kalyan Nakka Jimmy Dani Nitesh Saxena 43 1 0 08 Jun 2024
Benchmark Data Contamination of Large Language Models: A Survey Cheng Xu Shuhao Guan Derek Greene Mohand-Tahar Kechadi ELM ALM 38 38 0 06 Jun 2024
ACCORD: Closing the Commonsense Measurability Gap François Roewer-Després Jinyue Feng Zining Zhu Frank Rudzicz LRM 48 0 0 04 Jun 2024
Exploiting the Layered Intrinsic Dimensionality of Deep Models for Practical Adversarial Training Enes Altinisik Safa Messaoud H. Sencar Hassan Sajjad Sanjay Chawla AAML 45 0 0 27 May 2024
When LLMs Meet Cybersecurity: A Systematic Literature Review Jie Zhang Haoyu Bu Hui Wen Yu Chen Lun Li Hongsong Zhu 30 36 0 06 May 2024
Advancing the Robustness of Large Language Models through Self-Denoised Smoothing Jiabao Ji Bairu Hou Zhen Zhang Guanhua Zhang Wenqi Fan Qing Li Yang Zhang Gaowen Liu Sijia Liu Shiyu Chang AAML 30 5 0 18 Apr 2024
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward Xuan Xie Jiayang Song Zhehua Zhou Yuheng Huang Da Song Lei Ma OffRL 42 6 0 12 Apr 2024
Contrastive Instruction Tuning Tianyi Yan Fei Wang James Y. Huang Wenxuan Zhou Fan Yin Aram Galstyan Wenpeng Yin Muhao Chen ALM 23 5 0 17 Feb 2024
Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurance Tinghui Ouyang AprilPyone Maungmaung Koichi Konishi Yoshiki Seo Isao Echizen AI4MH 21 5 0 15 Jan 2024
State of What Art? A Call for Multi-Prompt LLM Evaluation Moran Mizrahi Guy Kaplan Daniel Malkin Rotem Dror Dafna Shahaf Gabriel Stanovsky ELM 19 126 0 31 Dec 2023
Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion? Yusuke Sakai Hidetaka Kamigaito Katsuhiko Hayashi Taro Watanabe 24 1 0 15 Nov 2023
Exploring the Effectiveness of GPT Models in Test-Taking: A Case Study of the Driver's License Knowledge Test Saba Rahimi T. Balch Manuela Veloso ELM 21 1 0 22 Aug 2023
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models Paul Röttger Hannah Rose Kirk Bertie Vidgen Giuseppe Attanasio Federico Bianchi Dirk Hovy ALM ELM AILaw 25 125 0 02 Aug 2023
LimeAttack: Local Explainable Method for Textual Hard-Label Adversarial Attack HaiXiang Zhu Zhaoqing Yang Weiwei Shang Yuren Wu AAML FAtt 10 3 0 01 Aug 2023
Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attacks Xinyu Zhang Hanbin Hong Yuan Hong Peng Huang Binghui Wang Zhongjie Ba Kui Ren SILM 29 18 0 31 Jul 2023
Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations Lifan Yuan Yangyi Chen Ganqu Cui Hongcheng Gao Fangyuan Zou Xingyi Cheng Heng Ji Zhiyuan Liu Maosong Sun 32 73 0 07 Jun 2023
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework Yangyi Chen Hongcheng Gao Ganqu Cui Lifan Yuan Dehan Kong ... Longtao Huang H. Xue Zhiyuan Liu Maosong Sun Heng Ji AAML ELM 27 6 0 29 May 2023
Entailment as Robust Self-Learner Jiaxin Ge Hongyin Luo Yoon Kim James R. Glass 39 3 0 26 May 2023
Adversarial Demonstration Attacks on Large Language Models Jiong Wang Zi-yang Liu Keun Hee Park Zhuojun Jiang Zhaoheng Zheng Zhuofeng Wu Muhao Chen Chaowei Xiao SILM 17 51 0 24 May 2023
On Robustness of Finetuned Transformer-based NLP Models Pavan Kalyan Reddy Neerudu S. Oota Mounika Marreddy Venkateswara Rao Kagita Manish Gupta 24 7 0 23 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 32 82 0 19 May 2023
Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility Wen-song Ye Mingfeng Ou Tianyi Li Yipeng Chen Xuetao Ma ... Sai Wu Jie Fu Gang Chen Haobo Wang J. Zhao 42 36 0 15 May 2023
Attacks in Adversarial Machine Learning: A Systematic Survey from the Life-cycle Perspective Baoyuan Wu Zihao Zhu Li Liu Qingshan Liu Zhaofeng He Siwei Lyu AAML 44 21 0 19 Feb 2023
Auditing large language models: a three-layered approach Jakob Mokander Jonas Schuett Hannah Rose Kirk Luciano Floridi AILaw MLAU 42 194 0 16 Feb 2023
Better Diffusion Models Further Improve Adversarial Training Zekai Wang Tianyu Pang Chao Du Min-Bin Lin Weiwei Liu Shuicheng Yan DiffM 18 208 0 09 Feb 2023
Large Language Models Can Be Easily Distracted by Irrelevant Context Freda Shi Xinyun Chen Kanishka Misra Nathan Scales David Dohan Ed H. Chi Nathanael Scharli Denny Zhou ReLM RALM LRM 28 529 0 31 Jan 2023
Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustness Shuaichen Chang J. Wang Mingwen Dong Lin Pan Henghui Zhu ... William Yang Wang Zhiguo Wang Vittorio Castelli Patrick K. L. Ng Bing Xiang OOD 33 34 0 21 Jan 2023
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective Linyi Yang Shuibai Zhang Libo Qin Yafu Li Yidong Wang Hanmeng Liu Jindong Wang Xingxu Xie Yue Zhang ELM 39 79 0 15 Nov 2022
Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutions Gaurav Verma Vishwa Vinay Ryan A. Rossi Srijan Kumar VLM AAML 11 8 0 04 Nov 2022
TCAB: A Large-Scale Text Classification Attack Benchmark Kalyani Asthana Zhouhang Xie Wencong You Adam Noack Jonathan Brophy Sameer Singh Daniel Lowd 24 3 0 21 Oct 2022
ROSE: Robust Selective Fine-tuning for Pre-trained Language Models Lan Jiang Hao Zhou Yankai Lin Peng Li Jie Zhou R. Jiang AAML 27 8 0 18 Oct 2022
Shortcut Learning of Large Language Models in Natural Language Understanding Mengnan Du Fengxiang He Na Zou Dacheng Tao Xia Hu KELM OffRL 28 83 0 25 Aug 2022
ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset Zhihua Jin Xingbo Wang Furui Cheng Chunhui Sun Qun Liu Huamin Qu 32 9 0 17 Aug 2022
Can large language models reason about medical questions? Valentin Liévin C. Hother Andreas Geert Motzfeldt Ole Winther ELM LM&MA AI4MH LRM 19 299 0 17 Jul 2022
Evaluation of Semantic Answer Similarity Metrics Farida Mustafazade Peter F. Ebbinghaus 8 2 0 25 Jun 2022
Adversarial Training for High-Stakes Reliability Daniel M. Ziegler Seraphina Nix Lawrence Chan Tim Bauman Peter Schmidt-Nielsen ... Noa Nabeshima Benjamin Weinstein-Raun D. Haas Buck Shlegeris Nate Thomas AAML 30 59 0 03 May 2022
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models Boxin Wang Wei Ping Chaowei Xiao P. Xu M. Patwary M. Shoeybi Bo-wen Li Anima Anandkumar Bryan Catanzaro 14 64 0 08 Feb 2022
Identifying Adversarial Attacks on Text Classifiers Zhouhang Xie Jonathan Brophy Adam Noack Wencong You Kalyani Asthana Carter Perkins Sabrina Reis Sameer Singh Daniel Lowd AAML 19 9 0 21 Jan 2022