Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward

12 April 2024

Papers citing "Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward"

24 / 24 papers shown

Title
ASPIRER: Bypassing System Prompts With Permutation-based Backdoors in LLMs Lu Yan Siyuan Cheng Xuan Chen Kaiyuan Zhang Guangyu Shen Zhuo Zhang Xiangyu Zhang AAML SILM 13 0 0 05 Oct 2024
LeCov: Multi-level Testing Criteria for Large Language Models Xuan Xie Jiayang Song Yuheng Huang Da Song Fuyuan Zhang Felix Juefei-Xu Lei Ma ELM 21 0 0 20 Aug 2024
Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture Jiayang Song Yuheng Huang Zhehua Zhou Lei Ma 34 6 0 10 Jul 2024
OR-Bench: An Over-Refusal Benchmark for Large Language Models Justin Cui Wei-Lin Chiang Ion Stoica Cho-Jui Hsieh ALM 25 32 0 31 May 2024
Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations Swapnaja Achintalwar Adriana Alvarado Garcia Ateret Anaby-Tavor Ioana Baldini Sara E. Berger ... Aashka Trivedi Kush R. Varshney Dennis L. Wei Shalisha Witherspooon Marcel Zalmanovici 14 10 0 09 Mar 2024
Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness? Kevin Liu Stephen Casper Dylan Hadfield-Menell Jacob Andreas HILM 44 35 0 27 Nov 2023
Summarization is (Almost) Dead Xiao Pu Mingqi Gao Xiaojun Wan HILM 56 38 0 18 Sep 2023
Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation Jiawei Liu Chun Xia Yuyao Wang Lingming Zhang ELM ALM 163 388 0 02 May 2023
The Internal State of an LLM Knows When It's Lying A. Azaria Tom Michael Mitchell HILM 210 297 0 26 Apr 2023
LINe: Out-of-Distribution Detection by Leveraging Important Neurons Yong Hyun Ahn Gyeong-Moon Park Seong Tae Kim OODD 97 31 0 24 Mar 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 197 2,953 0 22 Mar 2023
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models Potsawee Manakul Adian Liusie Mark J. F. Gales HILM LRM 145 386 0 15 Mar 2023
Unifying Evaluation of Machine Learning Safety Monitors Joris Guérin Raul Sena Ferreira Kevin Delmas Jérémie Guiochet 20 6 0 31 Aug 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing Yuheng Huang L. Ma Yuanchun Li ViT AAML 14 10 0 19 Nov 2021
Trustworthy AI: From Principles to Practices Bo-wen Li Peng Qi Bo Liu Shuai Di Jingen Liu Jiquan Pei Jinfeng Yi Bowen Zhou 102 349 0 04 Oct 2021
Challenges in Detoxifying Language Models Johannes Welbl Amelia Glaese J. Uesato Sumanth Dathathri John F. J. Mellor Lisa Anne Hendricks Kirsty Anderson Pushmeet Kohli Ben Coppin Po-Sen Huang LM&MA 236 191 0 15 Sep 2021
Types of Out-of-Distribution Texts and How to Detect Them Udit Arora William Huang He He OODD 204 97 0 14 Sep 2021
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation Tianyu Liu Yizhe Zhang Chris Brockett Yi Mao Zhifang Sui Weizhu Chen W. Dolan HILM 209 140 0 18 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 236 1,508 0 31 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks Guy Katz Clark W. Barrett D. Dill Kyle D. Julian Mykel Kochenderfer AAML 219 1,818 0 03 Feb 2017
Safety Verification of Deep Neural Networks Xiaowei Huang M. Kwiatkowska Sen Wang Min Wu AAML 172 883 0 21 Oct 2016
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 243 9,042 0 06 Jun 2015