ALMANACS: A Simulatability Benchmark for Language Model Explainability

ALMANACS: A Simulatability Benchmark for Language Model Explainability

20 December 2023

Stuart J. Russell

Papers citing "ALMANACS: A Simulatability Benchmark for Language Model Explainability"

12 / 12 papers shown

Title
MIB: A Mechanistic Interpretability Benchmark Aaron Mueller Atticus Geiger Sarah Wiegreffe Dana Arad Iván Arcuschin ... Alessandro Stolfo Martin Tutek Amir Zur David Bau Yonatan Belinkov 41 1 0 17 Apr 2025
Hypothesis Testing the Circuit Hypothesis in LLMs Claudia Shi Nicolas Beltran-Velez Achille Nazaret Carolina Zheng Adrià Garriga-Alonso Andrew Jesson Maggie Makar David M. Blei 37 6 0 16 Oct 2024
Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internals Joshua Clymer Caden Juang Severin Field CVBM 32 1 0 08 May 2024
Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought James Chua Edward Rees Hunar Batra Samuel R. Bowman Julian Michael Ethan Perez Miles Turpin LRM 39 13 0 08 Mar 2024
RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations Jing-ling Huang Zhengxuan Wu Christopher Potts Mor Geva Atticus Geiger 55 25 0 27 Feb 2024
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 206 498 0 03 May 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 210 491 0 01 Nov 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 4,048 0 24 May 2022
Measuring Association Between Labels and Free-Text Rationales Sarah Wiegreffe Ana Marasović Noah A. Smith 274 170 0 24 Oct 2020
e-SNLI: Natural Language Inference with Natural Language Explanations Oana-Maria Camburu Tim Rocktaschel Thomas Lukasiewicz Phil Blunsom LRM 252 620 0 04 Dec 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018
Towards A Rigorous Science of Interpretable Machine Learning Finale Doshi-Velez Been Kim XAI FaML 225 3,672 0 28 Feb 2017