Evaluating Superhuman Models with Consistency Checks

Evaluating Superhuman Models with Consistency Checks

16 June 2023

Florian Tramèr

Papers citing "Evaluating Superhuman Models with Consistency Checks"

14 / 14 papers shown

Title
The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems Richard Ren Arunim Agarwal Mantas Mazeika Cristina Menghini Robert Vacareanu ... Matias Geralnik Adam Khoja Dean Lee Summer Yue Dan Hendrycks HILM ALM 80 0 0 05 Mar 2025
Should I Trust You? Detecting Deception in Negotiations using Counterfactual RL Wichayaporn Wongkamjan Yanze Wang Feng Gu Denis Peskoff Jonathan K. Kummerfeld Jonathan May Jordan Boyd-Graber 42 0 0 18 Feb 2025
Weak-to-Strong Generalization beyond Accuracy: a Pilot Study in Safety, Toxicity, and Legal Reasoning Ruimeng Ye Yang Xiao Bo Hui ALM ELM OffRL 27 2 0 16 Oct 2024
ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities Ezra Karger Houtan Bastani Chen Yueh-Han Zachary Jacobs Danny Halawi Fred Zhang P. Tetlock 24 6 0 30 Sep 2024
SaGE: Evaluating Moral Consistency in Large Language Models Vamshi Bonagiri Sreeram Vennam Priyanshul Govil Ponnurangam Kumaraguru Manas Gaur ELM 35 0 0 21 Feb 2024
Deception Abilities Emerged in Large Language Models Thilo Hagendorff LLMAG 20 73 0 31 Jul 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 197 2,953 0 22 Mar 2023
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Truthful AI: Developing and governing AI that does not lie Owain Evans Owen Cotton-Barratt Lukas Finnveden Adam Bales Avital Balwit Peter Wills Luca Righetti William Saunders HILM 207 107 0 13 Oct 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 252 273 0 01 Feb 2021
Neural Legal Judgment Prediction in English Ilias Chalkidis Ion Androutsopoulos Nikolaos Aletras AILaw ELM 92 279 0 05 Jun 2019
AI safety via debate G. Irving Paul Christiano Dario Amodei 196 199 0 02 May 2018
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks Guy Katz Clark W. Barrett D. Dill Kyle D. Julian Mykel Kochenderfer AAML 219 1,818 0 03 Feb 2017
Fair prediction with disparate impact: A study of bias in recidivism prediction instruments Alexandra Chouldechova FaML 185 2,079 0 24 Oct 2016