Evaluating AI Evaluation: Perils and Prospects

Evaluating AI Evaluation: Perils and Prospects

12 July 2024

John Burden

Papers citing "Evaluating AI Evaluation: Perils and Prospects"

11 / 11 papers shown

Title
Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina Yuan Gao Dokyun Lee Gordon Burtch Sina Fazelpour LRM 40 7 0 25 Oct 2024
CogBench: a large language model walks into a psychology lab Julian Coda-Forno Marcel Binz Jane X. Wang Eric Schulz ELM ALM LLMAG LM&MA 67 33 0 28 Feb 2024
Visual cognition in multimodal large language models Luca M. Schulze Buschoff Elif Akata Matthias Bethge Eric Schulz LRM 49 12 0 27 Nov 2023
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets Samuel Marks Max Tegmark HILM 91 164 0 10 Oct 2023
Evaluating Superhuman Models with Consistency Checks Lukas Fluri Daniel Paleka Florian Tramèr ELM 31 41 0 16 Jun 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 209 2,232 0 22 Mar 2023
Using cognitive psychology to understand GPT-3 Marcel Binz Eric Schulz ELM LLMAG 236 435 0 21 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 170 3,504 0 10 Jun 2015