Utility is in the Eye of the User: A Critique of NLP Leaderboards

Utility is in the Eye of the User: A Critique of NLP Leaderboards

29 September 2020

Kawin Ethayarajh

Dan Jurafsky

Papers citing "Utility is in the Eye of the User: A Critique of NLP Leaderboards"

12 / 12 papers shown

Title
Richer Countries and Richer Representations Kaitlyn Zhou Kawin Ethayarajh Dan Jurafsky 38 9 0 10 May 2022
Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words Kaitlyn Zhou Kawin Ethayarajh Dallas Card Dan Jurafsky 29 65 0 10 May 2022
Mukayese: Turkish NLP Strikes Back Ali Safaya Emirhan Kurtulucs Arda Goktougan Deniz Yuret 28 22 0 02 Mar 2022
How not to Lie with a Benchmark: Rearranging NLP Leaderboards Tatiana Shavrina Valentin Malykh ALM ELM 416 10 0 02 Dec 2021
Survey of Low-Resource Machine Translation Barry Haddow Rachel Bawden Antonio Valerio Miceli Barone Jindvrich Helcl Alexandra Birch AIMat 29 147 0 01 Sep 2021
Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence Alexander Miserlis Hoyle Pranav Goel Denis Peskov Andrew Hian-Cheong Jordan L. Boyd-Graber Philip Resnik 30 125 0 05 Jul 2021
The Values Encoded in Machine Learning Research Abeba Birhane Pratyusha Kalluri Dallas Card William Agnew Ravit Dotan Michelle Bao 19 273 0 29 Jun 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 246 285 0 02 Feb 2021
Fairness in Machine Learning L. Oneto Silvia Chiappa FaML 235 488 0 31 Dec 2020
How Can We Accelerate Progress Towards Human-like Linguistic Generalization? Tal Linzen 218 188 0 03 May 2020
Certified Robustness to Adversarial Word Substitutions Robin Jia Aditi Raghunathan Kerem Göksel Percy Liang AAML 181 290 0 03 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,950 0 20 Apr 2018