The Benchmark Lottery

The Benchmark Lottery

14 July 2021

Mostafa Dehghani

Papers citing "The Benchmark Lottery"

15 / 65 papers shown

Title
How not to Lie with a Benchmark: Rearranging NLP Leaderboards Tatiana Shavrina Valentin Malykh ALM ELM 411 10 0 02 Dec 2021
The Efficiency Misnomer Daoyuan Chen Liuyi Yao Dawei Gao Ashish Vaswani Yaliang Li 30 98 0 25 Oct 2021
SCENIC: A JAX Library for Computer Vision Research and Beyond Mostafa Dehghani A. Gritsenko Anurag Arnab Matthias Minderer Yi Tay 41 67 0 18 Oct 2021
The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail Sam Bowman OffRL 17 45 0 15 Oct 2021
Exploring the Limits of Large Scale Pre-training Samira Abnar Mostafa Dehghani Behnam Neyshabur Hanie Sedghi AI4CE 50 114 0 05 Oct 2021
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 188 110 0 22 Sep 2021
YAHPO Gym -- An Efficient Multi-Objective Multi-Fidelity Benchmark for Hyperparameter Optimization Florian Pfisterer Lennart Schneider Julia Moosbauer Martin Binder B. Bischl 25 35 0 08 Sep 2021
Deep Reinforcement Learning at the Edge of the Statistical Precipice Rishabh Agarwal Max Schwarzer P. S. Castro Aaron Courville Marc G. Bellemare OffRL 21 630 0 30 Aug 2021
A Tale Of Two Long Tails Daniel D'souza Zach Nussbaum Chirag Agarwal Sara Hooker 14 22 0 27 Jul 2021
Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features F. Pargent Florian Pfisterer Janek Thomas B. Bischl 14 80 0 01 Apr 2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics Sebastian Gehrmann Tosin P. Adewumi Karmanya Aggarwal Pawan Sasanka Ammanamanchi Aremu Anuoluwapo ... Nishant Subramani Wei-ping Xu Diyi Yang Akhila Yerukola Jiawei Zhou VLM 246 283 0 02 Feb 2021
Uncertainty Estimation with Deep Learning for Rainfall-Runoff Modelling D. Klotz Frederik Kratzert M. Gauch A. Sampson G. Klambauer Sepp Hochreiter G. Nearing BDL UQCV 11 104 0 15 Dec 2020
MDP Playground: An Analysis and Debug Testbed for Reinforcement Learning Raghunandan Rajan Jessica Lizeth Borja Diaz Suresh Guttikonda Fabio Ferreira André Biedenkapp Jan Ole von Hartz Frank Hutter 14 3 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018
Emergence of Locomotion Behaviours in Rich Environments N. Heess TB Dhruva S. Sriram Jay Lemmon J. Merel ... Tom Erez Ziyun Wang S. M. Ali Eslami Martin Riedmiller David Silver 120 927 0 07 Jul 2017