The Efficiency Misnomer

25 October 2021

Papers citing "The Efficiency Misnomer"

37 / 87 papers shown

Title
Performance-Efficiency Trade-Offs in Adapting Language Models to Text Classification Tasks Laura Aina Nikos Voskarides Roi Blanco 12 0 0 21 Oct 2022
Transcending Scaling Laws with 0.1% Extra Compute Yi Tay Jason W. Wei Hyung Won Chung Vinh Q. Tran David R. So ... Donald Metzler Slav Petrov N. Houlsby Quoc V. Le Mostafa Dehghani LRM 18 67 0 20 Oct 2022
A Multi-dimensional Evaluation of Tokenizer-free Multilingual Pretrained Models Jimin Sun Patrick Fernandes Xinyi Wang Graham Neubig 12 9 0 13 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 11 31 0 13 Oct 2022
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias Ryo Karakida Tomoumi Takase Tomohiro Hayase Kazuki Osawa 11 14 0 06 Oct 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 23 105 0 31 Aug 2022
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling? Yi Tay Mostafa Dehghani Samira Abnar Hyung Won Chung W. Fedus J. Rao Sharan Narang Vinh Q. Tran Dani Yogatama Donald Metzler AI4CE 9 100 0 21 Jul 2022
Confident Adaptive Language Modeling Tal Schuster Adam Fisch Jai Gupta Mostafa Dehghani Dara Bahri Vinh Q. Tran Yi Tay Donald Metzler 41 159 0 14 Jul 2022
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale Reza Yazdani Aminabadi Samyam Rajbhandari Minjia Zhang A. A. Awan Cheng-rong Li ... Elton Zheng Jeff Rasley Shaden Smith Olatunji Ruwase Yuxiong He 16 322 0 30 Jun 2022
RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network Vitaliy Chiley Vithursan Thangarasa Abhay Gupta Anshul Samar Joel Hestness D. DeCoste 21 8 0 28 Jun 2022
MobileOne: An Improved One millisecond Mobile Backbone Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan 17 153 0 08 Jun 2022
RankGen: Improving Text Generation with Large Ranking Models Kalpesh Krishna Yapei Chang John Wieting Mohit Iyyer AIMat 11 68 0 19 May 2022
Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning Haokun Liu Derek Tam Mohammed Muqeeth Jay Mohta Tenghao Huang Mohit Bansal Colin Raffel 33 842 0 11 May 2022
UL2: Unifying Language Learning Paradigms Yi Tay Mostafa Dehghani Vinh Q. Tran Xavier Garcia Jason W. Wei ... Tal Schuster H. Zheng Denny Zhou N. Houlsby Donald Metzler AI4CE 25 292 0 10 May 2022
ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference Kai Hui Honglei Zhuang Tao Chen Zhen Qin Jing Lu ... Ji Ma Jai Gupta Cicero Nogueira dos Santos Yi Tay Donald Metzler 10 16 0 25 Apr 2022
Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey Kento Nozawa Issei Sato AI4TS 8 4 0 18 Apr 2022
ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity Ginger Delmas Rafael Sampaio de Rezende G. Csurka Diane Larlus VLM 13 98 0 15 Mar 2022
HyperPrompt: Prompt-based Task-Conditioning of Transformers Yun He H. Zheng Yi Tay Jai Gupta Yu Du ... Yaguang Li Zhaoji Chen Donald Metzler Heng-Tze Cheng Ed H. Chi LRM VLM 13 83 0 01 Mar 2022
Augmenting Convolutional networks with attention-based aggregation Hugo Touvron Matthieu Cord Alaaeldin El-Nouby Piotr Bojanowski Armand Joulin Gabriel Synnaeve Hervé Jégou ViT 14 42 0 27 Dec 2021
Learned Queries for Efficient Local Attention Moab Arar Ariel Shamir Amit H. Bermano ViT 28 29 0 21 Dec 2021
Can Multilinguality benefit Non-autoregressive Machine Translation? Sweta Agrawal Julia Kreutzer Colin Cherry AI4CE 19 1 0 16 Dec 2021
PolyViT: Co-training Vision Transformers on Images, Videos and Audio Valerii Likhosherstov Anurag Arnab K. Choromanski Mario Lucic Yi Tay Adrian Weller Mostafa Dehghani ViT 31 73 0 25 Nov 2021
LiT: Zero-Shot Transfer with Locked-image text Tuning Xiaohua Zhai Xiao Wang Basil Mustafa Andreas Steiner Daniel Keysers Alexander Kolesnikov Lucas Beyer VLM 14 445 0 15 Nov 2021
SCENIC: A JAX Library for Computer Vision Research and Beyond Mostafa Dehghani A. Gritsenko Anurag Arnab Matthias Minderer Yi Tay 41 67 0 18 Oct 2021
Exploring the Limits of Large Scale Pre-training Samira Abnar Mostafa Dehghani Behnam Neyshabur Hanie Sedghi AI4CE 41 114 0 05 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 198 477 0 01 Oct 2021
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 183 89 0 22 Sep 2021
Scalable and Efficient MoE Training for Multitask Multilingual Models Young Jin Kim A. A. Awan Alexandre Muzio Andres Felipe Cruz Salinas Liyang Lu Amr Hendy Samyam Rajbhandari Yuxiong He Hany Awadalla MoE 88 82 0 22 Sep 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 83 149 0 17 Sep 2021
Efficient Nearest Neighbor Language Models Junxian He Graham Neubig Taylor Berg-Kirkpatrick RALM 188 103 0 09 Sep 2021
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 6 512 0 18 Jun 2021
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better Gaurav Menghani VLM MedIm 14 359 0 16 Jun 2021
Carbon Emissions and Large Neural Network Training David A. Patterson Joseph E. Gonzalez Quoc V. Le Chen Liang Lluís-Miquel Munguía D. Rothchild David R. So Maud Texier J. Dean AI4CE 239 626 0 21 Apr 2021
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 25 937 0 14 Sep 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 4,424 0 23 Jan 2020
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,214 0 17 Apr 2017
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 261 10,106 0 16 Nov 2016