The Efficiency Misnomer

25 October 2021

Papers citing "The Efficiency Misnomer"

50 / 87 papers shown

Title
Efficient LLMs with AMP: Attention Heads and MLP Pruning Leandro Giusti Mugnaini Bruno Yamamoto Lucas Lauton de Alcantara Victor Zacarias Edson Bollis Lucas Pellicer A. H. R. Costa Artur Jordao 37 0 0 29 Apr 2025
APLA: A Simple Adaptation Method for Vision Transformers Moein Sorkhei Emir Konuk Kevin Smith Christos Matsoukas 43 0 0 14 Mar 2025
Position-Aware Depth Decay Decoding ( $D^3$ ): Boosting Large Language Model Inference Efficiency Siqi Fan Xuezhi Fang Xingrun Xing Peng Han Shuo Shang Yequan Wang 46 0 0 11 Mar 2025
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance Chu Myaet Thwal Ye Lin Tun Minh N. H. Nguyen Eui-nam Huh Choong Seon Hong VLM 74 0 0 05 Dec 2024
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 87 3 0 20 Nov 2024
Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives Vincent Hanke Tom Blanchard Franziska Boenisch Iyiola Emmanuel Olatunji Michael Backes Adam Dziedzic PILM 48 3 0 02 Nov 2024
A Simple Model of Inference Scaling Laws Noam Levi LRM 21 0 0 21 Oct 2024
Fortify Your Foundations: Practical Privacy and Security for Foundation Model Deployments In The Cloud Marcin Chrapek Anjo Vahldiek-Oberwagner Marcin Spoczynski Scott Constable Mona Vij Torsten Hoefler 22 1 0 08 Oct 2024
LowFormer: Hardware Efficient Design for Convolutional Transformer Backbones Moritz Nottebaum Matteo Dunnhofer C. Micheloni ViT 21 1 0 05 Sep 2024
Questionable practices in machine learning Gavin Leech Juan J. Vazquez Misha Yagudin Niclas Kupper Laurence Aitchison 42 2 0 17 Jul 2024
On the Limitations of Compute Thresholds as a Governance Strategy Sara Hooker 39 14 0 08 Jul 2024
Block Transformer: Global-to-Local Language Modeling for Fast Inference Namgyu Ho Sangmin Bae Taehyeon Kim Hyunjik Jo Yireun Kim Tal Schuster Adam Fisch James Thorne Se-Young Yun 45 6 0 04 Jun 2024
GAMA-IR: Global Additive Multidimensional Averaging for Fast Image Restoration Youssef Mansour Reinhard Heckel 21 0 0 31 Mar 2024
Not All Layers of LLMs Are Necessary During Inference Siqi Fan Xin Jiang Xiang Li Xuying Meng Peng Han Shuo Shang Aixin Sun Yequan Wang Zhongyuan Wang 38 19 0 04 Mar 2024
Efficient Backpropagation with Variance-Controlled Adaptive Sampling Ziteng Wang Jianfei Chen Jun Zhu BDL 27 2 0 27 Feb 2024
Time-, Memory- and Parameter-Efficient Visual Adaptation Otniel-Bogdan Mercea Alexey Gritsenko Cordelia Schmid Anurag Arnab VLM 35 13 0 05 Feb 2024
SBCFormer: Lightweight Network Capable of Full-size ImageNet Classification at 1 FPS on Single Board Computers Xiangyong Lu Masanori Suganuma Takayuki Okatani 20 9 0 07 Nov 2023
AutoMix: Automatically Mixing Language Models Pranjal Aggarwal Aman Madaan Ankit Anand Srividya Pranavi Potharaju Swaroop Mishra ... Karthik Kappaganthu Yiming Yang Shyam Upadhyay Manaal Faruqui Mausam 40 17 0 19 Oct 2023
Watt For What: Rethinking Deep Learning's Energy-Performance Relationship Shreyank N. Gowda Xinyue Hao Gen Li Laura Sevilla-Lara Shashank Narayana Gowda HAI 8 10 0 10 Oct 2023
Advancing The Rate-Distortion-Computation Frontier For Neural Image Compression David C. Minnen Nick Johnston 8 11 0 26 Sep 2023
Real-Time Semantic Segmentation: A Brief Survey & Comparative Study in Remote Sensing Clifford Broni-Bediako Junshi Xia Naoto Yokoya 23 9 0 12 Sep 2023
Towards quantitative precision for ECG analysis: Leveraging state space models, self-supervision and patient metadata Temesgen Mehari Nils Strodthoff 8 11 0 29 Aug 2023
Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers Tobias Christian Nauen Sebastián M. Palacio Federico Raue Andreas Dengel 29 3 0 18 Aug 2023
Which Tokens to Use? Investigating Token Reduction in Vision Transformers Joakim Bruslund Haurum Sergio Escalera Graham W. Taylor T. Moeslund ViT 27 33 0 09 Aug 2023
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation Hao Peng Qingqing Cao Jesse Dodge Matthew E. Peters Jared Fernandez ... Darrell Plessas Iz Beltagy Evan Pete Walsh Noah A. Smith Hannaneh Hajishirzi 19 7 0 19 Jul 2023
No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models Jean Kaddour Oscar Key Piotr Nawrot Pasquale Minervini Matt J. Kusner 13 41 0 12 Jul 2023
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution Mostafa Dehghani Basil Mustafa Josip Djolonga Jonathan Heek Matthias Minderer ... Avital Oliver Piotr Padlewski A. Gritsenko Mario Luvcić N. Houlsby ViT 13 41 0 12 Jul 2023
How can objects help action recognition? Xingyi Zhou Anurag Arnab Chen Sun Cordelia Schmid 27 14 0 20 Jun 2023
Traversing Between Modes in Function Space for Fast Ensembling Eunggu Yun Hyungi Lee G. Nam Juho Lee UQCV 4 2 0 20 Jun 2023
When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants Anuj Diwan Eunsol Choi David F. Harwath 21 0 0 14 Jun 2023
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks Alexander Tornede Difan Deng Theresa Eimer Joseph Giovanelli Aditya Mohan ... Sarah Segel Daphne Theodorakopoulos Tanja Tornede Henning Wachsmuth Marius Lindauer 17 22 0 13 Jun 2023
Optimizing ViViT Training: Time and Memory Reduction for Action Recognition Shreyank N. Gowda Anurag Arnab Jonathan Huang ViT 8 4 0 07 Jun 2023
Intriguing Properties of Quantization at Scale Arash Ahmadian Saurabh Dash Hongyu Chen Bharat Venkitesh Stephen Gou Phil Blunsom A. Ustun Sara Hooker MQ 27 38 0 30 May 2023
EfficientSpeech: An On-Device Text to Speech Model Rowel Atienza 9 4 0 23 May 2023
Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Design Ibrahim M. Alabdulmohsin Xiaohua Zhai Alexander Kolesnikov Lucas Beyer VLM 14 54 0 22 May 2023
How Does Generative Retrieval Scale to Millions of Passages? Ronak Pradeep Kai Hui Jai Gupta Á. Lelkes Honglei Zhuang Jimmy J. Lin Donald Metzler Vinh Q. Tran RALM 16 62 0 19 May 2023
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention Xinyu Liu Houwen Peng Ningxin Zheng Yuqing Yang Han Hu Yixuan Yuan ViT 15 266 0 11 May 2023
Accuracy is not the only Metric that matters: Estimating the Energy Consumption of Deep Learning Models Johannes Getzner Bertrand Charpentier Stephan Günnemann 11 5 0 03 Apr 2023
Vision Transformers with Mixed-Resolution Tokenization Tomer Ronen Omer Levy A. Golbert ViT 8 21 0 01 Apr 2023
Scaling Expert Language Models with Unsupervised Domain Discovery Suchin Gururangan Margaret Li M. Lewis Weijia Shi Tim Althoff Noah A. Smith Luke Zettlemoyer MoE 8 46 0 24 Mar 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 8 10 0 13 Feb 2023
Scaling Vision Transformers to 22 Billion Parameters Mostafa Dehghani Josip Djolonga Basil Mustafa Piotr Padlewski Jonathan Heek ... Mario Luvcić Xiaohua Zhai Daniel Keysers Jeremiah Harmsen N. Houlsby MLLM 26 562 0 10 Feb 2023
Adaptive Computation with Elastic Input Sequence Fuzhao Xue Valerii Likhosherstov Anurag Arnab N. Houlsby Mostafa Dehghani Yang You 19 18 0 30 Jan 2023
Skip-Attention: Improving Vision Transformers by Paying Less Attention Shashanka Venkataramanan Amir Ghodrati Yuki M. Asano Fatih Porikli A. Habibian ViT 6 25 0 05 Jan 2023
Cramming: Training a Language Model on a Single GPU in One Day Jonas Geiping Tom Goldstein MoE 22 83 0 28 Dec 2022
FlexiViT: One Model for All Patch Sizes Lucas Beyer Pavel Izmailov Alexander Kolesnikov Mathilde Caron Simon Kornblith Xiaohua Zhai Matthias Minderer Michael Tschannen Ibrahim M. Alabdulmohsin Filip Pavetić VLM 20 89 0 15 Dec 2022
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints Aran Komatsuzaki J. Puigcerver James Lee-Thorp Carlos Riquelme Ruiz Basil Mustafa Joshua Ainslie Yi Tay Mostafa Dehghani N. Houlsby MoMe MoE 17 109 0 09 Dec 2022
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 6 605 0 30 Nov 2022
Efficient Speech Quality Assessment using Self-supervised Framewise Embeddings Karl El Hajal Zihan Wu Neil Scheidwasser Gasser Elbanna Milos Cernak 11 9 0 12 Nov 2022
RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization Chengpeng Chen Zichao Guo Haien Zeng Pengfei Xiong Jian Dong 8 37 0 11 Nov 2022