Low-Rank Approximations for Conditional Feedforward Computation in Deep Neural Networks

16 December 2013

Papers citing "Low-Rank Approximations for Conditional Feedforward Computation in Deep Neural Networks"

21 / 21 papers shown

Title
Low-Rank Interconnected Adaptation Across Layers Yibo Zhong Yao Zhou OffRL MoE 48 1 0 13 Jul 2024
Video Relationship Detection Using Mixture of Experts A. Shaabana Zahra Gharaee Paul Fieguth 34 1 0 06 Mar 2024
Compression of Recurrent Neural Networks using Matrix Factorization Lucas Maison Hélion Marie du Mas des Bourboux Thomas Courtat 20 0 0 19 Oct 2023
Memorization Capacity of Neural Networks with Conditional Computation Erdem Koyuncu 38 4 0 20 Mar 2023
Understanding the Robustness of Multi-Exit Models under Common Corruptions Akshay Mehra Skyler Seto Navdeep Jaitly B. Theobald AAML 16 3 0 03 Dec 2022
Spatial Mixture-of-Experts Nikoli Dryden Torsten Hoefler MoE 34 9 0 24 Nov 2022
Hub-Pathway: Transfer Learning from A Hub of Pre-trained Models Yang Shu Zhangjie Cao Ziyang Zhang Jianmin Wang Mingsheng Long 17 4 0 08 Jun 2022
Efficient Large Scale Language Modeling with Mixtures of Experts Mikel Artetxe Shruti Bhosale Naman Goyal Todor Mihaylov Myle Ott ... Jeff Wang Luke Zettlemoyer Mona T. Diab Zornitsa Kozareva Ves Stoyanov MoE 61 188 0 20 Dec 2021
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline Xiangyang Liu Tianxiang Sun Junliang He Jiawen Wu Lingling Wu Xinyu Zhang Hao Jiang Bo Zhao Xuanjing Huang Xipeng Qiu ELM 28 46 0 13 Oct 2021
Zoo-Tuning: Adaptive Transfer from a Zoo of Models Yang Shu Zhi Kou Zhangjie Cao Jianmin Wang Mingsheng Long 29 44 0 29 Jun 2021
Scaling Vision with Sparse Mixture of Experts C. Riquelme J. Puigcerver Basil Mustafa Maxim Neumann Rodolphe Jenatton André Susano Pinto Daniel Keysers N. Houlsby MoE 17 575 0 10 Jun 2021
Pruning and Quantization for Deep Neural Network Acceleration: A Survey Tailin Liang C. Glossner Lei Wang Shaobo Shi Xiaotong Zhang MQ 150 675 0 24 Jan 2021
Bringing AI To Edge: From Deep Learning's Perspective Di Liu Hao Kong Xiangzhong Luo Weichen Liu Ravi Subramaniam 52 116 0 25 Nov 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding Dmitry Lepikhin HyoukJoong Lee Yuanzhong Xu Dehao Chen Orhan Firat Yanping Huang M. Krikun Noam M. Shazeer Z. Chen MoE 31 1,108 0 30 Jun 2020
Attention over Parameters for Dialogue Systems Andrea Madotto Zhaojiang Lin Chien-Sheng Wu Jamin Shin Pascale Fung 30 20 0 07 Jan 2020
Routing Networks and the Challenges of Modular and Compositional Computation Clemens Rosenbaum Ignacio Cases Matthew D Riemer Tim Klinger 40 78 0 29 Apr 2019
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks Victor Campos Brendan Jou Xavier Giró-i-Nieto Jordi Torres Shih-Fu Chang 21 217 0 22 Aug 2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer Noam M. Shazeer Azalia Mirhoseini Krzysztof Maziarz Andy Davis Quoc V. Le Geoffrey E. Hinton J. Dean MoE 55 2,513 0 23 Jan 2017
Variance Reduction in SGD by Distributed Importance Sampling Guillaume Alain Alex Lamb Chinnadhurai Sankar Aaron Courville Yoshua Bengio FedML 13 196 0 20 Nov 2015
Conditional Computation in Neural Networks for faster models Emmanuel Bengio Pierre-Luc Bacon Joelle Pineau Doina Precup AI4CE 23 315 0 19 Nov 2015
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 266 7,638 0 03 Jul 2012