Routing Networks and the Challenges of Modular and Compositional Computation

29 April 2019

Papers citing "Routing Networks and the Challenges of Modular and Compositional Computation"

22 / 22 papers shown

Title
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts R. Teo Tan M. Nguyen MoE 31 3 0 18 Oct 2024
More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing Sagi Shaier Francisco Pereira K. Wense Lawrence E Hunter Matt Jones MoE 46 0 0 10 Oct 2024
Breaking Neural Network Scaling Laws with Modularity Akhilan Boopathy Sunshine Jiang William Yue Jaedong Hwang Abhiram Iyer Ila Fiete OOD 36 2 0 09 Sep 2024
Revisiting Single-gated Mixtures of Experts Amelie Royer I. Karmanov Andrii Skliar B. Bejnordi Tijmen Blankevoort MoE MoMe 28 6 0 11 Apr 2023
Reusable Slotwise Mechanisms Trang Nguyen Amin Mansouri Kanika Madan Khuong N. Nguyen Kartik Ahuja Dianbo Liu Yoshua Bengio OCL 28 4 0 21 Feb 2023
Learning Label Modular Prompts for Text Classification in the Wild Hailin Chen Amrita Saha Shafiq R. Joty Steven C. H. Hoi OOD VLM 13 5 0 30 Nov 2022
A Short Survey of Systematic Generalization Yuanpeng Li AI4CE 27 1 0 22 Nov 2022
The Neural Race Reduction: Dynamics of Abstraction in Gated Networks Andrew M. Saxe Shagun Sodhani Sam Lewallen AI4CE 28 34 0 21 Jul 2022
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition Jorge Armando Mendez Mendez Eric Eaton KELM CLL 24 27 0 15 Jul 2022
Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions Vibhor Gupta Jyoti Narwariya Pankaj Malhotra L. Vig Gautam M. Shroff AI4TS 19 20 0 14 Mar 2022
Unified Scaling Laws for Routed Language Models Aidan Clark Diego de Las Casas Aurelia Guy A. Mensch Michela Paganini ... Oriol Vinyals Jack W. Rae Erich Elsen Koray Kavukcuoglu Karen Simonyan MoE 27 177 0 02 Feb 2022
All-In-One: Artificial Association Neural Networks Seokjun Kim Jaeeun Jang Hyeoncheol Kim 19 0 0 31 Oct 2021
Dynamic Inference with Neural Interpreters Nasim Rahaman Muhammad Waleed Gondal S. Joshi Peter V. Gehler Yoshua Bengio Francesco Locatello Bernhard Schölkopf 31 31 0 12 Oct 2021
Discrete-Valued Neural Communication Dianbo Liu DianboLiu Alex Lamb Kenji Kawaguchi Anirudh Goyal Chen Sun Michael C. Mozer Yoshua Bengio 23 50 0 06 Jul 2021
Scaling Vision with Sparse Mixture of Experts C. Riquelme J. Puigcerver Basil Mustafa Maxim Neumann Rodolphe Jenatton André Susano Pinto Daniel Keysers N. Houlsby MoE 12 575 0 10 Jun 2021
Neural Production Systems: Learning Rule-Governed Visual Dynamics Anirudh Goyal Aniket Didolkar Nan Rosemary Ke Charles Blundell Philippe Beaudoin N. Heess Michael C. Mozer Yoshua Bengio OCL 44 82 0 02 Mar 2021
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks Róbert Csordás Sjoerd van Steenkiste Jürgen Schmidhuber 21 87 0 05 Oct 2020
Multi-Task Learning with Deep Neural Networks: A Survey M. Crawshaw CVBM 16 608 0 10 Sep 2020
A Study of Compositional Generalization in Neural Models Tim Klinger D. Adjodah Vincent Marois Joshua Joseph Matthew D Riemer Alex Pentland Murray Campbell CoGe NAI 10 12 0 16 Jun 2020
Multi-Task Reinforcement Learning with Soft Modularization Ruihan Yang Huazhe Xu Yi Wu Xiaolong Wang 22 174 0 30 Mar 2020
Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitives Anirudh Goyal Shagun Sodhani Jonathan Binas Xue Bin Peng Sergey Levine Yoshua Bengio 14 47 0 25 Jun 2019
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 264 5,326 0 05 Nov 2016