Mixtures of Experts Unlock Parameter Scaling for Deep RL

Mixtures of Experts Unlock Parameter Scaling for Deep RL

13 February 2024

J. Obando-Ceron

Jesse Farebrother

Jakob N. Foerster

Gintare Karolina Dziugaite

Pablo Samuel Castro

Papers citing "Mixtures of Experts Unlock Parameter Scaling for Deep RL"

13 / 13 papers shown

Title
Handling Delay in Real-Time Reinforcement Learning Ivan Anokhin Rishav Rishav Matthew D Riemer Stephen Chung Irina Rish Samira Ebrahimi Kahou 36 0 0 30 Mar 2025
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 84 1 0 10 Mar 2025
Multi-Task Reinforcement Learning Enables Parameter Scaling Reginald McLean Evangelos Chataroulas Jordan Terry Isaac Woungang Nariman Farsad P. S. Castro LRM 39 0 0 07 Mar 2025
Yes, Q-learning Helps Offline In-Context RL Denis Tarasov Alexander Nikulin Ilya Zisman Albina Klepach Andrei Polubarov Nikita Lyubaykin Alexander Derevyagin Igor Kiselev Vladislav Kurenkov OffRL OnRL 82 0 0 24 Feb 2025
Learning Versatile Optimizers on a Compute Diet A. Moudgil Boris Knyazev Guillaume Lajoie Eugene Belilovsky 63 0 0 22 Jan 2025
Neuroplastic Expansion in Deep Reinforcement Learning Jiashun Liu J. Obando-Ceron Aaron C. Courville L. Pan 31 3 0 10 Oct 2024
Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL Ghada Sokar J. Obando-Ceron Aaron C. Courville Hugo Larochelle Pablo Samuel Castro MoE 54 2 0 02 Oct 2024
The Overcooked Generalisation Challenge Constantin Ruhdorfer Matteo Bortoletto Anna Penzkofer Andreas Bulling 44 3 0 25 Jun 2024
Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control Michal Nauman M. Ostaszewski Krzysztof Jankowski Piotr Milo's Marek Cygan OffRL 29 16 0 25 May 2024
Stop Regressing: Training Value Functions via Classification for Scalable Deep RL Jesse Farebrother Jordi Orbay Q. Vuong Adrien Ali Taïga Yevgen Chebotar ... Sergey Levine Pablo Samuel Castro Aleksandra Faust Aviral Kumar Rishabh Agarwal OffRL 56 55 0 06 Mar 2024
The Primacy Bias in Deep Reinforcement Learning Evgenii Nikishin Max Schwarzer P. DÓro Pierre-Luc Bacon Aaron C. Courville OnRL 85 178 0 16 May 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 147 323 0 18 Feb 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 223 4,424 0 23 Jan 2020