Disentangling the Causes of Plasticity Loss in Neural Networks

Disentangling the Causes of Plasticity Loss in Neural Networks

29 February 2024

Khimya Khetarpal

Papers citing "Disentangling the Causes of Plasticity Loss in Neural Networks"

13 / 13 papers shown

Title
Understanding and Exploiting Plasticity for Non-stationary Network Resource Adaptation Zhiqiang He Zhi Liu CLL 54 0 0 02 May 2025
MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL C. Voelcker Marcel Hussing Eric Eaton Amir-massoud Farahmand Igor Gilitschenski 39 1 0 11 Oct 2024
Neuroplastic Expansion in Deep Reinforcement Learning Jiashun Liu J. Obando-Ceron Aaron C. Courville L. Pan 31 3 0 10 Oct 2024
Don't flatten, tokenize! Unlocking the key to SoftMoE's efficacy in deep RL Ghada Sokar J. Obando-Ceron Aaron C. Courville Hugo Larochelle Pablo Samuel Castro MoE 51 2 0 02 Oct 2024
Can Learned Optimization Make Reinforcement Learning Less Difficult? Alexander David Goldie Chris Xiaoxuan Lu Matthew Jackson Shimon Whiteson Jakob N. Foerster 37 3 0 09 Jul 2024
Simplifying Deep Temporal Difference Learning Matteo Gallici Mattie Fellows Benjamin Ellis B. Pou Ivan Masmitja Jakob Foerster Mario Martin OffRL 46 12 0 05 Jul 2024
Fast TRAC: A Parameter-Free Optimizer for Lifelong Reinforcement Learning Aneesh Muppidi Zhiyu Zhang Heng Yang 29 4 0 26 May 2024
Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous control Michal Nauman M. Ostaszewski Krzysztof Jankowski Piotr Milo's Marek Cygan OffRL 29 16 0 25 May 2024
PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning Hojoon Lee Hanseul Cho Hyunseung Kim Daehoon Gwak Joonkee Kim Jaegul Choo Se-Young Yun Chulhee Yun OffRL 71 25 0 19 Jun 2023
The Primacy Bias in Deep Reinforcement Learning Evgenii Nikishin Max Schwarzer P. DÓro Pierre-Luc Bacon Aaron C. Courville OnRL 85 178 0 16 May 2022
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping James Martens Andy Ballard Guillaume Desjardins G. Swirszcz Valentin Dalibard Jascha Narain Sohl-Dickstein S. Schoenholz 83 43 0 05 Oct 2021
Stable ResNet Soufiane Hayou Eugenio Clerico Bo He George Deligiannidis Arnaud Doucet Judith Rousseau ODL SSeg 46 50 0 24 Oct 2020
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 220 347 0 14 Jun 2018