Low-Memory Neural Network Training: A Technical Report

Low-Memory Neural Network Training: A Technical Report

24 April 2019

Christopher R. Aberger

Megan Leszczynski

Christopher Ré

Papers citing "Low-Memory Neural Network Training: A Technical Report"

17 / 17 papers shown

Title
GPU Memory Usage Optimization for Backward Propagation in Deep Network Training Ding-Yong Hong Tzu-Hsien Tsai Ning Wang Pangfeng Liu Jan-Jan Wu 39 0 0 18 Feb 2025
Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splitting Chunlin Tian Li Li Kahou Tam Yebo Wu Chengzhong Xu FedML 24 1 0 12 Oct 2024
AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments Cheng Fang Sicong Liu Zimu Zhou Bin Guo Jiaqi Tang Ke Ma Zhiwen Yu TTA 31 1 0 10 Oct 2024
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models Byung-Kwan Lee Chae Won Kim Beomchan Park Yonghyun Ro MLLM LRM 33 17 0 24 May 2024
Breaking On-device Training Memory Wall: A Systematic Survey Shitian Li Chunlin Tian Kahou Tam Ruirui Ma Li Li 21 2 0 17 Jun 2023
Systems for Parallel and Distributed Large-Model Deep Learning Training Kabir Nagrecha GNN VLM MoE 26 7 0 06 Jan 2023
Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharing Guang Li Ren Togo Takahiro Ogawa Miki Haseyama DD 25 40 0 29 Sep 2022
On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling Mustafizur Rahman Subhankar Bose S. Chakrabartty 19 3 0 27 Jun 2022
FuncPipe: A Pipelined Serverless Framework for Fast and Cost-efficient Training of Deep Learning Models Yunzhuo Liu Bo Jiang Tian Guo Zimeng Huang Wen-ping Ma Xinbing Wang Chenghu Zhou 17 9 0 28 Apr 2022
DropIT: Dropping Intermediate Tensors for Memory-Efficient DNN Training Joya Chen Kai Xu Yuhui Wang Yifei Cheng Angela Yao 19 7 0 28 Feb 2022
Enabling On-Device Smartphone GPU based Training: Lessons Learned Anish Das Young D. Kwon Jagmohan Chauhan Cecilia Mascolo 3DH 27 10 0 21 Feb 2022
BitTrain: Sparse Bitmap Compression for Memory-Efficient Training on the Edge Abdelrahman I. Hosny Marina Neseem Sherief Reda MQ 33 4 0 29 Oct 2021
Hydra: A System for Large Multi-Model Deep Learning Kabir Nagrecha Arun Kumar MoE AI4CE 30 5 0 16 Oct 2021
Improving Formality Style Transfer with Context-Aware Rule Injection Zonghai Yao Hong-ye Yu 18 16 0 01 Jun 2021
Enabling Binary Neural Network Training on the Edge Erwei Wang James J. Davis Daniele Moro Piotr Zielinski Jia Jie Lim C. Coelho S. Chatterjee P. Cheung G. Constantinides MQ 20 24 0 08 Feb 2021
Dynamic Tensor Rematerialization Marisa Kirisame Steven Lyubomirsky Altan Haan Jennifer Brennan Mike He Jared Roesch Tianqi Chen Zachary Tatlock 16 93 0 17 Jun 2020
On improving deep learning generalization with adaptive sparse connectivity Shiwei Liu D. Mocanu Mykola Pechenizkiy ODL 12 7 0 27 Jun 2019