Faster Neural Network Training with Approximate Tensor Operations

Faster Neural Network Training with Approximate Tensor Operations

21 May 2018

Menachem Adelman

Papers citing "Faster Neural Network Training with Approximate Tensor Operations"

15 / 15 papers shown

Title
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 83 0 0 18 Feb 2025
TinyProp -- Adaptive Sparse Backpropagation for Efficient TinyML On-device Learning Marcus Rüb Daniel Maier Daniel Mueller-Gritschneder Axel Sikora 18 3 0 17 Aug 2023
Training Transformers with 4-bit Integers Haocheng Xi Changhao Li Jianfei Chen Jun Zhu MQ 25 47 0 21 Jun 2023
Fine-Tuning Language Models with Just Forward Passes Sadhika Malladi Tianyu Gao Eshaan Nichani Alexandru Damian Jason D. Lee Danqi Chen Sanjeev Arora 22 177 0 27 May 2023
Winner-Take-All Column Row Sampling for Memory Efficient Adaptation of Language Model Zirui Liu Guanchu Wang Shaochen Zhong Zhaozhuo Xu Daochen Zha ... Zhimeng Jiang Kaixiong Zhou V. Chaudhary Shuai Xu Xia Hu 30 11 0 24 May 2023
RSC: Accelerating Graph Neural Networks Training via Randomized Sparse Computations Zirui Liu Sheng-Wei Chen Kaixiong Zhou Daochen Zha Xiao Huang Xia Hu 29 14 0 19 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 29 47 0 13 Oct 2022
Memory-Efficient Backpropagation through Large Linear Layers Daniel Bershatsky A. Mikhalev A. Katrutsa Julia Gusak D. Merkulov Ivan V. Oseledets 11 4 0 31 Jan 2022
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization Jiaqi Gu Hanqing Zhu Chenghao Feng Zixuan Jiang Ray T. Chen D. Pan 16 28 0 27 Oct 2021
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks Itay Hubara Brian Chmiel Moshe Island Ron Banner S. Naor Daniel Soudry 44 110 0 16 Feb 2021
Randomized Automatic Differentiation Deniz Oktay N. McGreivy Joshua Aduol Alex Beatson Ryan P. Adams ODL 6 26 0 20 Jul 2020
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights Shail Dave Riyadh Baghdadi Tony Nowatzki Sasikanth Avancha Aviral Shrivastava Baoxin Li 38 81 0 02 Jul 2020
Dithered backprop: A sparse and quantized backpropagation algorithm for more efficient deep neural network training Simon Wiedemann Temesgen Mehari Kevin Kepp Wojciech Samek 6 18 0 09 Apr 2020
Improved Subsampled Randomized Hadamard Transform for Linear SVM Zijian Lei Liang Lan 11 8 0 05 Feb 2020
Accelerated CNN Training Through Gradient Approximation Ziheng Wang Sree Harsha Nelaturu 55 5 0 15 Aug 2019