ResearchTrend.AI
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats
v1v2v3v4 (latest)

Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats

Brian Chmiel
Ron Banner
Elad Hoffer
Hilla Ben Yaacov
Daniel Soudry
    MQ
ArXiv (abs)PDFHTML

Papers citing "Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats"

15 / 15 papers shown
Title
FP4 All the Way: Fully Quantized Training of LLMs
FP4 All the Way: Fully Quantized Training of LLMs
Brian Chmiel
Maxim Fishman
Ron Banner
Daniel Soudry
MQ
145
1
0
25 May 2025
Quartet: Native FP4 Training Can Be Optimal for Large Language Models
Quartet: Native FP4 Training Can Be Optimal for Large Language Models
Roberto L. Castro
Andrei Panferov
Soroush Tabesh
Oliver Sieberling
Jiale Chen
Mahdi Nikdan
Saleh Ashkboos
Dan Alistarh
MQ
155
0
0
20 May 2025
HOT: Hadamard-based Optimized Training
HOT: Hadamard-based Optimized Training
Seonggon Kim
Juncheol Shin
Seung-taek Woo
Eunhyeok Park
144
0
0
27 Mar 2025
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization
Minsu Kim
Seongmin Hong
RyeoWook Ko
S. Choi
Hunjong Lee
Junsoo Kim
Joo-Young Kim
Jongse Park
157
0
0
24 Mar 2025
EXAQ: Exponent Aware Quantization For LLMs Acceleration
EXAQ: Exponent Aware Quantization For LLMs Acceleration
Moran Shkolnik
Maxim Fishman
Brian Chmiel
Hilla Ben-Yaacov
Ron Banner
Kfir Y. Levy
MQ
73
0
0
04 Oct 2024
HLQ: Fast and Efficient Backpropagation via Hadamard Low-rank Quantization
HLQ: Fast and Efficient Backpropagation via Hadamard Low-rank Quantization
Seonggon Kim
Eunhyeok Park
106
2
0
21 Jun 2024
LoQT: Low Rank Adapters for Quantized Training
LoQT: Low Rank Adapters for Quantized Training
Sebastian Loeschcke
M. Toftrup
M. Kastoryano
Serge Belongie
Vésteinn Snæbjarnarson
MQ
100
5
0
26 May 2024
BOLD: Boolean Logic Deep Learning
BOLD: Boolean Logic Deep Learning
Van Minh Nguyen
Cristian Ocampo
Aymen Askri
Louis Leconte
Ba-Hien Tran
AI4CE
144
1
0
25 May 2024
Boolean Logic as an Error feedback mechanism
Boolean Logic as an Error feedback mechanism
Louis Leconte
42
0
0
29 Jan 2024
Boolean Variation and Boolean Logic BackPropagation
Boolean Variation and Boolean Logic BackPropagation
Van Minh Nguyen
133
2
0
13 Nov 2023
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization
Jangwhan Lee
Minsoo Kim
Seungcheol Baek
Seok Joong Hwang
Wonyong Sung
Jungwook Choi
MQ
126
17
0
09 Nov 2023
Hadamard Domain Training with Integers for Class Incremental Quantized Learning
Hadamard Domain Training with Integers for Class Incremental Quantized Learning
Martin Schiemer
Clemens J. S. Schaefer
Jayden Parker Vap
Mark Horeni
Yu Emma Wang
Juan Ye
Siddharth Joshi
98
2
0
05 Oct 2023
Accuracy Booster: Enabling 4-bit Fixed-point Arithmetic for DNN Training
Accuracy Booster: Enabling 4-bit Fixed-point Arithmetic for DNN Training
Simla Burcu Harma
Canberk Sonmez
Nicholas Sperry
Babak Falsafi
Martin Jaggi
Yunho Oh
MQ
111
5
0
19 Nov 2022
AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks
AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks
Louis Leconte
S. Schechtman
Eric Moulines
131
4
0
07 Nov 2022
Energy Efficient Hardware Acceleration of Neural Networks with Power-of-Two Quantisation
Energy Efficient Hardware Acceleration of Neural Networks with Power-of-Two Quantisation
Dominika Przewlocka-Rus
T. Kryjak
MQ
69
5
0
30 Sep 2022