Accurate Neural Training with 4-bit Matrix Multiplications at Standard
Formats

v1v2v3v4 (latest)

Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats

19 December 2021

Hilla Ben Yaacov

ArXiv (abs)PDF HTML

Papers citing "Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats"

15 / 15 papers shown

Title
FP4 All the Way: Fully Quantized Training of LLMs Brian Chmiel Maxim Fishman Ron Banner Daniel Soudry MQ 133 0 0 25 May 2025
Quartet: Native FP4 Training Can Be Optimal for Large Language Models Roberto L. Castro Andrei Panferov Soroush Tabesh Oliver Sieberling Jiale Chen Mahdi Nikdan Saleh Ashkboos Dan Alistarh MQ 155 0 0 20 May 2025
HOT: Hadamard-based Optimized Training Seonggon Kim Juncheol Shin Seung-taek Woo Eunhyeok Park 144 0 0 27 Mar 2025
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization Minsu Kim Seongmin Hong RyeoWook Ko S. Choi Hunjong Lee Junsoo Kim Joo-Young Kim Jongse Park 157 0 0 24 Mar 2025
EXAQ: Exponent Aware Quantization For LLMs Acceleration Moran Shkolnik Maxim Fishman Brian Chmiel Hilla Ben-Yaacov Ron Banner Kfir Y. Levy MQ 73 0 0 04 Oct 2024
HLQ: Fast and Efficient Backpropagation via Hadamard Low-rank Quantization Seonggon Kim Eunhyeok Park 106 2 0 21 Jun 2024
LoQT: Low Rank Adapters for Quantized Training Sebastian Loeschcke M. Toftrup M. Kastoryano Serge Belongie Vésteinn Snæbjarnarson MQ 100 5 0 26 May 2024
BOLD: Boolean Logic Deep Learning Van Minh Nguyen Cristian Ocampo Aymen Askri Louis Leconte Ba-Hien Tran AI4CE 140 1 0 25 May 2024
Boolean Logic as an Error feedback mechanism Louis Leconte 42 0 0 29 Jan 2024
Boolean Variation and Boolean Logic BackPropagation Van Minh Nguyen 125 2 0 13 Nov 2023
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization Jangwhan Lee Minsoo Kim Seungcheol Baek Seok Joong Hwang Wonyong Sung Jungwook Choi MQ 122 17 0 09 Nov 2023
Hadamard Domain Training with Integers for Class Incremental Quantized Learning Martin Schiemer Clemens J. S. Schaefer Jayden Parker Vap Mark Horeni Yu Emma Wang Juan Ye Siddharth Joshi 94 2 0 05 Oct 2023
Accuracy Booster: Enabling 4-bit Fixed-point Arithmetic for DNN Training Simla Burcu Harma Canberk Sonmez Nicholas Sperry Babak Falsafi Martin Jaggi Yunho Oh MQ 111 5 0 19 Nov 2022
AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks Louis Leconte S. Schechtman Eric Moulines 127 4 0 07 Nov 2022
Energy Efficient Hardware Acceleration of Neural Networks with Power-of-Two Quantisation Dominika Przewlocka-Rus T. Kryjak MQ 69 5 0 30 Sep 2022

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.