BitNet: Scaling 1-bit Transformers for Large Language Models

17 October 2023

Fan Yang

Papers citing "BitNet: Scaling 1-bit Transformers for Large Language Models"

22 / 22 papers shown

Title
PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNs Lukas Meiner Jens Mehnert A. P. Condurache MQ 30 0 0 06 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 47 0 0 05 May 2025
Precision Where It Matters: A Novel Spike Aware Mixed-Precision Quantization Strategy for LLaMA-based Language Models Lucas Maisonnave Cyril Moineau Olivier Bichler Fabrice Rastello MQ 69 1 0 30 Apr 2025
BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs Hongyu Wang Shuming Ma Furu Wei MQ 48 1 0 25 Apr 2025
Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs Lucas Maisonnave Cyril Moineau Olivier Bichler Fabrice Rastello MQ 37 0 0 18 Apr 2025
Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning Sanghwan Bae Jiwoo Hong Min Young Lee Hanbyul Kim Jeongyeon Nam Donghyun Kwak OffRL LRM 48 3 0 04 Apr 2025
Towards Lossless Implicit Neural Representation via Bit Plane Decomposition Woo Kyoung Han Byeonghun Lee Hyunmin Cho Sunghoon Im Kyong Hwan Jin MQ 82 0 0 28 Feb 2025
Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study Eric Aubinais Philippe Formont Pablo Piantanida Elisabeth Gassiat 38 0 0 10 Feb 2025
LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator Guoyu Li Shengyu Ye C. L. P. Chen Yang Wang Fan Yang Ting Cao Cheng Liu Mohamed M. Sabry Mao Yang MQ 78 0 0 18 Jan 2025
Scaling Laws for Floating Point Quantization Training X. Sun Shuaipeng Li Ruobing Xie Weidong Han Kan Wu ... Yangyu Tao Zhanhui Kang C. Xu Di Wang Jie Jiang MQ AIFin 53 0 0 05 Jan 2025
Unsupervised detection of semantic correlations in big data Santiago Acevedo Alex Rodriguez A. Laio 65 2 0 04 Nov 2024
Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models Qitan Lv Jie Wang Hanzhu Chen Bin Li Yongdong Zhang Feng Wu HILM 17 3 0 19 Oct 2024
MoDeGPT: Modular Decomposition for Large Language Model Compression Chi-Heng Lin Shangqian Gao James Seale Smith Abhishek Patel Shikhar Tuli Yilin Shen Hongxia Jin Yen-Chang Hsu 71 6 0 19 Aug 2024
Towards Resilient and Efficient LLMs: A Comparative Study of Efficiency, Performance, and Adversarial Robustness Xiaojing Fan Chunliang Tao AAML 32 28 0 08 Aug 2024
$u-$\mu$P: The Unit-Scaled Maximal Update Parametrization$ u- $\mu$ P: The Unit-Scaled Maximal Update Parametrization Charlie Blake C. Eichenberg Josef Dean Lukas Balles Luke Y. Prince Bjorn Deiseroth Andres Felipe Cruz Salinas Carlo Luschi Samuel Weinbach Douglas Orr 51 9 0 24 Jul 2024
T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edge Jianyu Wei Shijie Cao Ting Cao Lingxiao Ma Lei Wang Yanyong Zhang Mao Yang MQ 45 11 0 25 Jun 2024
BitNet b1.58 Reloaded: State-of-the-art Performance Also on Smaller Networks Jacob Nielsen Peter Schneider-Kamp MQ 35 4 0 24 Jun 2024
TerDiT: Ternary Diffusion Models with Transformers Xudong Lu Aojun Zhou Ziyi Lin Qi Liu Yuhui Xu Renrui Zhang Yafei Wen Shuai Ren Peng Gao Junchi Yan MQ 37 2 0 23 May 2024
GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning Han Zou Qiyang Zhao Lina Bariah Yu Tian M. Bennis S. Lasaulce 91 12 0 26 Feb 2024
BiT: Robustly Binarized Multi-distilled Transformer Zechun Liu Barlas Oğuz Aasish Pappu Lin Xiao Scott Yih Meng Li Raghuraman Krishnamoorthi Yashar Mehdad MQ 43 52 0 25 May 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,815 0 17 Sep 2019