A Speed Odyssey for Deployable Quantization of LLMs

16 November 2023

Papers citing "A Speed Odyssey for Deployable Quantization of LLMs"

3 / 3 papers shown

Title
iServe: An Intent-based Serving System for LLMs Dimitrios Liakopoulos Tianrui Hu Prasoon Sinha N. Yadwadkar VLM 54 0 0 08 Jan 2025
QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Models Saleh Ashkboos Ilia Markov Elias Frantar Tingxuan Zhong Xincheng Wang Jie Ren Torsten Hoefler Dan Alistarh MQ SyDa 115 21 0 13 Oct 2023
ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation Z. Yao Xiaoxia Wu Cheng-rong Li Stephen Youn Yuxiong He MQ 63 56 0 15 Mar 2023