LLMEasyQuant -- An Easy to Use Toolkit for LLM Quantization
- MQ
Main:8 Pages
8 Figures
Bibliography:3 Pages
5 Tables
Appendix:14 Pages
Abstract
Currently, there are many quantization methods appeared for LLM quantization, yet few are user-friendly and easy to be deployed locally. Packages like TensorRT and Quantohave many underlying structures and self-invoking internal functions, which are not conducive to developers' personalized development and learning for deployment. Therefore, we develop LLMEasyQuant, it is a package aiming to for easy quantization deployment which is user-friendly and suitable for beginners' learning.
View on arXivComments on this paper
