Title
Uncertainty-Aware Hybrid Inference with On-Device Small and Remote Large Language Models Seungeun Oh Jinhyuk Kim Jihong Park Seung-Woo Ko Tony Q. S. Quek Seong-Lyun Kim 63 3 0 17 Dec 2024
Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families Felipe Maia Polo S. Kamath S Leshem Choshen Yuekai Sun Mikhail Yurochkin 84 5 0 09 Dec 2024
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 77 1 0 28 Nov 2024
Are Transformers Truly Foundational for Robotics? James A. R. Marshall Andrew B. Barron AI4CE 71 0 0 25 Nov 2024
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 139 3 0 20 Nov 2024
DELIFT: Data Efficient Language model Instruction Fine Tuning Ishika Agarwal Krishnateja Killamsetty Lucian Popa Marina Danilevksy ALM VLM 46 2 0 07 Nov 2024
Sparsing Law: Towards Large Language Models with Greater Activation Sparsity Yuqi Luo Chenyang Song Xu Han Y. Chen Chaojun Xiao Zhiyuan Liu Maosong Sun 47 3 0 04 Nov 2024
How Does Critical Batch Size Scale in Pre-training? Hanlin Zhang Depen Morwani Nikhil Vyas Jingfeng Wu Difan Zou Udaya Ghai Dean Phillips Foster Sham Kakade 64 8 0 29 Oct 2024
Beyond Autoregression: Fast LLMs via Self-Distillation Through Time Justin Deschenaux Çağlar Gülçehre 39 2 0 28 Oct 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Y. Lu Kurt Keutzer Jianfei Chen Song Han MQ 60 9 0 25 Oct 2024
Scaling up Masked Diffusion Models on Text Shen Nie Fengqi Zhu Chao Du Tianyu Pang Qian Liu Guangtao Zeng Min-Bin Lin Chongxuan Li AI4CE 45 13 0 24 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 44 3 0 24 Oct 2024
Scaling Diffusion Language Models via Adaptation from Autoregressive Models Shansan Gong Shivam Agarwal Yizhe Zhang Jiacheng Ye Lin Zheng ... Peilin Zhao W. Bi Jiawei Han Hao Peng Lingpeng Kong AI4CE 70 14 0 23 Oct 2024
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 65 5 0 22 Oct 2024
CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts Zhenpeng Su Xing Wu Zijia Lin Yizhe Xiong Minxuan Lv Guangyuan Ma Hui Chen Songlin Hu Guiguang Ding MoE 26 2 0 21 Oct 2024
Compute-Constrained Data Selection Junjie Oscar Yin Alexander M. Rush 39 0 0 21 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 59 3 0 20 Oct 2024
Scalable Multi-Domain Adaptation of Language Models using Modular Experts Peter Schafhalter Shun Liao Yanqi Zhou Chih-Kuan Yeh Arun Kandoor James Laudon MoE 24 1 0 14 Oct 2024
VideoAgent: Self-Improving Video Generation Achint Soni Sreyas Venkataraman Abhranil Chandra Sebastian Fischmeister Percy Liang Bo Dai Sherry Yang LM&Ro VGen 50 7 0 14 Oct 2024
Self-Data Distillation for Recovering Quality in Pruned Large Language Models Vithursan Thangarasa Ganesh Venkatesh Mike Lasby Nish Sinnadurai Sean Lie SyDa 33 0 0 13 Oct 2024
Scaling Laws for Predicting Downstream Performance in LLMs Yangyi Chen Binxuan Huang Yifan Gao Zhengyang Wang Jingfeng Yang Heng Ji LRM 43 7 0 11 Oct 2024
Data Processing for the OpenGPT-X Model Family Nicolo' Brandizzi Hammam Abdelwahab Anirban Bhowmick Lennard Helmer Benny Jörg Stein ... Georg Rehm Dennis Wegener Nicolas Flores-Herr Joachim Kohler Johannes Leveling VLM 79 2 0 11 Oct 2024
COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act Philipp Guldimann Alexander Spiridonov Robin Staab Nikola Jovanović Mark Vero ... Mislav Balunović Nikola Konstantinov Pavol Bielik Petar Tsankov Martin Vechev ELM 45 4 0 10 Oct 2024
Data Selection via Optimal Control for Language Models Yuxian Gu Li Dong Hongning Wang Y. Hao Qingxiu Dong Furu Wei Minlie Huang AI4CE 48 4 0 09 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi-An Ma 23 1 0 09 Oct 2024
DEPT: Decoupled Embeddings for Pre-training Language Models Alex Iacob Lorenzo Sani Meghdad Kurmanji William F. Shen Xinchi Qiu Dongqi Cai Yan Gao Nicholas D. Lane VLM 91 0 0 07 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 58 6 0 06 Oct 2024
Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective Jinhao Li Jiaming Xu Shan Huang Yonghua Chen Wen Li ... Jiayi Pan Li Ding Hao Zhou Yu Wang Guohao Dai 57 15 0 06 Oct 2024
How Much Can We Forget about Data Contamination? Sebastian Bordt Suraj Srinivas Valentyn Boreiko U. V. Luxburg 43 1 0 04 Oct 2024
No Need to Talk: Asynchronous Mixture of Language Models Anastasiia Filippova Angelos Katharopoulos David Grangier Ronan Collobert MoE 33 0 0 04 Oct 2024
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization Xinhao Yao Hongjin Qian Xiaolin Hu Gengze Xu Wei Liu Jian Luan B. Wang Y. Liu 48 0 0 03 Oct 2024
Towards Understanding the Universality of Transformers for Next-Token Prediction Michael E. Sander Gabriel Peyré CML 34 0 0 03 Oct 2024
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 29 0 0 01 Oct 2024
Scaling Optimal LR Across Token Horizons Johan Bjorck Alon Benhaim Vishrav Chaudhary Furu Wei Xia Song 46 4 0 30 Sep 2024
Can Models Learn Skill Composition from Examples? Haoyu Zhao Simran Kaur Dingli Yu Anirudh Goyal Sanjeev Arora CoGe MoE 53 2 0 29 Sep 2024
How Feature Learning Can Improve Neural Scaling Laws Blake Bordelon Alexander B. Atanasov C. Pehlevan 49 12 0 26 Sep 2024
Small Language Models: Survey, Measurements, and Insights Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane Mengwei Xu ObjD LRM 51 36 0 24 Sep 2024
Flash STU: Fast Spectral Transform Units Y. Isabel Liu Windsor Nguyen Yagiz Devre Evan Dogariu Anirudha Majumdar Elad Hazan AI4TS 64 1 0 16 Sep 2024
AlpaPICO: Extraction of PICO Frames from Clinical Trial Documents Using LLMs Madhusudan Ghosh Shrimon Mukherjee Asmit Ganguly Partha Basuchowdhuri S. Naskar Debasis Ganguly 29 7 0 15 Sep 2024
meds_reader: A fast and efficient EHR processing library E. Steinberg Michael Wornow Suhana Bedi Jason Alan Fries Matthew B. A. McDermott N. Shah LM&MA 25 0 0 12 Sep 2024
Retro-li: Small-Scale Retrieval Augmented Generation Supporting Noisy Similarity Searches and Domain Shift Generalization Gentiana Rashiti G. Karunaratne Mrinmaya Sachan Abu Sebastian Abbas Rahimi RALM 32 0 0 12 Sep 2024
Improving Pretraining Data Using Perplexity Correlations Tristan Thrush Christopher Potts Tatsunori Hashimoto 32 17 0 09 Sep 2024
Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budget Andy T. Liu Yi-Cheng Lin Haibin Wu Stefan Winkler Hung-yi Lee 31 1 0 09 Sep 2024
How Does Code Pretraining Affect Language Model Task Performance? Jackson Petty Sjoerd van Steenkiste Tal Linzen 60 7 0 06 Sep 2024
Shaping the Future of Endangered and Low-Resource Languages -- Our Role in the Age of LLMs: A Keynote at ECIR 2024 Josiane Mothe 34 2 0 05 Sep 2024
AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation Zanlin Ni Yulin Wang Renping Zhou Rui Lu Jiayi Guo Jinyi Hu Zhiyuan Liu Yuan Yao Gao Huang 25 7 0 31 Aug 2024
CLOCR-C: Context Leveraging OCR Correction with Pre-trained Language Models Jonathan Bourne 49 4 0 30 Aug 2024
An Investigation of Warning Erroneous Chat Translations in Cross-lingual Communication Yunmeng Li Jun Suzuki Makoto Morishita Kaori Abe Kentaro Inui 53 1 0 28 Aug 2024
Scalable Autoregressive Image Generation with Mamba Haopeng Li Jinyue Yang Kexin Wang Xuerui Qiu Yuhong Chou Xin Li Guoqi Li Mamba 53 12 0 22 Aug 2024
Coupling without Communication and Drafter-Invariant Speculative Decoding Majid Daliri Christopher Musco A. Suresh 46 1 0 15 Aug 2024