Title
Don't be lazy: CompleteP enables compute-efficient deep transformers Nolan Dey Bin Claire Zhang Lorenzo Noci Mufan Bill Li Blake Bordelon Shane Bergsma C. Pehlevan Boris Hanin Joel Hestness 35 0 0 02 May 2025
FineScope : Precision Pruning for Domain-Specialized Large Language Models Using SAE-Guided Self-Data Cultivation Chaitali Bhattacharyya Yeseong Kim 38 0 0 01 May 2025
ICL CIPHERS: Quantifying "Learning'' in In-Context Learning via Substitution Ciphers Zhouxiang Fang Aayush Mishra Muhan Gao Anqi Liu Daniel Khashabi 36 0 0 28 Apr 2025
Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection Atharva Kulkarni Yuan-kang Zhang Joel Ruben Antony Moniz Xiou Ge Bo-Hsiang Tseng Dhivya Piraviperumal S. Hong-ye Yu HILM 76 0 0 25 Apr 2025
SEA-LION: Southeast Asian Languages in One Network Raymond Ng Thanh Ngan Nguyen Yuli Huang Ngee Chia Tai Wai Yi Leong ... David Ong Tat-Wee B. Liu William-Chandra Tjhi Erik Cambria Leslie Teo 17 11 0 08 Apr 2025
Not All Data Are Unlearned Equally Aravind Krishnan Siva Reddy Marius Mosbach MU 34 0 0 07 Apr 2025
ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Merging Haoming Xu Shuxun Wang Yanqiu Zhao Yi Zhong Ziyan Jiang Ningyuan Zhao Shumin Deng H. Chen N. Zhang MoMe MU 59 0 0 27 Mar 2025
Cross-Tokenizer Distillation via Approximate Likelihood Matching Benjamin Minixhofer Ivan Vulić E. Ponti 41 0 0 25 Mar 2025
The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generation Olivier Gouvert Julie Hunter Jérôme Louradour Christophe Cerisara Evan Dufraisse Yaya Sy Laura Rivière Jean-Pierre Lorré OpenLLM-France community 56 0 0 15 Mar 2025
Are formal and functional linguistic mechanisms dissociated in language models? Michael Hanna Sandro Pezzelle Yonatan Belinkov 41 0 0 14 Mar 2025
Evaluating and Aligning Human Economic Risk Preferences in LLMs J. Liu Yi Yang K. Tam 43 0 0 09 Mar 2025
Triple Phase Transitions: Understanding the Learning Dynamics of Large Language Models from a Neuroscience Perspective Yuko Nakagi Keigo Tada Sota Yoshino Shinji Nishimoto Yu Takagi LRM 32 0 0 28 Feb 2025
FOReCAst: The Future Outcome Reasoning and Confidence Assessment Benchmark Zhangdie Yuan Zifeng Ding Andreas Vlachos AI4TS 63 0 0 27 Feb 2025
Prediction hubs are context-informed frequent tokens in LLMs Beatrix M. G. Nielsen Iuri Macocco Marco Baroni 112 1 0 17 Feb 2025
EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models Xingrun Xing Zheng Liu Shitao Xiao Boyan Gao Yiming Liang Wanpeng Zhang Haokun Lin Guoqi Li Jiajun Zhang LRM 47 1 0 10 Feb 2025
Steel-LLM:From Scratch to Open Source -- A Personal Journey in Building a Chinese-Centric LLM Qingshui Gu Shu Li Tianyu Zheng Zhaoxiang Zhang 100 0 0 10 Feb 2025
Language Models Largely Exhibit Human-like Constituent Ordering Preferences Ada Defne Tur Gaurav Kamath Siva Reddy 52 0 0 08 Feb 2025
The Pitfalls of "Security by Obscurity" And What They Mean for Transparent AI Peter Hall Olivia Mundahl Sunoo Park 59 0 0 30 Jan 2025
Baichuan-Omni-1.5 Technical Report Yadong Li J. Liu Tao Zhang Tao Zhang S. Chen ... Jianhua Xu Haoze Sun Mingan Lin Zenan Zhou Weipeng Chen AuLLM 64 10 0 28 Jan 2025
The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories Raj Sanjay Shah Sashank Varma LRM 80 0 0 22 Jan 2025
Merging Feed-Forward Sublayers for Compressed Transformers Neha Verma Kenton W. Murray Kevin Duh AI4CE 36 0 0 10 Jan 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 54 4 0 31 Dec 2024
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 44 0 0 30 Dec 2024
Profiling Bias in LLMs: Stereotype Dimensions in Contextual Word Embeddings Carolin M. Schuster Maria-Alexandra Dinisor Shashwat Ghatiwala Georg Groh 54 1 0 25 Nov 2024
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 74 3 0 20 Nov 2024
OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models Siming Huang Tianhao Cheng J.K. Liu Jiaran Hao L. Song ... Ge Zhang Zili Wang Yuan Qi Yinghui Xu Wei Chu ALM 48 16 0 07 Nov 2024
How Does Critical Batch Size Scale in Pre-training? Hanlin Zhang Depen Morwani Nikhil Vyas Jingfeng Wu Difan Zou Udaya Ghai Dean Phillips Foster Sham Kakade 51 8 0 29 Oct 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Y. Lu Kurt Keutzer Jianfei Chen Song Han MQ 38 9 0 25 Oct 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 32 3 0 23 Oct 2024
Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotation M. Lin Z. Chen Yanchi Liu Xujiang Zhao Zongyu Wu Junxiang Wang Xiang Zhang Suhang Wang Haifeng Chen AI4TS 15 7 0 22 Oct 2024
Self-calibration for Language Model Quantization and Pruning Miles Williams G. Chrysostomou Nikolaos Aletras MQ 25 0 0 22 Oct 2024
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 37 5 0 22 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 81 12 0 11 Oct 2024
Scaling Laws for Predicting Downstream Performance in LLMs Yangyi Chen Binxuan Huang Yifan Gao Zhengyang Wang Jingfeng Yang Heng Ji LRM 41 7 0 11 Oct 2024
Detecting Training Data of Large Language Models via Expectation Maximization Gyuwan Kim Yang Li Evangelia Spiliopoulou Jie Ma Miguel Ballesteros William Yang Wang MIALM 80 3 2 10 Oct 2024
Data Selection via Optimal Control for Language Models Yuxian Gu Li Dong Hongning Wang Y. Hao Qingxiu Dong Furu Wei Minlie Huang AI4CE 22 4 0 09 Oct 2024
AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text Ximing Lu Melanie Sclar Skyler Hallinan Niloofar Mireshghallah Jiacheng Liu ... Allyson Ettinger Liwei Jiang Khyathi Raghavi Chandu Nouha Dziri Yejin Choi DeLMO 30 11 0 05 Oct 2024
How Much Can We Forget about Data Contamination? Sebastian Bordt Suraj Srinivas Valentyn Boreiko U. V. Luxburg 29 1 0 04 Oct 2024
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Jiyeon Kim Hyunji Lee Hyowon Cho Joel Jang Hyeonbin Hwang Seungpil Won Youbin Ahn Dohaeng Lee Minjoon Seo KELM 31 2 0 02 Oct 2024
Hyper-Connections Defa Zhu Hongzhi Huang Zihao Huang Yutao Zeng Yunyao Mao Banggu Wu Qiyang Min Xun Zhou 19 3 0 29 Sep 2024
Speechworthy Instruction-tuned Language Models Hyundong Justin Cho Nicolaas Jedema Leonardo F. R. Ribeiro Karishma Sharma Pedro Szekely Alessandro Moschitti Ruben Janssen Jonathan May ALM 21 1 0 23 Sep 2024
SOAP: Improving and Stabilizing Shampoo using Adam Nikhil Vyas Depen Morwani Rosie Zhao Itai Shapira David Brandfonbrener Lucas Janson Sham Kakade Sham Kakade 44 23 0 17 Sep 2024
Improving Pretraining Data Using Perplexity Correlations Tristan Thrush Christopher Potts Tatsunori Hashimoto 30 17 0 09 Sep 2024
Programming Refusal with Conditional Activation Steering Bruce W. Lee Inkit Padhi K. Ramamurthy Erik Miehling Pierre L. Dognin Manish Nagireddy Amit Dhurandhar LLMSV 82 13 0 06 Sep 2024
How Does Code Pretraining Affect Language Model Task Performance? Jackson Petty Sjoerd van Steenkiste Tal Linzen 30 7 0 06 Sep 2024
Does Alignment Tuning Really Break LLMs' Internal Confidence? Hongseok Oh Wonseok Hwang 22 0 0 31 Aug 2024
Counterfactuals As a Means for Evaluating Faithfulness of Attribution Methods in Autoregressive Language Models Sepehr Kamahi Yadollah Yaghoobzadeh 22 0 0 21 Aug 2024
Training on the Test Task Confounds Evaluation and Emergence Ricardo Dominguez-Olmedo Florian E. Dorner Moritz Hardt ELM 36 6 1 10 Jul 2024
Deconstructing What Makes a Good Optimizer for Language Models Rosie Zhao Depen Morwani David Brandfonbrener Nikhil Vyas Sham Kakade 29 17 0 10 Jul 2024
From Loops to Oops: Fallback Behaviors of Language Models Under Uncertainty Maor Ivgi Ori Yoran Jonathan Berant Mor Geva HILM 31 8 0 08 Jul 2024