Large Language Monkeys: Scaling Inference Compute with Repeated Sampling

3 January 2025

Papers citing "Large Language Monkeys: Scaling Inference Compute with Repeated Sampling"

50 / 153 papers shown

Title
Simple and Provable Scaling Laws for the Test-Time Compute of Large Language Models Yanxi Chen Xuchen Pan Yaliang Li Bolin Ding Jingren Zhou LRM 60 9 0 29 Nov 2024
Inference Scaling fLaws: The Limits of LLM Resampling with Imperfect Verifiers Benedikt Stroebl Sayash Kapoor Arvind Narayanan LRM 80 6 0 26 Nov 2024
VLRewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models Lei Li Y. X. Wei Zhihui Xie Xuqing Yang Yifan Song ... Tianyu Liu Sujian Li Bill Yuchen Lin Lingpeng Kong Q. Liu CoGe VLM 107 24 0 26 Nov 2024
Drowning in Documents: Consequences of Scaling Reranker Inference Mathew Jacob Erik Lindgren Matei A. Zaharia Michael Carbin Omar Khattab Andrew Drozdov OffRL 74 4 0 18 Nov 2024
AtomThink: A Slow Thinking Framework for Multimodal Mathematical Reasoning Kun Xiang Zhili Liu Zihao Jiang Yunshuang Nie Runhui Huang ... Yihan Zeng J. Han Lanqing Hong Hang Xu Xiaodan Liang LRM 99 10 0 18 Nov 2024
Scaling Laws for Precision Tanishq Kumar Zachary Ankner Benjamin Spector Blake Bordelon Niklas Muennighoff Mansheej Paul C. Pehlevan Christopher Ré Aditi Raghunathan AIFin MoMe 33 12 0 07 Nov 2024
Scaling LLM Inference with Optimized Sample Compute Allocation Kexun Zhang Shang Zhou Danqing Wang William Yang Wang Lei Li 35 5 0 29 Oct 2024
Beyond Autoregression: Fast LLMs via Self-Distillation Through Time Justin Deschenaux Çağlar Gülçehre 32 2 0 28 Oct 2024
Library Learning Doesn't: The Curious Case of the Single-Use "Library" Ian Berlot-Attwell Frank Rudzicz Xujie Si 34 1 0 26 Oct 2024
SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement Antonis Antoniades Albert Örwall Kexun Zhang Yuxi Xie Anirudh Goyal William Yang Wang LLMAG 31 11 0 26 Oct 2024
$C^2$ : Scalable Auto-Feedback for LLM-based Chart Generation Woosung Koh Jang Han Yoon M. Lee Youngjin Song Jaegwan Cho Jaehyun Kang Taehyeon Kim Se-Young Yun Youngjae Yu B. Lee 34 0 0 24 Oct 2024
Little Giants: Synthesizing High-Quality Embedding Data at Scale Haonan Chen Liang Wang Nan Yang Y. X. Zhu Ziliang Zhao Furu Wei Zhicheng Dou SyDa 18 1 0 24 Oct 2024
A Simple Model of Inference Scaling Laws Noam Levi LRM 19 0 0 21 Oct 2024
Keep Guessing? When Considering Inference Scaling, Mind the Baselines G. Yona Or Honovich Omer Levy Roee Aharoni UQLM LRM 21 0 0 20 Oct 2024
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling Jiahao Qiu Yifu Lu Yifan Zeng Jiacheng Guo Jiayi Geng Huazheng Wang Kaixuan Huang Yue Wu Mengdi Wang 34 19 0 18 Oct 2024
GenEOL: Harnessing the Generative Power of LLMs for Training-Free Sentence Embeddings Raghuveer Thirukovalluru Bhuwan Dhingra 18 2 0 18 Oct 2024
How Numerical Precision Affects Mathematical Reasoning Capabilities of LLMs Guhao Feng Kai-Bo Yang Yuntian Gu Xinyue Ai Shengjie Luo Jiacheng Sun Di He Z. Li Liwei Wang LRM 25 1 0 17 Oct 2024
MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation S. Gorti Ilan Gofman Zhaoyan Liu Jiapeng Wu Noël Vouitsis Guangwei Yu Jesse C. Cresswell Rasa Hosseinzadeh SyDa 41 6 0 16 Oct 2024
JudgeBench: A Benchmark for Evaluating LLM-based Judges Sijun Tan Siyuan Zhuang Kyle Montgomery William Y. Tang Alejandro Cuadron Chenguang Wang Raluca A. Popa Ion Stoica ELM ALM 39 35 0 16 Oct 2024
Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System Weize Chen Jiarui Yuan Chen Qian Cheng Yang Zhiyuan Liu Maosong Sun LLMAG 26 4 0 10 Oct 2024
Efficient Reinforcement Learning with Large Language Model Priors Xue Yan Yan Song Xidong Feng Mengyue Yang Haifeng Zhang Haitham Bou Ammar Jun Wang OffRL 18 1 0 10 Oct 2024
ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time Yi Ding Bolian Li Ruqi Zhang MLLM 49 4 0 09 Oct 2024
Learning How Hard to Think: Input-Adaptive Allocation of LM Computation Mehul Damani Idan Shenfeld Andi Peng Andreea Bobu Jacob Andreas 26 14 0 07 Oct 2024
Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification Zhenwen Liang Ye Liu Tong Niu Xiangliang Zhang Yingbo Zhou Semih Yavuz LRM 17 8 0 05 Oct 2024
DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search Murong Yue Wenlin Yao Haitao Mi Dian Yu Ziyu Yao Dong Yu LRM 28 4 0 04 Oct 2024
SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains? John Yang Carlos E. Jimenez Alex Zhang K. Lieret Joyce Yang ... Gabriel Synnaeve Karthik Narasimhan Diyi Yang Sida I. Wang Ofir Press 19 17 0 04 Oct 2024
ToolGen: Unified Tool Retrieval and Calling via Generation Renxi Wang Xudong Han Lei Ji Shu Wang Timothy Baldwin Haonan Li LLMAG 43 6 0 04 Oct 2024
LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning Di Zhang Jianbo Wu Jingdi Lei Tong Che Jiatong Li ... Shufei Zhang Marco Pavone Yuqiang Li Wanli Ouyang Dongzhan Zhou LRM 19 42 0 03 Oct 2024
Training Language Models on Synthetic Edit Sequences Improves Code Synthesis Ulyana Piterbarg Lerrel Pinto Rob Fergus SyDa 24 2 0 03 Oct 2024
Recursive Abstractive Processing for Retrieval in Dynamic Datasets Charbel Chucri Rami Azouz Joachim Ott 41 0 0 02 Oct 2024
OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data Shubham Toshniwal Wei Du Ivan Moshkov Branislav Kisacanin Alexan Ayrapetyan Igor Gitman LRM 16 6 0 02 Oct 2024
Integrative Decoding: Improve Factuality via Implicit Self-consistency Yi Cheng Xiao Liang Yeyun Gong Wen Xiao Song Wang ... Wenjie Li Jian Jiao Qi Chen Peng Cheng Wayne Xiong HILM 41 1 0 02 Oct 2024
Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling Jinghan Li Zhicheng Sun Fei Li 68 1 0 02 Oct 2024
TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking Danqing Wang Jianxin Ma Fei Fang Lei Li LLMAG LRM 45 0 0 02 Oct 2024
Revisiting the Superficial Alignment Hypothesis Mohit Raghavendra Vaskar Nath Sean Hendryx LRM 21 0 0 27 Sep 2024
Archon: An Architecture Search Framework for Inference-Time Techniques Jon Saad-Falcon Adrian Gamarra Lafuente Shlok Natarajan Nahum Maru Hristo Todorov ... E. Kelly Buchanan Mayee Chen Neel Guha Christopher Ré Azalia Mirhoseini AI4CE 21 12 0 23 Sep 2024
Takin: A Cohort of Superior Quality Zero-shot Speech Generation Models Sijing Chen Yuan Feng Laipeng He Tianwei He Wendi He ... Huimin Zhang Xiang Zhang Guangcheng Zhao Hongbin Zhou Pengpeng Zou 20 4 0 18 Sep 2024
CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark Zachary S. Siegel Sayash Kapoor Nitya Nagdir Benedikt Stroebl Arvind Narayanan 14 0 0 17 Sep 2024
Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling Hritik Bansal Arian Hosseini Rishabh Agarwal Vinh Q. Tran Mehran Kazemi SyDa OffRL LRM 25 28 0 29 Aug 2024
Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning Xinglin Wang Shaoxiong Feng Yiwei Li Peiwen Yuan Y. Zhang Boyuan Pan Heda Wang Yao Hu Kan Li LRM 37 16 0 24 Aug 2024
Preference-Guided Reflective Sampling for Aligning Language Models Hai Ye Hwee Tou Ng 19 1 0 22 Aug 2024
Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents Pranav Putta Edmund Mills Naman Garg S. Motwani Chelsea Finn Divyansh Garg Rafael Rafailov LLMAG LRM 18 19 0 13 Aug 2024
Variational Best-of-N Alignment Afra Amini Tim Vieira Ryan Cotterell Ryan Cotterell BDL 35 17 0 08 Jul 2024
Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts Haoxiang Wang Wei Xiong Tengyang Xie Han Zhao Tong Zhang 40 13 0 18 Jun 2024
Scaling Large Language Model-based Multi-Agent Collaboration Chen Qian Zihao Xie YiFei Wang Wei Liu Yufan Dang ... Zhuoyun Du Weize Chen Cheng Yang Zhiyuan Liu Maosong Sun AI4CE LLMAG LM&Ro 52 42 0 11 Jun 2024
Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass Ethan Shen Alan Fan Sarah M Pratt Jae Sung Park Matthew Wallingford Sham Kakade Ari Holtzman Ranjay Krishna Ali Farhadi Aditya Kusupati 25 2 0 28 May 2024
GP-MoLFormer: A Foundation Model For Molecular Generation Jerret Ross Brian M. Belgodere Samuel C. Hoffman Vijil Chenthamarakshan Youssef Mroueh Payel Das Payel Das 27 4 0 04 Apr 2024
Hydragen: High-Throughput LLM Inference with Shared Prefixes Jordan Juravsky Bradley Brown Ryan Ehrlich Daniel Y. Fu Christopher Ré Azalia Mirhoseini 41 35 0 07 Feb 2024
Theoretical guarantees on the best-of-n alignment policy Ahmad Beirami Alekh Agarwal Jonathan Berant Alex DÁmour Jacob Eisenstein Chirag Nagpal A. Suresh 42 42 0 03 Jan 2024
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022