Theoretical guarantees on the best-of-n alignment policy

3 January 2024

Papers citing "Theoretical guarantees on the best-of-n alignment policy"

35 / 35 papers shown

Title
Soft Best-of-n Sampling for Model Alignment C. M. Verdun Alex Oesterling Himabindu Lakkaraju Flavio du Pin Calmon BDL 44 0 0 06 May 2025
Bridging the Gap Between Preference Alignment and Machine Unlearning Xiaohua Feng Yuyuan Li Huwei Ji Jiaming Zhang L. Zhang Tianyu Du Chaochao Chen MU 33 0 0 09 Apr 2025
Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations Pedro Ferreira Wilker Aziz Ivan Titov LRM 18 0 0 07 Apr 2025
T1: Tool-integrated Self-verification for Test-time Compute Scaling in Small Language Models Minki Kang Jongwon Jeong Jaewoong Cho ALM LRM 33 2 0 07 Apr 2025
Review, Refine, Repeat: Understanding Iterative Decoding of AI Agents with Dynamic Evaluation and Selection Souradip Chakraborty Mohammadreza Pourreza Ruoxi Sun Yiwen Song Nino Scherrer ... Furong Huang Amrit Singh Bedi Ahmad Beirami Hamid Palangi Tomas Pfister 44 0 0 02 Apr 2025
Cultural Alignment in Large Language Models Using Soft Prompt Tuning Reem I. Masoud Martin Ferianc Philip C. Treleaven Miguel R. D. Rodrigues ALM 41 0 0 20 Mar 2025
Robust Multi-Objective Controlled Decoding of Large Language Models Seongho Son William Bankes Sangwoong Yoon Shyam Sundhar Ramesh Xiaohang Tang Ilija Bogunovic 34 0 0 11 Mar 2025
Dynamic Search for Inference-Time Alignment in Diffusion Models Xiner Li Masatoshi Uehara Xingyu Su Gabriele Scalia Tommaso Biancalani Aviv Regev Sergey Levine Shuiwang Ji 39 0 0 03 Mar 2025
Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding Y. Wang Pei Zhang Siyuan Huang Baosong Yang Z. Zhang Fei Huang Rui Wang BDL LRM 59 6 0 03 Mar 2025
Faster WIND: Accelerating Iterative Best-of- $N$ Distillation for LLM Alignment Tong Yang Jincheng Mei H. Dai Zixin Wen Shicong Cen Dale Schuurmans Yuejie Chi Bo Dai 24 4 0 20 Feb 2025
Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics Daniel J.H. Chung Zhiqi Gao Yurii Kvasiuk Tianyi Li Moritz Münchmeyer Maja Rudolph Frederic Sala Sai Chaitanya Tadepalli AIMat 39 3 0 19 Feb 2025
Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization Jiangweizhi Peng Zhiwei Tang Gaowen Liu Charles Fleming Mingyi Hong 66 2 0 05 Dec 2024
Fast Best-of-N Decoding via Speculative Rejection Hanshi Sun Momin Haider Ruiqi Zhang Huitao Yang Jiahao Qiu Ming Yin Mengdi Wang Peter L. Bartlett Andrea Zanette BDL 29 26 0 26 Oct 2024
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling Jiahao Qiu Yifu Lu Yifan Zeng Jiacheng Guo Jiayi Geng Huazheng Wang Kaixuan Huang Yue Wu Mengdi Wang 34 19 0 18 Oct 2024
Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction Jarrid Rector-Brooks Mohsin Hasan Zhangzhi Peng Zachary Quinn Chenghao Liu ... Michael Bronstein Yoshua Bengio Pranam Chatterjee Alexander Tong Avishek Joey Bose DiffM 34 4 0 10 Oct 2024
Learning How Hard to Think: Input-Adaptive Allocation of LM Computation Mehul Damani Idan Shenfeld Andi Peng Andreea Bobu Jacob Andreas 26 14 0 07 Oct 2024
Inference-Time Language Model Alignment via Integrated Value Guidance Zhixuan Liu Zhanhui Zhou Yuanfu Wang Chao Yang Yu Qiao 24 7 0 26 Sep 2024
Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding Xiner Li Yulai Zhao Chenyu Wang Gabriele Scalia Gökçen Eraslan Surag Nair Tommaso Biancalani Aviv Regev Sergey Levine Masatoshi Uehara 41 22 0 15 Aug 2024
Variational Best-of-N Alignment Afra Amini Tim Vieira Ryan Cotterell Ryan Cotterell BDL 35 17 0 08 Jul 2024
From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models Sean Welleck Amanda Bertsch Matthew Finlayson Hailey Schoelkopf Alex Xie Graham Neubig Ilia Kulikov Zaid Harchaoui 31 45 0 24 Jun 2024
Towards Comprehensive Preference Data Collection for Reward Modeling Yulan Hu Qingyang Li Sheng Ouyang Ge Chen Kaihui Chen Lijun Mei Xucheng Ye Fuzheng Zhang Yong Liu SyDa 26 4 0 24 Jun 2024
Legend: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets Duanyu Feng Bowen Qin Chen Huang Youcheng Huang Zheng-Wei Zhang Wenqiang Lei 36 2 0 12 Jun 2024
Information Theoretic Guarantees For Policy Alignment In Large Language Models Youssef Mroueh 21 6 0 09 Jun 2024
BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling Lin Gui Cristina Garbacea Victor Veitch BDL LM&MA 34 35 0 02 Jun 2024
Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models Zhanhui Zhou Zhixuan Liu Jie Liu Zhichen Dong Chao Yang Yu Qiao ALM 28 20 0 29 May 2024
RLHF Workflow: From Reward Modeling to Online RLHF Hanze Dong Wei Xiong Bo Pang Haoxiang Wang Han Zhao Yingbo Zhou Nan Jiang Doyen Sahoo Caiming Xiong Tong Zhang OffRL 21 92 0 13 May 2024
Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment Zhaofeng Wu Ananth Balashankar Yoon Kim Jacob Eisenstein Ahmad Beirami 35 13 0 18 Apr 2024
Asymptotics of Language Model Alignment Joy Qiping Yang Salman Salamatian Ziteng Sun A. Suresh Ahmad Beirami 61 21 0 02 Apr 2024
Regularized Best-of-N Sampling with Minimum Bayes Risk Objective for Language Model Alignment Yuu Jinnai Tetsuro Morimura Kaito Ariu Kenshi Abe 46 7 0 01 Apr 2024
DMoERM: Recipes of Mixture-of-Experts for Effective Reward Modeling Shanghaoran Quan MoE OffRL 33 9 0 02 Mar 2024
Bayesian Reward Models for LLM Alignment Adam X. Yang Maxime Robeyns Thomas Coste Zhengyan Shi Jun Wang Haitham Bou-Ammar Laurence Aitchison 26 17 0 20 Feb 2024
Online Iterative Reinforcement Learning from Human Feedback with General Preference Model Chen Ye Wei Xiong Yuheng Zhang Nan Jiang Tong Zhang OffRL 25 3 0 11 Feb 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 92 92 0 22 Jan 2024
Controlled Decoding from Language Models Sidharth Mudgal Jong Lee H. Ganapathy Yaguang Li Tao Wang ... Michael Collins Trevor Strohman Jilin Chen Alex Beutel Ahmad Beirami 24 69 0 25 Oct 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022