Compositional preference models for aligning LMs

17 October 2023

Papers citing "Compositional preference models for aligning LMs"

20 / 20 papers shown

Title
Capturing Individual Human Preferences with Reward Features André Barreto Vincent Dumoulin Yiran Mao Nicolas Perez-Nieves Bobak Shahriari Yann Dauphin Doina Precup Hugo Larochelle ALM 62 1 0 21 Mar 2025
Dataset Featurization: Uncovering Natural Language Features through Unsupervised Data Reconstruction Michal Bravansky Vaclav Kubon Suhas Hariharan Robert Kirk 69 0 0 24 Feb 2025
Interpreting Language Reward Models via Contrastive Explanations Junqi Jiang Tom Bewley Saumitra Mishra Freddy Lecue Manuela Veloso 74 0 0 25 Nov 2024
Fast Best-of-N Decoding via Speculative Rejection Hanshi Sun Momin Haider Ruiqi Zhang Huitao Yang Jiahao Qiu Ming Yin Mengdi Wang Peter L. Bartlett Andrea Zanette BDL 42 28 0 26 Oct 2024
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling Jiahao Qiu Yifu Lu Yifan Zeng Jiacheng Guo Jiayi Geng Huazheng Wang Kaixuan Huang Yue Wu Mengdi Wang 34 22 0 18 Oct 2024
Personality Alignment of Large Language Models Minjun Zhu Linyi Yang Yue Zhang Yue Zhang ALM 64 5 0 21 Aug 2024
HAF-RM: A Hybrid Alignment Framework for Reward Model Training Shujun Liu Xiaoyu Shen Yuhang Lai Siyuan Wang Shengbin Yue Zengfeng Huang Xuanjing Huang Zhongyu Wei 31 1 0 04 Jul 2024
Multi-objective Reinforcement learning from AI Feedback Marcus Williams 38 1 0 11 Jun 2024
Information Theoretic Guarantees For Policy Alignment In Large Language Models Youssef Mroueh 29 6 0 09 Jun 2024
Inverse Constitutional AI: Compressing Preferences into Principles Arduin Findeis Timo Kaufmann Eyke Hüllermeier Samuel Albanie Robert Mullins SyDa 43 9 0 02 Jun 2024
AdvisorQA: Towards Helpful and Harmless Advice-seeking Question Answering with Collective Intelligence Minbeom Kim Hwanhee Lee Joonsuk Park Hwaran Lee Kyomin Jung 32 1 0 18 Apr 2024
MetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models Kailai Yang Zhiwei Liu Qianqian Xie Jimin Huang Tianlin Zhang Sophia Ananiadou 32 14 0 25 Mar 2024
ALaRM: Align Language Models via Hierarchical Rewards Modeling Yuhang Lai Siyuan Wang Shujun Liu Xuanjing Huang Zhongyu Wei 26 4 0 11 Mar 2024
Theoretical guarantees on the best-of-n alignment policy Ahmad Beirami Alekh Agarwal Jonathan Berant Alex DÁmour Jacob Eisenstein Chirag Nagpal A. Suresh 42 42 0 03 Jan 2024
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 227 500 0 28 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,915 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 355 8,457 0 28 Jan 2022
$Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information$ Understanding Dataset Difficulty with $\mathcal{V}$ -Usable Information Kawin Ethayarajh Yejin Choi Swabha Swayamdipta 161 157 0 16 Oct 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 280 1,587 0 18 Sep 2019
AI safety via debate G. Irving Paul Christiano Dario Amodei 204 199 0 02 May 2018