SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF

9 October 2023

Yi Dong

Zhilin Wang

Makesh Narsimhan Sreedhar

Papers citing "SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF"

28 / 28 papers shown

Title
A Survey on Progress in LLM Alignment from the Perspective of Reward Design Miaomiao Ji Yanqiu Wu Zhibin Wu Shoujin Wang Jian Yang Mark Dras Usman Naseem 39 0 0 05 May 2025
CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives Ayoung Lee Ryan Sungmo Kwon Peter Railton Lu Wang ELM 49 0 0 15 Apr 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian-Yu Guan J. Wu J. Li Chuanqi Cheng Wei Yu Wu LM&MA 69 0 0 21 Mar 2025
OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses Angela Lopez-Cardona Sebastian Idesis Miguel Barreda-Ángeles Sergi Abadal Ioannis Arapakis 46 0 0 13 Mar 2025
ChipAlign: Instruction Alignment in Large Language Models for Chip Design via Geodesic Interpolation Chenhui Deng Yunsheng Bai Haoxing Ren 26 1 0 31 Dec 2024
Identifying and Manipulating Personality Traits in LLMs Through Activation Engineering Rumi A. Allbert James K. Wiles Vlad Grankovsky LLMSV AI4CE 75 1 0 10 Dec 2024
Interpreting Language Reward Models via Contrastive Explanations Junqi Jiang Tom Bewley Saumitra Mishra Freddy Lecue Manuela Veloso 74 0 0 25 Nov 2024
On the Loss of Context-awareness in General Instruction Fine-tuning Yihan Wang Andrew Bai Nanyun Peng Cho-Jui Hsieh 73 1 0 05 Nov 2024
SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment Qin Liu Fei Wang Chaowei Xiao Muhao Chen 79 0 0 18 Oct 2024
Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models Hongchuan Zeng Senyu Han Lu Chen Kai Yu 57 6 0 15 Oct 2024
Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements Jingyu Zhang Ahmed Elgohary Ahmed Magooda Daniel Khashabi Benjamin Van Durme 74 2 0 11 Oct 2024
Reward-Augmented Data Enhances Direct Preference Alignment of LLMs Shenao Zhang Zhihan Liu Boyi Liu Y. Zhang Yingxiang Yang Y. Liu Liyu Chen Tao Sun Z. Wang 87 2 0 10 Oct 2024
Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal Ioannis Arapakis ALM 48 2 0 02 Oct 2024
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown Xingzhou Lou Dong Yan Wei Shen Yuzi Yan Jian Xie Junge Zhang 45 21 0 01 Oct 2024
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Wenxuan Zhang Philip H. S. Torr Mohamed Elhoseiny Adel Bibi 51 9 0 27 Aug 2024
Towards Aligning Language Models with Textual Feedback Sauc Abadal Lloret S. Dhuliawala K. Murugesan Mrinmaya Sachan VLM 38 1 0 24 Jul 2024
OpenLLM-Ro -- Technical Report on Open-source Romanian LLMs Mihai Masala Denis C. Ilie-Ablachim D. Corlatescu Miruna Zavelca Marius Leordeanu Horia Velicu Marius Popescu Mihai Dascalu Traian Rebedea 35 2 0 13 May 2024
NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment Gerald Shen Zhilin Wang Olivier Delalleau Jiaqi Zeng Yi Dong ... Sahil Jain Ali Taghibakhshi Markel Sanz Ausin Ashwath Aithal Oleksii Kuchaiev 26 13 0 02 May 2024
Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback Vincent Conitzer Rachel Freedman J. Heitzig Wesley H. Holliday Bob M. Jacobs ... Eric Pacuit Stuart Russell Hailey Schoelkopf Emanuel Tewolde W. Zwicker 31 28 0 16 Apr 2024
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards Haoxiang Wang Yong Lin Wei Xiong Rui Yang Shizhe Diao Shuang Qiu Han Zhao Tong Zhang 40 70 0 28 Feb 2024
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 579 0 06 Apr 2023
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 218 441 0 23 Aug 2022
Offline RL for Natural Language Generation with Implicit Language Q Learning Charles Burton Snell Ilya Kostrikov Yi Su Mengjiao Yang Sergey Levine OffRL 121 101 0 05 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 205 1,654 0 15 Oct 2021
Extracting and Inferring Personal Attributes from Dialogue Zhilin Wang Xuhui Zhou Rik Koncel-Kedziorski Alex Marin Fei Xia 25 14 0 26 Sep 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
NeMo: a toolkit for building AI applications using Neural Modules Oleksii Kuchaiev Jason Chun Lok Li Huyen Nguyen Oleksii Hrinchuk Ryan Leary ... Jack Cook P. Castonguay Mariya Popova Jocelyn Huang Jonathan M. Cohen 185 291 0 14 Sep 2019