Online Merging Optimizers for Boosting Rewards and Mitigating Tax in
Alignment

Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment

28 May 2024

Keming Lu

Bowen Yu

Fei Huang

Yang Fan

Runji Lin

Chang Zhou

Papers citing "Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment"

11 / 11 papers shown

Title
Reinforcement Learning Enhanced LLMs: A Survey Shuhe Wang Shengyu Zhang J. Zhang Runyi Hu Xiaoya Li Tianwei Zhang Jiwei Li Fei Wu G. Wang Eduard H. Hovy OffRL 121 6 0 05 Dec 2024
From Lists to Emojis: How Format Bias Affects Model Alignment Xuanchang Zhang Wei Xiong Lichang Chen Tianyi Zhou Heng Huang Tong Zhang ALM 33 10 0 18 Sep 2024
Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Prompts Tingchen Fu Yupeng Hou Julian McAuley Rui Yan 28 3 0 09 Aug 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 27 17 0 08 Jul 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 34 3 0 11 Jun 2024
Direct Preference Optimization with an Offset Afra Amini Tim Vieira Ryan Cotterell 71 54 0 16 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 159 437 0 02 Feb 2024
Language Model Alignment with Elastic Reset Michael Noukhovitch Samuel Lavoie Florian Strub Aaron Courville KELM 87 25 0 06 Dec 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 136 177 0 13 Sep 2021
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models Cheolhyoung Lee Kyunghyun Cho Wanmo Kang MoE 225 204 0 25 Sep 2019