AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback

22 May 2023

Jimmy Ba

Tatsunori B. Hashimoto

ALM

ArXiv PDF HTML

Papers citing "AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback"

50 / 451 papers shown

Title
Data Diversity Matters for Robust Instruction Tuning Alexander Bukharin Tuo Zhao 72 35 0 21 Nov 2023
Diffusion Model Alignment Using Direct Preference Optimization Bram Wallace Meihua Dang Rafael Rafailov Linqi Zhou Aaron Lou Senthil Purushwalkam Stefano Ermon Caiming Xiong Shafiq R. Joty Nikhil Naik EGVM 16 220 0 21 Nov 2023
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2 Hamish Ivison Yizhong Wang Valentina Pyatkin Nathan Lambert Matthew E. Peters ... Joel Jang David Wadden Noah A. Smith Iz Beltagy Hanna Hajishirzi ALM ELM 22 178 0 17 Nov 2023
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM Zhilin Wang Yi Dong Jiaqi Zeng Virginia Adams Makesh Narsimhan Sreedhar ... Olivier Delalleau Jane Polak Scowcroft Neel Kant Aidan Swope Oleksii Kuchaiev 3DV 12 63 0 16 Nov 2023
Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization Yixin Liu Alexander R. Fabbri Jiawen Chen Yilun Zhao Simeng Han Shafiq R. Joty Pengfei Liu Dragomir R. Radev Chien-Sheng Wu Arman Cohan ELM 39 57 0 15 Nov 2023
Safer-Instruct: Aligning Language Models with Automated Preference Data Taiwei Shi Kai Chen Jieyu Zhao ALM SyDa 11 20 0 15 Nov 2023
Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning Shengguang Wu Keming Lu Benfeng Xu Junyang Lin Qi Su Chang Zhou SyDa ALM 23 35 0 14 Nov 2023
Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision Seongyun Lee Sue Hyun Park Yongrae Jo Minjoon Seo 20 50 0 13 Nov 2023
WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models Shangqing Tu Yuliang Sun Yushi Bai Jifan Yu Lei Hou Juanzi Li WaLM 33 7 0 13 Nov 2023
Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMs Xue-Yong Fu Md Tahmid Rahman Laskar Cheng-Hsiung Chen TN ShashiBhushan HILM ELM 65 17 0 01 Nov 2023
Vanishing Gradients in Reinforcement Finetuning of Language Models Noam Razin Hattie Zhou Omid Saremi Vimal Thilak Arwen Bradley Preetum Nakkiran Josh Susskind Etai Littwin 10 7 0 31 Oct 2023
FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models Yuxin Jiang Yufei Wang Xingshan Zeng Wanjun Zhong Liangyou Li Fei Mi Lifeng Shang Xin Jiang Qun Liu Wei Wang ALM 13 25 0 31 Oct 2023
FP8-LM: Training FP8 Large Language Models Houwen Peng Kan Wu Yixuan Wei Guoshuai Zhao Yuxiang Yang ... Zheng-Wei Zhang Shuguang Liu Joe Chau Han Hu Peng Cheng MQ 59 37 0 27 Oct 2023
Knowledge Editing for Large Language Models: A Survey Song Wang Yaochen Zhu Haochen Liu Zaiyi Zheng Chen Chen Jundong Li KELM 66 127 0 24 Oct 2023
Instruct and Extract: Instruction Tuning for On-Demand Information Extraction Yizhu Jiao Ming Zhong Sha Li Ruining Zhao Siru Ouyang Heng Ji Jiawei Han 25 23 0 24 Oct 2023
Branch-Solve-Merge Improves Large Language Model Evaluation and Generation Swarnadeep Saha Omer Levy Asli Celikyilmaz Mohit Bansal Jason Weston Xian Li MoMe 16 69 0 23 Oct 2023
Revisiting Instruction Fine-tuned Model Evaluation to Guide Industrial Applications Manuel Faysse Gautier Viaud C´eline Hudelot Pierre Colombo 22 9 0 21 Oct 2023
Tuna: Instruction Tuning using Feedback from Large Language Models Haoran Li Yiran Liu Xingxing Zhang Wei Lu Furu Wei ALM 25 3 0 20 Oct 2023
An Emulator for Fine-Tuning Large Language Models using Small Language Models Eric Mitchell Rafael Rafailov Archit Sharma Chelsea Finn Christopher D. Manning ALM 27 51 0 19 Oct 2023
Know Where to Go: Make LLM a Relevant, Responsible, and Trustworthy Searcher Xiang Shi Jiawei Liu Yinpeng Liu Qikai Cheng Wei Lu RALM HILM KELM 22 6 0 19 Oct 2023
Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging Joel Jang Seungone Kim Bill Yuchen Lin Yizhong Wang Jack Hessel Luke Zettlemoyer Hannaneh Hajishirzi Yejin Choi Prithviraj Ammanabrolu MoMe 26 130 0 17 Oct 2023
CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations Myra Cheng Tiziano Piccardi Diyi Yang LLMAG 11 66 0 17 Oct 2023
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective Ming Zhong Chenxin An Weizhu Chen Jiawei Han Pengcheng He 21 8 0 17 Oct 2023
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models Ziniu Li Tian Xu Yushun Zhang Zhihang Lin Yang Yu Ruoyu Sun Zhimin Luo 19 45 0 16 Oct 2023
Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis Kai Chen Chunwei Wang Kuo Yang Jianhua Han Lanqing Hong ... Zhenguo Li Dit-Yan Yeung Lifeng Shang Xin Jiang Qun Liu 35 32 0 16 Oct 2023
AdaLomo: Low-memory Optimization with Adaptive Learning Rate Kai Lv Hang Yan Qipeng Guo Haijun Lv Xipeng Qiu ODL 19 20 0 16 Oct 2023
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models Seungone Kim Jamin Shin Yejin Cho Joel Jang Shayne Longpre ... Sangdoo Yun Seongjin Shin Sungdong Kim James Thorne Minjoon Seo ALM LM&MA ELM 6 202 0 12 Oct 2023
Evaluating Large Language Models at Evaluating Instruction Following Zhiyuan Zeng Jiatong Yu Tianyu Gao Yu Meng Tanya Goyal Danqi Chen ELM ALM 15 159 0 11 Oct 2023
Teaching Language Models to Hallucinate Less with Synthetic Tasks Erik Jones Hamid Palangi Clarisse Simoes Varun Chandrasekaran Subhabrata Mukherjee Arindam Mitra Ahmed Hassan Awadallah Ece Kamar HILM 13 23 0 10 Oct 2023
TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models Xiao Wang Yuan Zhang Tianze Chen Songyang Gao Senjie Jin ... Rui Zheng Yicheng Zou Tao Gui Qi Zhang Xuanjing Huang ALM LRM CLL 35 18 0 10 Oct 2023
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning Mengzhou Xia Tianyu Gao Zhiyuan Zeng Danqi Chen 24 262 0 10 Oct 2023
Understanding the Effects of RLHF on LLM Generalisation and Diversity Robert Kirk Ishita Mediratta Christoforos Nalmpantis Jelena Luketina Eric Hambro Edward Grefenstette Roberta Raileanu AI4CE ALM 97 121 0 10 Oct 2023
Constructive Large Language Models Alignment with Diverse Feedback Tianshu Yu Ting-En Lin Yuchuan Wu Min Yang Fei Huang Yongbin Li ALM 30 8 0 10 Oct 2023
NEFTune: Noisy Embeddings Improve Instruction Finetuning Neel Jain Ping Yeh-Chiang Yuxin Wen John Kirchenbauer Hong-Min Chu ... Avi Schwarzschild Aniruddha Saha Micah Goldblum Jonas Geiping Tom Goldstein 20 73 0 09 Oct 2023
SALMON: Self-Alignment with Instructable Reward Models Zhiqing Sun Yikang Shen Hongxin Zhang Qinhong Zhou Zhenfang Chen David D. Cox Yiming Yang Chuang Gan ALM SyDa 11 35 0 09 Oct 2023
LAiW: A Chinese Legal Large Language Models Benchmark Yongfu Dai Duanyu Feng Jimin Huang Haochen Jia Qianqian Xie Yifang Zhang Weiguang Han Wei Tian Hao Wang ELM AILaw ALM 13 15 0 09 Oct 2023
Generative Judge for Evaluating Alignment Junlong Li Shichao Sun Weizhe Yuan Run-Ze Fan Hai Zhao Pengfei Liu ELM ALM 12 76 0 09 Oct 2023
SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF Yi Dong Zhilin Wang Makesh Narsimhan Sreedhar Xianchao Wu Oleksii Kuchaiev ALM LLMSV 26 64 0 09 Oct 2023
Do Large Language Models Know about Facts? Xuming Hu Junzhe Chen Xiaochuan Li Yufei Guo Lijie Wen Philip S. Yu Zhijiang Guo HILM KELM 18 48 0 08 Oct 2023
A Long Way to Go: Investigating Length Correlations in RLHF Prasann Singhal Tanya Goyal Jiacheng Xu Greg Durrett 34 139 0 05 Oct 2023
Predicting Emergent Abilities with Infinite Resolution Evaluation Shengding Hu Xin Liu Xu Han Xinrong Zhang Chaoqun He ... Ning Ding Zebin Ou Guoyang Zeng Zhiyuan Liu Maosong Sun ELM LRM 15 13 0 05 Oct 2023
Reward Model Ensembles Help Mitigate Overoptimization Thomas Coste Usman Anwar Robert Kirk David M. Krueger NoLa ALM 10 115 0 04 Oct 2023
SmartPlay: A Benchmark for LLMs as Intelligent Agents Yue Wu Xuan Tang Tom Michael Mitchell Yuanzhi Li ELM LLMAG 19 60 0 02 Oct 2023
BooookScore: A systematic exploration of book-length summarization in the era of LLMs Yapei Chang Kyle Lo Tanya Goyal Mohit Iyyer ALM 8 105 0 01 Oct 2023
Large Language Model Alignment: A Survey Tianhao Shen Renren Jin Yufei Huang Chuang Liu Weilong Dong Zishan Guo Xinwei Wu Yan Liu Deyi Xiong LM&MA 14 169 0 26 Sep 2023
Aligning Large Multimodal Models with Factually Augmented RLHF Zhiqing Sun Sheng Shen Shengcao Cao Haotian Liu Chunyuan Li ... Liangyan Gui Yu-xiong Wang Yiming Yang Kurt Keutzer Trevor Darrell VLM 28 307 0 25 Sep 2023
Identifying the Risks of LM Agents with an LM-Emulated Sandbox Yangjun Ruan Honghua Dong Andrew Wang Silviu Pitis Yongchao Zhou Jimmy Ba Yann Dubois Chris J. Maddison Tatsunori Hashimoto LLMAG ELM 8 94 0 25 Sep 2023
EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria Tae Soo Kim Yoonjoo Lee Jamin Shin Young-Ho Kim Juho Kim 21 40 0 24 Sep 2023
Calibrating LLM-Based Evaluator Yuxuan Liu Tianchi Yang Shaohan Huang Zihan Zhang Haizhen Huang Furu Wei Weiwei Deng Feng Sun Qi Zhang 33 31 0 23 Sep 2023
AceGPT, Localizing Large Language Models in Arabic Huang Huang Fei Yu Jianqing Zhu Xuening Sun Hao Cheng ... Lian Zhang Ruoyu Sun Xiang Wan Haizhou Li Jinchao Xu 14 48 0 21 Sep 2023