AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback

22 May 2023

Jimmy Ba

Tatsunori B. Hashimoto

ALM

ArXiv PDF HTML

Papers citing "AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback"

50 / 451 papers shown

Title
Faster WIND: Accelerating Iterative Best-of- $N$ Distillation for LLM Alignment Tong Yang Jincheng Mei H. Dai Zixin Wen Shicong Cen Dale Schuurmans Yuejie Chi Bo Dai 36 4 0 20 Feb 2025
An Empirical Analysis of Uncertainty in Large Language Model Evaluations Qiujie Xie Qingqiu Li Zhuohao Yu Yuejie Zhang Yue Zhang Linyi Yang ELM 58 1 0 15 Feb 2025
Economics of Sourcing Human Data Sebastin Santy Prasanta Bhattacharya Manoel Horta Ribeiro Kelsey Allen Sewoong Oh 69 0 0 11 Feb 2025
AI Alignment at Your Discretion Maarten Buyl Hadi Khalaf C. M. Verdun Lucas Monteiro Paes Caio Vieira Machado Flavio du Pin Calmon 33 0 0 10 Feb 2025
Toward Copyright Integrity and Verifiability via Multi-Bit Watermarking for Intelligent Transportation Systems Yihao Wang Lingxiao Li Yifan Tang Ru Zhang Jianyi Liu 35 1 0 08 Feb 2025
ARIES: Stimulating Self-Refinement of Large Language Models by Iterative Preference Optimization Yongcheng Zeng Xinyu Cui Xuanfa Jin Guoqing Liu Zexu Sun ... Dong Li Ning Yang Jianye Hao H. Zhang J. Wang LRM LLMAG 76 1 0 08 Feb 2025
Generative Psycho-Lexical Approach for Constructing Value Systems in Large Language Models Haoran Ye T. Zhang Yuhang Xie Liyuan Zhang Yuanyi Ren Xin Zhang Guojie Song PILM 74 0 0 04 Feb 2025
The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking Yuchun Miao Sen Zhang Liang Ding Yuqi Zhang L. Zhang Dacheng Tao 81 3 0 31 Jan 2025
Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment Chaoqi Wang Zhuokai Zhao Yibo Jiang Zhaorun Chen Chen Zhu ... Jiayi Liu Lizhu Zhang Xiangjun Fan Hao Ma Sinong Wang 70 3 0 17 Jan 2025
Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedback Lester James Validad Miranda Yizhong Wang Yanai Elazar Sachin Kumar Valentina Pyatkin Faeze Brahman Noah A. Smith Hannaneh Hajishirzi Pradeep Dasigi 45 8 0 08 Jan 2025
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation Yuhui Zhang Yuchang Su Yiming Liu Xiaohan Wang James Burgess ... Josiah Aklilu Alejandro Lozano Anjiang Wei Ludwig Schmidt Serena Yeung-Levy 50 3 0 06 Jan 2025
A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls Sheikh Shafayat Dongkeun Yoon Woori Jang Jiwoo Choi Alice H. Oh Seohyon Jung 91 1 0 03 Jan 2025
PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations Ruosen Li Teerth Patel Xinya Du LLMAG ALM 52 94 0 03 Jan 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 85 11 0 31 Dec 2024
Hansel: Output Length Controlling Framework for Large Language Models Seoha Song Junhyun Lee Hyeonmok Ko 70 0 0 18 Dec 2024
Smaller Language Models Are Better Instruction Evolvers Tingfeng Hui Lulu Zhao Guanting Dong Yaqi Zhang Hua Zhou Sen Su ALM 79 1 0 15 Dec 2024
Self-Generated Critiques Boost Reward Modeling for Language Models Yue Yu Zhengxing Chen Aston Zhang L Tan Chenguang Zhu ... Suchin Gururangan Chao-Yue Zhang Melanie Kambadur Dhruv Mahajan Rui Hou LRM ALM 87 14 0 25 Nov 2024
A dataset of questions on decision-theoretic reasoning in Newcomb-like problems Caspar Oesterheld Emery Cooper Miles Kodama Linh Chi Nguyen Ethan Perez 29 1 0 15 Nov 2024
LLM-Powered AI Tutors with Personas for d/Deaf and Hard-of-Hearing Online Learners Haocong Cheng Si Chen Christopher Perdriau Yun Huang 50 1 0 15 Nov 2024
Bayesian Calibration of Win Rate Estimation with LLM Evaluators Yicheng Gao G. Xu Zhe Wang Arman Cohan 31 6 0 07 Nov 2024
M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models Chuhan Li Ziyao Shangguan Yilun Zhao Deyuan Li Y. Liu Arman Cohan 22 0 0 06 Nov 2024
AI Metropolis: Scaling Large Language Model-based Multi-Agent Simulation with Out-of-order Execution Zhiqiang Xie Hao Kang Ying Sheng Tushar Krishna Kayvon Fatahalian Christos Kozyrakis LRM AI4CE LLMAG LM&Ro 35 1 0 05 Nov 2024
Defense Against Prompt Injection Attack by Leveraging Attack Techniques Yulin Chen Haoran Li Zihao Zheng Y. Song Dekai Wu Bryan Hooi SILM AAML 47 4 0 01 Nov 2024
Leveraging Language Models and Bandit Algorithms to Drive Adoption of Battery-Electric Vehicles Keiichi Namikoshi David A. Shamma Rumen Iliev Jingchao Fang Alexandre L. S. Filipowicz Candice L Hogan Charlene C. Wu Nikos Aréchiga 23 0 0 30 Oct 2024
VPO: Leveraging the Number of Votes in Preference Optimization Jae Hyeon Cho Minkyung Park Byung-Jun Lee 22 0 0 30 Oct 2024
A Theoretical Perspective for Speculative Decoding Algorithm Ming Yin Minshuo Chen Kaixuan Huang Mengdi Wang 32 0 0 30 Oct 2024
Project MPG: towards a generalized performance benchmark for LLM capabilities Lucas Spangher Tianle Li William Arnold Nick Masiewicki Xerxes Dotiwalla Rama Parusmathi Peter Grabowski Eugene Ie Dan Gruhl 36 0 0 28 Oct 2024
LongReward: Improving Long-context Large Language Models with AI Feedback J. Zhang Zhongni Hou Xin Lv S. Cao Zhenyu Hou Yilin Niu Lei Hou Yuxiao Dong Ling Feng Juanzi Li OffRL LRM 30 7 0 28 Oct 2024
Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation Dongryeol Lee Yerin Hwang Yongil Kim Joonsuk Park Kyomin Jung ELM 68 5 0 28 Oct 2024
Fast Best-of-N Decoding via Speculative Rejection Hanshi Sun Momin Haider Ruiqi Zhang Huitao Yang Jiahao Qiu Ming Yin Mengdi Wang Peter L. Bartlett Andrea Zanette BDL 40 26 0 26 Oct 2024
Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks Annalisa Szymanski Noah Ziems Heather A. Eicher-Miller T. Li Meng-Long Jiang Ronald A Metoyer ALM ELM 39 19 0 26 Oct 2024
RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction Tanqiu Jiang Zian Wang Jiacheng Liang Changjiang Li Yuhui Wang Ting Wang AAML 29 3 0 25 Oct 2024
ChunkRAG: Novel LLM-Chunk Filtering Method for RAG Systems Ishneet Sukhvinder Singh Ritvik Aggarwal Ibrahim Allahverdiyev Muhammad Taha Aslihan Akalin Kevin Zhu Sean O'Brien 23 8 0 25 Oct 2024
Decoding Time Series with LLMs: A Multi-Agent Framework for Cross-Domain Annotation M. Lin Z. Chen Yanchi Liu Xujiang Zhao Zongyu Wu Junxiang Wang Xiang Zhang Suhang Wang Haifeng Chen AI4TS 23 7 0 22 Oct 2024
CompassJudger-1: All-in-one Judge Model Helps Model Evaluation and Evolution Maosong Cao Alexander Lam Haodong Duan Hongwei Liu S. Zhang Kai Chen AILaw ELM 37 11 0 21 Oct 2024
Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototyping Ryan Li Yanzhe Zhang Diyi Yang 3DV 16 4 0 21 Oct 2024
Self-Explained Keywords Empower Large Language Models for Code Generation Lishui Fan Mouxiang Chen Zhongxin Liu 38 1 0 21 Oct 2024
LlamaLens: Specialized Multilingual LLM for Analyzing News and Social Media Content Mohamed Bayan Kmainasi Ali Ezzat Shahroor Maram Hasanain Sahinur Rahman Laskar Naeemul Hassan Firoj Alam 31 1 0 20 Oct 2024
How to Evaluate Reward Models for RLHF Evan Frick Tianle Li Connor Chen Wei-Lin Chiang Anastasios Nikolas Angelopoulos Jiantao Jiao Banghua Zhu Joseph E. Gonzalez Ion Stoica ALM OffRL 22 9 0 18 Oct 2024
Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens Zhepeng Cen Yao Liu Siliang Zeng Pratik Chaudhar Huzefa Rangwala George Karypis Rasool Fakoor SyDa AIFin 18 3 0 18 Oct 2024
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling Jiahao Qiu Yifu Lu Yifan Zeng Jiacheng Guo Jiayi Geng Huazheng Wang Kaixuan Huang Yue Wu Mengdi Wang 34 22 0 18 Oct 2024
Anchored Alignment for Self-Explanations Enhancement Luis Felipe Villa-Arenas Ata Nizamoglu Qianli Wang Sebastian Möller Vera Schmitt 19 0 0 17 Oct 2024
A Little Human Data Goes A Long Way Dhananjay Ashok Jonathan May SyDa 39 2 0 17 Oct 2024
MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedback Zonghai Yao Aditya Parashar Huixue Zhou Won Seok Jang Feiyun Ouyang Zhichao Yang Hong-ye Yu ELM 37 2 0 17 Oct 2024
Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data Florian E. Dorner Vivian Y. Nastl Moritz Hardt ELM ALM 33 5 0 17 Oct 2024
POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization Batuhan K. Karaman Ishmam Zabir Alon Benhaim Vishrav Chaudhary M. Sabuncu Xia Song AI4CE 32 0 0 16 Oct 2024
CREAM: Consistency Regularized Self-Rewarding Language Models Z. Wang Weilei He Zhiyuan Liang Xuchao Zhang Chetan Bansal Ying Wei Weitong Zhang Huaxiu Yao ALM 96 7 0 16 Oct 2024
JudgeBench: A Benchmark for Evaluating LLM-based Judges Sijun Tan Siyuan Zhuang Kyle Montgomery William Y. Tang Alejandro Cuadron Chenguang Wang Raluca A. Popa Ion Stoica ELM ALM 49 36 0 16 Oct 2024
Preference Optimization with Multi-Sample Comparisons Chaoqi Wang Zhuokai Zhao Chen Zhu Karthik Abinav Sankararaman Michal Valko ... Zhaorun Chen Madian Khabsa Yuxin Chen Hao Ma Sinong Wang 59 10 0 16 Oct 2024
Understanding Likelihood Over-optimisation in Direct Alignment Algorithms Zhengyan Shi Sander Land Acyr F. Locatelli Matthieu Geist Max Bartolo 46 4 0 15 Oct 2024