An Adaptive Placement and Parallelism Framework for Accelerating RLHF
Training

An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training

19 December 2023

Lei Liang

Papers citing "An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training"

9 / 9 papers shown

Title
Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning Nvidia A. Azzolini Hannah Brandon Prithvijit Chattopadhyay Huayu Chen ... Yao Xu X. Yang Zhuolin Yang Xiaohui Zeng Z. Zhang LM&Ro LRM AI4CE 52 5 0 18 Mar 2025
Understanding and Alleviating Memory Consumption in RLHF for LLMs Jin Zhou Hanmei Yang Steven Tang Mingcan Xiang Hui Guan Tongping Liu 29 0 0 21 Oct 2024
Efficient Training of Large Language Models on Distributed Infrastructures: A Survey Jiangfei Duan Shuo Zhang Zerui Wang Lijuan Jiang Wenwen Qu ... Dahua Lin Yonggang Wen Xin Jin Tianwei Zhang Peng Sun 71 8 0 29 Jul 2024
AntDT: A Self-Adaptive Distributed Training Framework for Leader and Straggler Nodes Youshao Xiao Lin Ju Zhenglei Zhou Siyuan Li Zhaoxin Huan ... Rujie Jiang Lin Wang Xiaolu Zhang Lei Liang Jun Zhou 19 1 0 15 Apr 2024
G-Meta: Distributed Meta Learning in GPU Clusters for Large-Scale Recommender Systems Youshao Xiao Shangchun Zhao Zhenglei Zhou Zhaoxin Huan Lin Ju Xiaolu Zhang Lin Wang Jun Zhou OffRL 36 8 0 09 Jan 2024
ZeRO++: Extremely Efficient Collective Communication for Giant Model Training Guanhua Wang Heyang Qin S. A. Jacobs Connor Holmes Samyam Rajbhandari Olatunji Ruwase Feng Yan Lei Yang Yuxiong He VLM 55 56 0 16 Jun 2023
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 225 500 0 28 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019