Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models

15 November 2023

Keming Lu

Hongyi Yuan

Runji Lin

Junyang Lin

Zheng Yuan

Chang Zhou

Jingren Zhou

MoE

LRM

ArXiv PDF HTML

Papers citing "Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models"

41 / 41 papers shown

Title
Position: Enough of Scaling LLMs! Lets Focus on Downscaling Ayan Sengupta Yash Goel Tanmoy Chakraborty 34 0 0 02 May 2025
Can a Crow Hatch a Falcon? Lineage Matters in Predicting Large Language Model Performance Takuya Tamura Taro Yano Masafumi Enomoto M. Oyamada 39 0 0 28 Apr 2025
Ensemble Bayesian Inference: Leveraging Small Language Models to Achieve LLM-level Accuracy in Profile Matching Tasks Haru-Tada Sato Fuka Matsuzaki Jun-ichiro Takahashi UQCV 44 0 0 24 Apr 2025
Resource-efficient Inference with Foundation Model Programs Lunyiu Nie Zhimin Ding Kevin Yu Marco Cheung C. Jermaine S. Chaudhuri 22 0 0 09 Apr 2025
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 43 0 0 09 Apr 2025
Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute Jianhao Chen Zishuo Xun Bocheng Zhou Han Qi Qiaosheng Zhang ... Wei Hu Yuzhong Qu W. Ouyang Wanli Ouyang Shuyue Hu 74 0 0 01 Apr 2025
EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routing Yizhang Zhu Runzhi Jiang Boyan Li Nan Tang Yuyu Luo 34 0 0 28 Mar 2025
LeForecast: Enterprise Hybrid Forecast by Time Series Intelligence Zheng Tan Yiwen Nie Wenfa Wu Guanyu Zhang Yanze Liu ... Chao Yang Jiaxuan Fan Yuan He Hongsheng Qi Yangzhou Du AI4TS 34 0 0 27 Mar 2025
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey Mari Ashiga Wei Jie Fan Wu Vardan K. Voskanyan Fateme Dinmohammadi P. Brookes Jingzhi Gong Zheng Wang 38 0 0 13 Mar 2025
Collaborative Speculative Inference for Efficient LLM Inference Serving Luyao Gao Jianchun Liu Hongli Xu Liusheng Huang 38 0 0 13 Mar 2025
Life-Cycle Routing Vulnerabilities of LLM Router Qiqi Lin Xiaoyang Ji Shengfang Zhai Qingni Shen Zhi-Li Zhang Yuejian Fang Yansong Gao AAML 52 0 0 09 Mar 2025
Multi-LLM Collaborative Search for Complex Problem Solving Sen Yang Yafu Li Wai Lam Yu Cheng LLMAG LRM 66 1 0 26 Feb 2025
Harnessing Multiple Large Language Models: A Survey on LLM Ensemble Zhijun Chen Jingzheng Li Pengpeng Chen Zhuoran Li Kai Sun Yuankai Luo Qianren Mao Dingqi Yang Hailong Sun Philip S. Yu ELM 50 2 0 25 Feb 2025
A Unified Approach to Routing and Cascading for LLMs Jasper Dekoninck Maximilian Baader Martin Vechev 60 2 0 17 Feb 2025
Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial? Wenzhe Li Yong Lin Mengzhou Xia Chi Jin MoE 74 2 0 02 Feb 2025
Smoothie: Label Free Language Model Routing Neel Guha Mayee F. Chen Trevor Chow Ishan S. Khare Christopher Ré 71 3 0 06 Dec 2024
LLM Chain Ensembles for Scalable and Accurate Data Annotation David Farr Nico Manzonelli Iain Cruickshank Kate Starbird Jevin West 24 3 0 16 Oct 2024
Plug-and-Play Performance Estimation for LLM Services without Relying on Labeled Data Can Wang Dianbo Sui Hongliang Sun Hao Ding Bolin Zhang Zhiying Tu 19 0 0 10 Oct 2024
Glider: Global and Local Instruction-Driven Expert Router Pingzhi Li Prateek Yadav Jaehong Yoon Jie Peng Yi-Lin Sung Mohit Bansal Tianlong Chen MoMe MoE 25 1 0 09 Oct 2024
EmbedLLM: Learning Compact Representations of Large Language Models Richard Zhuang Tianhao Wu Zhaojin Wen Andrew Li Jiantao Jiao Kannan Ramchandran AIFin 29 1 0 03 Oct 2024
Efficiently Deploying LLMs with Controlled Risk Michael J. Zellinger Matt Thomson 36 1 0 03 Oct 2024
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling Yuxuan Yao Han Wu Mingyang Liu Sichun Luo Xiongwei Han Jie Liu Zhijiang Guo Linqi Song 47 4 0 03 Oct 2024
RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models Shuhao Chen Weisen Jiang Baijiong Lin James T. Kwok Yu Zhang RALM MQ 40 5 0 30 Sep 2024
Hit the Sweet Spot! Span-Level Ensemble for Large Language Models Yangyifan Xu Jianghao Chen Junhong Wu Jiajun Zhang MoE 17 2 0 27 Sep 2024
Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models Yuncheng Yang Yulei Qin Tong Wu Zihan Xu Gang Li ... Yuchen Shi Ke Li Xing Sun Jie Yang Yun Gu ALM OffRL MoE 46 0 0 28 Aug 2024
A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model Jingwen Zhou Qinghua Lu Jieshan Chen Liming Zhu Xiwei Xu Zhenchang Xing Stefan Harrer 36 0 0 06 Aug 2024
MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs Quang H. Nguyen Duy C. Hoang Juliette Decugis Saurav Manchanda Nitesh V. Chawla Khoa D. Doan Khoa D. Doan 37 6 0 15 Jul 2024
RouteLLM: Learning to Route LLMs with Preference Data Isaac Ong Amjad Almahairi Vincent Wu Wei-Lin Chiang Tianhao Wu Joseph E. Gonzalez M. W. Kadous Ion Stoica 67 66 0 26 Jun 2024
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models Jing Liu Ruihao Gong Mingyang Zhang Yefei He Jianfei Cai Bohan Zhuang MoE 37 0 0 13 Jun 2024
Mixture-of-Agents Enhances Large Language Model Capabilities Junlin Wang Jue Wang Ben Athiwaratkun Ce Zhang James Zou LLMAG AIFin 36 94 0 07 Jun 2024
Cascade-Aware Training of Language Models Congchao Wang Sean Augenstein Keith Rush Wittawat Jitkrittum Harikrishna Narasimhan A. S. Rawat A. Menon Alec Go 22 4 0 29 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su E. Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Page-Caccia Alessandro Sordoni MoMe 27 30 0 18 May 2024
SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts R. Prabhakar R. Sivaramakrishnan Darshan Gandhi Yun Du Mingran Wang ... Urmish Thakker Dawei Huang Sumti Jairath Kevin J. Brown K. Olukotun MoE 36 12 0 13 May 2024
Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selection Guillem Ramírez Alexandra Birch Ivan Titov 38 8 0 03 May 2024
Ensemble Learning for Heterogeneous Large Language Models with Deep Parallel Collaboration Yi-Chong Huang Xiaocheng Feng Baohang Li Yang Xiang Hui Wang Bing Qin Ting Liu FedML 28 22 0 19 Apr 2024
Post-Hoc Reversal: Are We Selecting Models Prematurely? Rishabh Ranjan Saurabh Garg Mrigank Raman Carlos Guestrin Zachary Chase Lipton 19 0 0 11 Apr 2024
RouterBench: A Benchmark for Multi-LLM Routing System Qitian Jason Hu Jacob Bieker Xiuyu Li Nan Jiang Benjamin Keigwin Gaurav Ranganath Kurt Keutzer Shriyash Kaustubh Upadhyay 42 36 0 18 Mar 2024
DAM: Dynamic Adapter Merging for Continual Video QA Learning Feng Cheng Ziyang Wang Yi-Lin Sung Yan-Bo Lin Mohit Bansal Gedas Bertasius CLL MoMe 26 10 0 13 Mar 2024
Learning to Route Among Specialized Experts for Zero-Shot Generalization Mohammed Muqeeth Haokun Liu Yufan Liu Colin Raffel MoMe 25 14 0 08 Feb 2024
More Agents Is All You Need Junyou Li Qin Zhang Yangbin Yu Qiang Fu Deheng Ye LLMAG 133 57 0 03 Feb 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022