Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models

8 July 2024

Jiajun Zhang

Papers citing "Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models"

44 / 44 papers shown

Title
A Weighted Byzantine Fault Tolerance Consensus Driven Trusted Multiple Large Language Models Network Haoxiang Luo Gang Sun Yinqiu Liu Dongcheng Zhao Dusit Niyato Hongfang Yu Schahram Dustdar 31 0 0 08 May 2025
AI-powered virtual eye: perspective, challenges and opportunities Yue Wu Yibo Guo Yulong Yan Jiancheng Yang Xin Zhou Ching-Yu Cheng Danli Shi M. He 17 0 0 07 May 2025
A Trustworthy Multi-LLM Network: Challenges,Solutions, and A Use Case Haoxiang Luo Gang Sun Yinqiu Liu Dusit Niyato Hongfang Yu Mohammed Atiquzzaman Schahram Dustdar 20 0 0 06 May 2025
Improving Model Alignment Through Collective Intelligence of Open-Source LLMS Junlin Wang Roy Xie Shang Zhu Jue Wang Ben Athiwaratkun Bhuwan Dhingra S. Song Ce Zhang James Y. Zou ALM 24 0 0 05 May 2025
Position: Enough of Scaling LLMs! Lets Focus on Downscaling Ayan Sengupta Yash Goel Tanmoy Chakraborty 34 0 0 02 May 2025
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey Mari Ashiga Wei Jie Fan Wu Vardan K. Voskanyan Fateme Dinmohammadi P. Brookes Jingzhi Gong Zheng Wang 38 0 0 13 Mar 2025
Neighboring Autoregressive Modeling for Efficient Visual Generation Yefei He Yuanyu He Shaoxuan He Feng Chen Hong Zhou K. Zhang Bohan Zhuang 51 1 0 12 Mar 2025
Collective Reasoning Among LLMs A Framework for Answer Validation Without Ground Truth Seyed Pouyan Mousavi Davoudi Alireza Shafiee Fard Alireza Amiri-Margavi LRM 51 0 0 28 Feb 2025
Harnessing Multiple Large Language Models: A Survey on LLM Ensemble Zhijun Chen Jingzheng Li Pengpeng Chen Zhuoran Li Kai Sun Yuankai Luo Qianren Mao Dingqi Yang Hailong Sun Philip S. Yu ELM 47 2 0 25 Feb 2025
Optimizing Model Selection for Compound AI Systems Lingjiao Chen Jared Quincy Davis Boris Hanin Peter Bailis Matei A. Zaharia James Y. Zou Ion Stoica 40 0 0 20 Feb 2025
Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review Di Wu Xian Wei Guang Chen Hao Shen Xiangfeng Wang Wenhao Li Bo Jin 43 2 0 17 Feb 2025
Ensembles of Low-Rank Expert Adapters Yinghao Li Vianne Gao Chao Zhang MohamadAli Torkamani 55 0 0 31 Jan 2025
How to Merge Your Multimodal Models Over Time? Sebastian Dziadzio Vishaal Udandarao Karsten Roth Ameya Prabhu Zeynep Akata Samuel Albanie Matthias Bethge MoMe 82 2 0 09 Dec 2024
Enhancing Answer Reliability Through Inter-Model Consensus of Large Language Models Alireza Amiri-Margavi Iman Jebellat Ehsan Jebellat Seyed Pouyan Mousavi Davoudi 69 1 0 25 Nov 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng H. Chen MoMe 29 1 0 16 Oct 2024
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch Hasan Hammoud Umberto Michieli Fabio Pizzati Philip H. S. Torr Adel Bibi Bernard Ghanem Mete Ozay MoMe 26 14 0 20 Jun 2024
Diver: Large Language Model Decoding with Span-Level Mutual Information Verification Jinliang Lu Chen Wang Jiajun Zhang 48 2 0 04 Jun 2024
Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decoding Fanyi Qu Hao Sun Yunfang Wu 26 8 0 03 Jun 2024
Scalable Ensembling For Mitigating Reward Overoptimisation Ahmed M. Ahmed Rafael Rafailov Stepan Sharkov Xuechen Li Oluwasanmi Koyejo 24 5 0 03 Jun 2024
SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths Kaixuan Huang Xudong Guo Mengdi Wang 21 16 0 30 May 2024
Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models Zhanhui Zhou Zhixuan Liu Jie Liu Zhichen Dong Chao Yang Yu Qiao ALM 33 20 0 29 May 2024
Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routing KV Aditya Srivatsa Kaushal Kumar Maurya Ekaterina Kochmar 38 15 0 01 May 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 68 136 0 29 Apr 2024
Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing Dujian Ding Ankur Mallick Chi Wang Robert Sim Subhabrata Mukherjee Victor Rühle L. Lakshmanan Ahmed Hassan Awadallah 77 73 0 22 Apr 2024
Bridging the Gap between Different Vocabularies for LLM Ensemble Yangyifan Xu Jinliang Lu Jiajun Zhang 33 19 0 15 Apr 2024
Lossless Acceleration of Large Language Model via Adaptive N-gram Parallel Decoding Jie Ou Yueming Chen Wenhong Tian 29 10 0 10 Apr 2024
PROMPT-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression Muhammad Asif Ali Zhengping Li Shu Yang Keyuan Cheng Yang Cao Tianhao Huang Lijie Hu Lu Yu Di Wang VLM RALM 26 9 0 30 Mar 2024
DPPA: Pruning Method for Large Language Model to Model Merging Yaochen Zhu Rui Xia Jiajun Zhang MoMe 17 4 0 05 Mar 2024
Representation Surgery for Multi-Task Model Merging Enneng Yang Li Shen Zhenyi Wang Guibing Guo Xiaojun Chen Xingwei Wang Dacheng Tao MoMe 33 5 0 05 Feb 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 92 92 0 22 Jan 2024
A Survey of Resource-efficient LLM and Multimodal Foundation Models Mengwei Xu Wangsong Yin Dongqi Cai Rongjie Yi Daliang Xu ... Shangguang Wang Yuanchun Li Yunxin Liu Xin Jin Xuanzhe Liu VLM 64 70 0 16 Jan 2024
Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles Yuanzhao Zhai Han Zhang Yu Lei Yue Yu Kele Xu Dawei Feng Bo Ding Huaimin Wang AI4CE 55 31 0 30 Dec 2023
Detecting and Mitigating Hallucinations in Multilingual Summarisation Yifu Qiu Yftah Ziser Anna Korhonen E. Ponti Shay B. Cohen HILM 49 42 0 23 May 2023
How Language Model Hallucinations Can Snowball Muru Zhang Ofir Press William Merrill Alisa Liu Noah A. Smith HILM LRM 75 246 0 22 May 2023
Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources Xingxuan Li Ruochen Zhao Yew Ken Chia Bosheng Ding Shafiq R. Joty Soujanya Poria Lidong Bing HILM BDL LRM 77 85 0 22 May 2023
Varifocal Question Generation for Fact-checking N. Ousidhoum Moy Yuan Andreas Vlachos 17 24 0 22 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 237 840 0 05 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 239 313 0 11 Sep 2022
Diverse Weight Averaging for Out-of-Distribution Generalization Alexandre Ramé Matthieu Kirchmeyer Thibaud Rahier A. Rakotomamonjy Patrick Gallinari Matthieu Cord OOD 186 128 0 19 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 203 1,651 0 15 Oct 2021
Entity-level Factual Consistency of Abstractive Text Summarization Feng Nan Ramesh Nallapati Zhiguo Wang Cicero Nogueira dos Santos Henghui Zhu Dejiao Zhang Kathleen McKeown Bing Xiang HILM 136 156 0 18 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020