Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by
Self-Contrast

Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast

23 May 2024

Yujiu Yang

Papers citing "Unchosen Experts Can Contribute Too: Unleashing MoE Models' Power by Self-Contrast"

13 / 13 papers shown

Title
A Survey on the Honesty of Large Language Models Siheng Li Cheng Yang Taiqiang Wu Chufan Shi Yuji Zhang ... Jie Zhou Yujiu Yang Ngai Wong Xixin Wu Wai Lam HILM 27 4 0 27 Sep 2024
PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQL Ruilin Luo Liyuan Wang Binghuai Lin Zicheng Lin Yujiu Yang LRM 25 5 0 21 Sep 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 27 17 0 08 Jul 2024
A Closer Look into Mixture-of-Experts in Large Language Models Ka Man Lo Zeyu Huang Zihan Qiu Zili Wang Jie Fu MoE 18 9 0 26 Jun 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 41 19 0 16 Jun 2024
Harder Tasks Need More Experts: Dynamic Routing in MoE Models Quzhe Huang Zhenwei An Zhuang Nan Mingxu Tao Chen Zhang ... Kun Xu Kun Xu Liwei Chen Songfang Huang Yansong Feng MoE 35 25 0 12 Mar 2024
Towards an empirical understanding of MoE design choices Dongyang Fan Bettina Messmer Martin Jaggi 26 10 0 20 Feb 2024
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism DeepSeek-AI Xiao Bi : Xiao Bi Deli Chen Guanting Chen ... Yao Zhao Shangyan Zhou Shunfeng Zhou Qihao Zhu Yuheng Zou LRM ALM 139 298 0 05 Jan 2024
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 147 323 0 18 Feb 2022
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 245 671 0 06 Jan 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 223 4,424 0 23 Jan 2020