When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications

21 October 2023

Xiangyu Zhao

Papers citing "When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications"

48 / 48 papers shown

Title
Guiding LLM-based Smart Contract Generation with Finite State Machine Hao Luo Yuhao Lin Xiao Yan Xintong Hu Y. Wang Qiming Zeng Hao Wang Jiawei Jiang 14 0 0 13 May 2025
Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety Zihan Guan Mengxuan Hu Ronghang Zhu Sheng R. Li Anil Vullikanti AAML 16 0 0 11 May 2025
Bridge the Domains: Large Language Models Enhanced Cross-domain Sequential Recommendation Qidong Liu Xiangyu Zhao Yejing Wang Zijian Zhang Howard Zhong Chong Chen X. Li Wei Huang Feng Tian AI4TS 17 0 0 25 Apr 2025
ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphs Fahmida Liza Piya Rahmatollah Beheshti 34 0 0 23 Apr 2025
Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models Hongcheng Guo Juntao Yao Boyang Wang Junjia Du Shaosheng Cao Donglin Di Shun Zhang Z. Li MoE 35 0 0 10 Apr 2025
Mixture of Routers Jia-Chen Zhang Yu-Jie Xiong Xi-He Qiu Chun-Ming Xia Fei Dai MoE 52 0 0 30 Mar 2025
LLaVA-CMoE: Towards Continual Mixture of Experts for Large Vision-Language Models Hengyuan Zhao Ziqin Wang Qixin Sun Kaiyou Song Yilin Li Xiaolin Hu Qingpei Guo Si Liu KELM CLL MoE 56 0 0 27 Mar 2025
Gamma: Toward Generic Image Assessment with Mixture of Assessment Experts Hantao Zhou Rui Yang Longxiang Tang Guanyi Qin Y. Zhang R. Hu Xiu Li 39 0 0 09 Mar 2025
Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model Wenke Huang Jian Liang Xianda Guo Yiyang Fang Guancheng Wan ... Bin Yang He Li Jiawei Shao Mang Ye Bo Du OffRL LRM MLLM KELM VLM 63 1 0 06 Mar 2025
CoPL: Collaborative Preference Learning for Personalizing LLMs Youngbin Choi Seunghyuk Cho M. Lee Moonjeong Park Yesong Ko Jungseul Ok Dongwoo Kim 55 0 0 03 Mar 2025
Patient-Level Anatomy Meets Scanning-Level Physics: Personalized Federated Low-Dose CT Denoising Empowered by Large Language Model Ziyuan Yang Yingyu Chen Z. Wang Hongming Shan Yang Chen Yi Zhang 32 1 0 02 Mar 2025
R-LoRA: Random Initialization of Multi-Head LoRA for Multi-Task Learning Jinda Liu Yi-Ju Chang Yuan Wu 50 0 0 24 Feb 2025
Pastiche Novel Generation Creating: Fan Fiction You Love in Your Favorite Author's Style Xueran Han Yuhan Liu Mingzhe Li W. Liu Sen Hu Rui Yan Zhiqiang Xu Xiuying Chen 59 0 0 24 Feb 2025
Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning Peizhuang Cong Wenpu Liu Wenhan Yu Haochen Zhao Tong Yang ALM MoE 72 0 0 06 Feb 2025
Ensembles of Low-Rank Expert Adapters Yinghao Li Vianne Gao Chao Zhang MohamadAli Torkamani 57 0 0 31 Jan 2025
Large Language Model Distilling Medication Recommendation Model Qidong Liu Xian Wu Xiangyu Zhao Yuanshao Zhu Zijian Zhang Feng Tian Yefeng Zheng LM&MA 85 13 0 28 Jan 2025
Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learning Hanwen Zhong Jiaxin Chen Yutong Zhang Di Huang Yunhong Wang MoE 37 0 0 12 Jan 2025
TAPO: Task-Referenced Adaptation for Prompt Optimization Wenxin Luo W. Wang Xiaopeng Li Weibo Zhou Pengyue Jia Xiangyu Zhao 45 0 0 12 Jan 2025
A Contrastive Pretrain Model with Prompt Tuning for Multi-center Medication Recommendation Q. Liu Zhaopeng Qiu Xiangyu Zhao X. Wu Zijian Zhang Tong Bill Xu Feng Tian 23 0 0 31 Dec 2024
MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning Yufei Ma Zihan Liang Huangyu Dai B. Chen D. Gao ... Linbo Jin Wen Jiang Guannan Zhang Xiaoyan Cai Libin Yang MoE MoMe 91 1 0 10 Dec 2024
MALoRA: Mixture of Asymmetric Low-Rank Adaptation for Enhanced Multi-Task Learning Xujia Wang Haiyan Zhao Shuo Wang Hanqing Wang Zhiyuan Liu MoMe MoE 25 0 0 30 Oct 2024
Fine-Tuning LLMs for Reliable Medical Question-Answering Services Ali Anaissi Ali Braytee Junaid Akram LM&MA AI4MH 23 2 0 21 Oct 2024
Benchmarking Pathology Foundation Models: Adaptation Strategies and Scenarios Jeaung Lee Jeewoo Lim Keunho Byeon Jin Tae Kwak 15 3 0 21 Oct 2024
Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding Derong Xu Ziheng Zhang Zhihong Zhu Zhenxi Lin Q. Liu X. Wu Tong Bill Xu Xiangyu Zhao Yefeng Zheng Enhong Chen 20 2 0 21 Oct 2024
DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptation Meilu Zhu Axiu Mao Jun Liu Yixuan Yuan 29 1 0 16 Oct 2024
MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning Yaming Yang Dilxat Muhtar Yelong Shen Yuefeng Zhan Jianfeng Liu ... Denvy Deng Feng Sun Qi Zhang Weizhu Chen Yunhai Tong MoE MoMe 38 2 0 12 Oct 2024
Large Language Model Empowered Embedding Generator for Sequential Recommendation Qidong Liu Xian Wu Wanyu Wang Yejing Wang Yuanshao Zhu Xiangyu Zhao Feng Tian Yefeng Zheng 22 4 0 30 Sep 2024
Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoE Xun Zhu Ying Hu Fanbin Mo Miao Li Ji Wu 41 8 0 26 Sep 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 52 22 0 08 Jul 2024
LLM4MSR: An LLM-Enhanced Paradigm for Multi-Scenario Recommendation Yuhao Wang Yichao Wang Zichuan Fu Xiangyang Li Xiangyu Zhao Huifeng Guo Ruiming Tang 19 10 0 18 Jun 2024
LoRA-Switch: Boosting the Efficiency of Dynamic LLM Adapters via System-Algorithm Co-design Rui Kong Qiyang Li Xinyu Fang Qingtian Feng Qingfeng He Yazhu Dong Weijun Wang Yuanchun Li Linghe Kong Yunxin Liu MoE 28 4 0 28 May 2024
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts Dengchun Li Yingzi Ma Naizheng Wang Zhengmao Ye Zhiyuan Cheng ... Yan Zhang Lei Duan Jie Zuo Cal Yang Mingjie Tang MoE 24 13 0 22 Apr 2024
MING-MOE: Enhancing Medical Multi-Task Learning in Large Language Models with Sparse Mixture of Low-Rank Adapter Experts Yusheng Liao Shuyang Jiang Yu Wang Yanfeng Wang MoE 28 4 0 13 Apr 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 130 301 0 21 Mar 2024
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models Derong Xu Ziheng Zhang Zhihong Zhu Zhenxi Lin Qidong Liu ... Wanyu Wang Yuyang Ye Xiangyu Zhao Yefeng Zheng Enhong Chen KELM 22 6 0 28 Feb 2024
Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agent Xiaoyan Yu Tongxu Luo Yifan Wei Fangyu Lei Yiming Huang Peng Hao Liehuang Zhu LLMAG 25 22 0 21 Feb 2024
LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild Ziyu Zhao Leilei Gan Guoyin Wang Wangchunshu Zhou Hongxia Yang Kun Kuang Fei Wu MoMe 15 28 0 15 Feb 2024
Higher Layers Need More LoRA Experts Chongyang Gao Kezhen Chen Jinmeng Rao Baochen Sun Ruibo Liu Daiyi Peng Yawen Zhang Xiaoyuan Guo Jie Yang V. Subrahmanian MoE 11 37 0 13 Feb 2024
LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs Shaoxiang Chen Zequn Jie Lin Ma MoE 38 46 0 29 Jan 2024
A Framework to Implement 1+N Multi-task Fine-tuning Pattern in LLMs Using the CGC-LORA Algorithm Chao Song Zhihao Ye Qiqiang Lin Qiuying Peng Jun Wang 11 0 0 22 Jan 2024
Large Language Models for Generative Information Extraction: A Survey Derong Xu Wei-neng Chen Wenjun Peng Chao Zhang Tong Bill Xu Xiangyu Zhao Xian Wu Yefeng Zheng Yang Wang Enhong Chen 38 137 0 29 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 22 130 0 19 Dec 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 240 1,070 0 05 Oct 2022
Hyper-X: A Unified Hypernetwork for Multi-Task Multilingual Transfer A. Ustun Arianna Bisazza G. Bouma Gertjan van Noord Sebastian Ruder 44 32 0 24 May 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 118 98 0 24 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generation Yasuhide Miura Yuhao Zhang Emily Bao Tsai C. Langlotz Dan Jurafsky MedIm 146 152 0 20 Oct 2020