LoRA Dropout as a Sparsity Regularizer for Overfitting Control

15 April 2024

Papers citing "LoRA Dropout as a Sparsity Regularizer for Overfitting Control"

24 / 24 papers shown

Title
Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning Yupei Li M. Milling Björn Schuller AI4CE 102 0 0 27 Mar 2025
A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models Mengyang Sun Yihao Wang Tao Feng Dan Zhang Yifan Zhu J. Tang MoE 29 0 0 20 Feb 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection Antoine Simoulin Namyong Park Xiaoyi Liu Grey Yang 110 0 0 31 Jan 2025
ABXI: Invariant Interest Adaptation for Task-Guided Cross-Domain Sequential Recommendation Qingtian Bian Marcus Vinícius de Carvalho Tieying Li Jiaxing Xu Hui Fang Yiping Ke 34 0 0 25 Jan 2025
MoR: Mixture of Ranks for Low-Rank Adaptation Tuning Chuanyu Tang Yilong Chen Zhenyu Zhang Junyuan Shang Wenyuan Zhang Yong Huang Tingwen Liu MoE 21 0 0 17 Oct 2024
DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptation Meilu Zhu Axiu Mao Jun Liu Yixuan Yuan 32 1 0 16 Oct 2024
Aligning (Medical) LLMs for (Counterfactual) Fairness Raphael Poulain Hamed Fayyaz Rahmatollah Beheshti 21 3 0 22 Aug 2024
Flexora: Flexible Low Rank Adaptation for Large Language Models Chenxing Wei Yao Shu Ying Tiffany He Fei Richard Yu AI4CE 27 3 0 20 Aug 2024
Crafting Efficient Fine-Tuning Strategies for Large Language Models Michael Oliver Guan Wang 22 1 0 18 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 52 22 0 08 Jul 2024
Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models Zihan Wang Deli Chen Damai Dai Runxin Xu Zhuoshu Li Y. Wu MoE ALM 25 2 0 02 Jul 2024
Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning Siwei Li Yifan Yang Yifei Shen Fangyun Wei Zongqing Lu L. Qiu Yuqing Yang AI4CE 30 1 0 01 Jul 2024
Mixture-of-Subspaces in Low-Rank Adaptation Taiqiang Wu Jiahao Wang Zhe Zhao Ngai Wong 39 19 0 16 Jun 2024
Vertical LoRA: Dense Expectation-Maximization Interpretation of Transformers Zhuolin Fu 32 0 0 13 Jun 2024
FouRA: Fourier Low Rank Adaptation Shubhankar Borse Shreya Kadambi N. Pandey Kartikeya Bhardwaj Viswanath Ganapathy Sweta Priyadarshi Risheek Garrepalli Rafael Esteves Munawar Hayat Fatih Porikli 26 7 0 13 Jun 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 38 6 0 05 Apr 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 130 301 0 21 Mar 2024
On the Calibration of Large Language Models and Alignment Chiwei Zhu Benfeng Xu Quan Wang Yongdong Zhang Zhendong Mao 61 32 0 22 Nov 2023
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis Yuxin Xiao Paul Pu Liang Umang Bhatt W. Neiswanger Ruslan Salakhutdinov Louis-Philippe Morency 167 86 0 10 Oct 2022
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 136 177 0 13 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 247 9,042 0 06 Jun 2015
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 243 7,597 0 03 Jul 2012