A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA

28 November 2023

Papers citing "A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA"

50 / 63 papers shown

Title
Memorization and Knowledge Injection in Gated LLMs Xu Pan Ely Hahami Zechen Zhang H. Sompolinsky KELM CLL RALM 104 0 0 30 Apr 2025
X-Cross: Dynamic Integration of Language Models for Cross-Domain Sequential Recommendation Guy Hadad Haggai Roitman Yotam Eshel Bracha Shapira L. Rokach BDL VLM LRM 39 0 0 29 Apr 2025
Embodied Image Captioning: Self-supervised Learning Agents for Spatially Coherent Image Descriptions Tommaso Galliena Tommaso Apicella Stefano Rosa Pietro Morerio Alessio Del Bue Lorenzo Natale 32 0 0 11 Apr 2025
Leaking LoRa: An Evaluation of Password Leaks and Knowledge Storage in Large Language Models Ryan Marinelli Magnus Eckhoff PILM 52 0 0 29 Mar 2025
Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities Raman Dutt Harleen Hanspal Guoxuan Xia Petru-Daniel Tudosiu Alexander Black Yongxin Yang Steven G. McDonagh Sarah Parisot MoE 38 0 0 28 Mar 2025
Hiding Images in Diffusion Models by Editing Learned Score Functions Haoyu Chen Yunqiao Yang Nan Zhong Kede Ma DiffM 63 0 0 24 Mar 2025
A Foundational individual Mobility Prediction Model based on Open-Source Large Language Models Zhenlin Qin Leizhen Wang Francisco Camara Pereira Zhenlinag Ma 32 0 0 19 Mar 2025
KV-Distill: Nearly Lossless Learnable Context Compression for LLMs Vivek Chari Guanghui Qin Benjamin Van Durme VLM 61 0 0 13 Mar 2025
RigoChat 2: an adapted language model to Spanish using a bounded dataset and reduced hardware Gonzalo Santamaría Gómez Guillem García Subies Pablo Gutiérrez Ruiz Mario González Valero Natàlia Fuertes ... Nuria Aldama García David Betancur Sánchez Kateryna Sushkova Marta Guerrero Nieto Á. Jiménez 51 0 0 11 Mar 2025
Adapting Large Language Models for Parameter-Efficient Log Anomaly Detection Ying Fu Lim Jiawen Zhu Guansong Pang 34 0 0 11 Mar 2025
SwiLTra-Bench: The Swiss Legal Translation Benchmark Joel Niklaus Jakob Merane Luka Nenadic Sina Ahmadi Yingqiang Gao ... Matthew Guillod Robin Mamié Daniel Brunner Julio Pereyra Niko Grupen AILaw ELM 74 0 0 03 Mar 2025
Starjob: Dataset for LLM-Driven Job Shop Scheduling Henrik Abgaryan Tristan Cazenave Ararat Harutyunyan 31 0 0 26 Feb 2025
Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models Cao Yuxuan Wu Jiayang Alistair Cheong Liang Chuen Bryan Shan Guanrong Theodore Lee Chong Jen Sherman Chann Zhi Shen 157 0 0 25 Feb 2025
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment Chenghao Fan Zhenyi Lu Sichen Liu Xiaoye Qu Wei Wei Chengfeng Gu Yu-Xi Cheng MoE 45 0 0 24 Feb 2025
Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs Jan Betley Daniel Tan Niels Warncke Anna Sztyber-Betley Xuchan Bao Martín Soto Nathan Labenz Owain Evans AAML 73 8 0 24 Feb 2025
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning Raghav Singhal Kaustubh Ponkshe Rohit Vartak Lav R. Varshney Praneeth Vepakomma FedML 71 0 0 24 Feb 2025
Towards Conditioning Clinical Text Generation for User Control Osman Alperen Koras Rabi Bahnan Jens Kleesiek Amin Dada 28 0 0 24 Feb 2025
Instruction-Tuning LLMs for Event Extraction with Annotation Guidelines Saurabh Srivastava Sweta Pati Ziyu Yao 36 0 0 22 Feb 2025
A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models Mengyang Sun Yihao Wang Tao Feng Dan Zhang Yifan Zhu J. Tang MoE 29 0 0 20 Feb 2025
GoRA: Gradient-driven Adaptive Low Rank Adaptation Haonan He Peng Ye Yuchen Ren Yuan Yuan Lei Chen AI4TS AI4CE 70 0 0 13 Feb 2025
MADation: Face Morphing Attack Detection with Foundation Models Eduarda Caldeira Guray Ozgur Tahar Chettaoui Marija Ivanovska Peter Peer Fadi Boutros Vitomir Štruc Naser Damer CVBM 29 0 1 28 Jan 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 33 4 0 28 Jan 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 37 6 0 23 Jan 2025
How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization Shezheng Song Hao Xu Jun Ma Shasha Li Long Peng Qian Wan Xiaodong Liu Jie Yu KELM CLL ALM 65 0 0 23 Jan 2025
A Domain Adaptation Framework for Speech Recognition Systems with Only Synthetic data Minh Tran Yutong Pang Debjyoti Paul Laxmi Pandey Kevin Jiang Jinxi Guo Ke Li Shun Zhang X. Zhang Xin Lei AI4CE 28 0 0 21 Jan 2025
EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition Hamid Nasiri Peter Garraghan 36 1 0 21 Jan 2025
Aggregating Low Rank Adapters in Federated Fine-tuning Evelyn Trautmann Ian Hales Martin F. Volk AI4CE FedML 34 0 0 10 Jan 2025
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation Jun Liu Zhenglun Kong Peiyan Dong Changdi Yang Xuan Shen ... Wei Niu Wenbin Zhang Xue Lin Dong Huang Yanzhi Wang ALM 34 2 0 08 Jan 2025
FoundPAD: Foundation Models Reloaded for Face Presentation Attack Detection Guray Ozgur Eduarda Caldeira Tahar Chettaoui Fadi Boutros Raghavendra Ramachandra Naser Damer AAML CVBM 29 0 1 06 Jan 2025
DoTA: Weight-Decomposed Tensor Adaptation for Large Language Models Xiaolin Hu Xiang Cheng Peiyu Liu Wei Liu Jian Luan Bin Wang Y. Liu 40 1 0 31 Dec 2024
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning Kaustubh Ponkshe Raghav Singhal Eduard A. Gorbunov Alexey Tumanov Samuel Horváth Praneeth Vepakomma 63 1 0 29 Nov 2024
FRoundation: Are Foundation Models Ready for Face Recognition? Tahar Chettaoui Naser Damer Fadi Boutros CVBM 31 4 0 31 Oct 2024
LoRA vs Full Fine-tuning: An Illusion of Equivalence Reece Shuttleworth Jacob Andreas Antonio Torralba Pratyusha Sharma 27 8 0 28 Oct 2024
Improving Multimodal Large Language Models Using Continual Learning Shikhar Srivastava Md Yousuf Harun Robik Shrestha Christopher Kanan KELM VLM CLL 28 1 0 25 Oct 2024
Fine-Tuning LLMs for Reliable Medical Question-Answering Services Ali Anaissi Ali Braytee Junaid Akram LM&MA AI4MH 23 2 0 21 Oct 2024
Language Models are Symbolic Learners in Arithmetic Chunyuan Deng Zhiqi Li Roy Xie Ruidi Chang Hanjie Chen 27 3 0 21 Oct 2024
LangGFM: A Large Language Model Alone Can be a Powerful Graph Foundation Model Tianqianjin Lin Pengwei Yan Kaisong Song Zhuoren Jiang Yangyang Kang Jun Lin Weikang Yuan Junjie Cao Changlong Sun Xiaozhong Liu AI4CE 30 0 0 19 Oct 2024
A Lightweight Multi Aspect Controlled Text Generation Solution For Large Language Models Chenyang Zhang Jiayi Lin Haibo Tong Bingxuan Hou Dongyu Zhang Jialin Li Junli Wang 26 0 0 18 Oct 2024
VividMed: Vision Language Model with Versatile Visual Grounding for Medicine Lingxiao Luo Bingda Tang Xuanzhong Chen Rong Han Ting Chen VLM 21 2 0 16 Oct 2024
One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation Fabian Paischer Lukas Hauzenberger Thomas Schmied Benedikt Alkin Marc Peter Deisenroth Sepp Hochreiter 29 4 0 09 Oct 2024
HyperINF: Unleashing the HyperPower of the Schulz's Method for Data Influence Estimation Xinyu Zhou Simin Fan Martin Jaggi TDI 17 0 0 07 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 29 0 0 05 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 31 8 0 02 Oct 2024
Scrambled text: training Language Models to correct OCR errors using synthetic data Jonathan Bourne SyDa 34 2 0 29 Sep 2024
Co-occurrence is not Factual Association in Language Models Xiao Zhang Miao Li Ji Wu KELM 59 2 0 21 Sep 2024
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists Dongyang Fan Bettina Messmer N. Doikov Martin Jaggi MoMe MoE 37 1 0 20 Sep 2024
Eir: Thai Medical Large Language Models Yutthakorn Thiprak Rungtam Ngodngamthaweesuk Songtam Ngodngamtaweesuk LM&MA ELM 33 0 0 13 Sep 2024
SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning Yuze Zhao Jintao Huang Jinghan Hu Xingjun Wang Yunlin Mao ... Zhikai Wu Baole Ai Ang Wang Wenmeng Zhou Yingda Chen 29 3 0 10 Aug 2024
Examining Gender and Power on Wikipedia Through Face and Politeness Adil Soubki Shyne Choi Owen Rambow CVBM 23 0 0 05 Aug 2024
LoRA-Pro: Are Low-Rank Adapters Properly Optimized? Zhengbo Wang Jian Liang Ran He Zilei Wang Tieniu Tan 41 15 0 25 Jul 2024