MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning

20 May 2024

Papers citing "MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning"

15 / 15 papers shown

Title
Endo-FASt3r: Endoscopic Foundation model Adaptation for Structure from motion Mona Sheikh Zeinoddin Mobarakol Islam Zafer Tandogdu Greg Shaw Mathew J. Clarkson E. Mazomenos Danail Stoyanov 59 0 0 10 Mar 2025
Tensor Product Attention Is All You Need Yifan Zhang Yifeng Liu Huizhuo Yuan Zhen Qin Yang Yuan Q. Gu Andrew Chi-Chih Yao 70 8 0 11 Jan 2025
CURing Large Models: Compression via CUR Decomposition Sanghyeon Park Soo-Mook Moon 38 0 0 08 Jan 2025
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 66 2 0 06 Nov 2024
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 28 1 0 12 Sep 2024
DARES: Depth Anything in Robotic Endoscopic Surgery with Self-supervised Vector-LoRA of the Foundation Model Mona Sheikh Zeinoddin Chiara Lena Jiongqi Qu Luca Carlini Mattia Magro ... E. Mazomenos Daniel C. Alexander Danail Stoyanov Matthew J. Clarkson Mobarakol Islam 24 1 0 30 Aug 2024
Rapid Biomedical Research Classification: The Pandemic PACT Advanced Categorisation Engine Omid Rohanian Mohammadmahdi Nouriborji Olena Seminog Rodrigo Furst Thomas Mendy ... Georgina Humphreys E. Antonio Adrian Bucher Alice Norton David A. Clifton 24 0 0 14 Jul 2024
Low-Rank Interconnected Adaptation Across Layers Yibo Zhong Yao Zhou OffRL MoE 36 1 0 13 Jul 2024
The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 34 10 0 12 Jun 2024
PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization Xiangdi Meng Damai Dai Weiyao Luo Zhe Yang Shaoxiang Wu Xiaochen Wang Peiyi Wang Qingxiu Dong Liang Chen Zhifang Sui 106 11 0 25 Feb 2024
DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning Wei Chen Qiushi Wang Zefei Long Xianyin Zhang Zhongtian Lu ... Siyuan Wang Jiarong Xu Xiang Bai Xuanjing Huang Zhongyu Wei 68 43 0 23 Oct 2023
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020
PubMedQA: A Dataset for Biomedical Research Question Answering Qiao Jin Bhuwan Dhingra Zhengping Liu William W. Cohen Xinghua Lu 202 791 0 13 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018