Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

11 May 2022

Papers citing "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

50 / 93 papers shown

Title
ReCIT: Reconstructing Full Private Data from Gradient in Parameter-Efficient Fine-Tuning of Large Language Models Jin Xie Ruishi He Songze Li Xiaojun Jia Shouling Ji SILM AAML 63 0 0 29 Apr 2025
Can Differentially Private Fine-tuning LLMs Protect Against Privacy Attacks? Hao Du Shang Liu Yang Cao AAML 40 0 0 28 Apr 2025
Re-Imagining Multimodal Instruction Tuning: A Representation View Yiyang Liu James Liang Ruixiang Tang Yugyung Lee Majid Rabbani ... Raghuveer M. Rao Lifu Huang Dongfang Liu Qifan Wang Cheng Han 34 0 0 02 Mar 2025
Parameter Efficient Merging for Multimodal Large Language Models with Complementary Parameter Adaptation Fanhu Zeng Haiyang Guo Fei Zhu Li Shen Hao Tang MoMe 43 1 0 24 Feb 2025
A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks Thomas Schmied Thomas Adler Vihang Patil M. Beck Korbinian Poppel Johannes Brandstetter G. Klambauer Razvan Pascanu Sepp Hochreiter 55 4 0 21 Feb 2025
SuperMerge: An Approach For Gradient-Based Model Merging Haoyu Yang Zheng Zhang Saket Sathe MoMe 118 0 0 17 Feb 2025
LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits Zikai Zhou Qizheng Zhang Hermann Kumbong Kunle Olukotun MQ 118 0 0 12 Feb 2025
RankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language Models Can Jin Hongwu Peng Anxiang Zhang Nuo Chen Jiahui Zhao ... K. Li Shuya Feng Kai Zhong Caiwen Ding Dimitris N. Metaxas 99 2 0 02 Feb 2025
LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation Can Jin Ying Li Mingyu Zhao Shiyu Zhao Z. Wang Xiaoxiao He Ligong Han Tong Che Dimitris N. Metaxas VPVLM VLM 91 1 0 02 Feb 2025
FedTLU: Federated Learning with Targeted Layer Updates Jong-Ik Park Carlee Joe-Wong FedML 73 0 0 28 Jan 2025
FlanEC: Exploring Flan-T5 for Post-ASR Error Correction Moreno La Quatra Valerio Mario Salerno Yu Tsao Sabato Marco Siniscalchi 76 0 0 22 Jan 2025
PsychAdapter: Adapting LLM Transformers to Reflect Traits, Personality and Mental Health Huy-Hien Vu Huy Anh Nguyen Adithya V Ganesan Swanie Juhng O. Kjell ... Margaret L. Kern Ryan L. Boyd L. Ungar H. A. Schwartz J. Eichstaedt 72 0 0 03 Jan 2025
TabVer: Tabular Fact Verification with Natural Logic Rami Aly Andreas Vlachos LMTD 26 0 0 02 Nov 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 38 3 0 24 Oct 2024
Closed-form merging of parameter-efficient modules for Federated Continual Learning Riccardo Salami Pietro Buzzega Matteo Mosconi Jacopo Bonato Luigi Sabetta Simone Calderara FedML MoMe CLL 19 2 0 23 Oct 2024
Opportunities and Challenges of Generative-AI in Finance Akshar Prabhu Desai Ganesh Satish Mallya Mohammad Luqman Tejasvi Ravi Nithya Kota Pranjul Yadav AIFin 28 2 0 21 Oct 2024
Deep Transfer Learning: Model Framework and Error Analysis Yuling Jiao Huazhen Lin Yuchen Luo Jerry Zhijian Yang 26 1 0 12 Oct 2024
Fine-tuning can Help Detect Pretraining Data from Large Language Models H. Zhang Songxin Zhang Bingyi Jing Hongxin Wei 31 0 0 09 Oct 2024
PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches Rana Muhammad Shahroz Khan Pingzhi Li Sukwon Yun Zhenyu Wang S. Nirjon Chau-Wai Wong Tianlong Chen KELM 24 2 0 08 Oct 2024
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 52 6 0 06 Oct 2024
TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models Zefang Liu Yinzhu Quan 16 0 0 02 Oct 2024
HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models Bingshen Mu Kun Wei Qijie Shao Yong Xu Lei Xie MoE 29 1 0 30 Sep 2024
Customizing Language Models with Instance-wise LoRA for Sequential Recommendation Xiaoyu Kong Jiancan Wu An Zhang Leheng Sheng Hui Lin Xiang Wang Xiangnan He AI4TS 43 4 0 19 Aug 2024
LLM-based MOFs Synthesis Condition Extraction using Few-Shot Demonstrations Lei Shi Zhimeng Liu Yi Yang Weize Wu Yuyang Zhang ... Zipeng Liu Huobin Tan Hongyi Gao Yue Zhang Ge Wang 26 0 0 06 Aug 2024
LoPT: Low-Rank Prompt Tuning for Parameter Efficient Language Models Shouchang Guo Sonam Damani Keng-hao Chang VLM 21 1 0 27 Jun 2024
From Instance Training to Instruction Learning: Task Adapters Generation from Instructions Huanxuan Liao Yao Xu Shizhu He Yuanzhe Zhang Yanchao Hao Shengping Liu Kang Liu Jun Zhao 35 1 0 18 Jun 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris S. Jyothi 19 3 0 16 Jun 2024
The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 21 10 0 12 Jun 2024
$$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning$ $\textit{Trans-LoRA}$ : towards data-free Transferable Parameter Efficient Finetuning Runqian Wang Soumya Ghosh David D. Cox Diego Antognini Aude Oliva Rogerio Feris Leonid Karlinsky 20 1 0 27 May 2024
On Fairness of Low-Rank Adaptation of Large Models Zhoujie Ding Ken Ziyu Liu Pura Peetathawatchai Berivan Isik Sanmi Koyejo 25 4 0 27 May 2024
Rényi Neural Processes Xuesong Wang He Zhao Edwin V. Bonilla UQCV BDL 12 0 0 25 May 2024
Implicit In-context Learning Zhuowei Li Zihao Xu Ligong Han Yunhe Gao Song Wen Di Liu Hao Wang Dimitris N. Metaxas 28 1 0 23 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su E. Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Page-Caccia Alessandro Sordoni MoMe 21 30 0 18 May 2024
Contextual Emotion Recognition using Large Vision Language Models Yasaman Etesam Özge Nilay Yalçin Chuxuan Zhang Angelica Lim VLM 49 3 0 14 May 2024
CANAL -- Cyber Activity News Alerting Language Model: Empirical Approach vs. Expensive LLM Urjitkumar Patel Fang-Chun Yeh Chinmay Gondhalekar 11 3 0 10 May 2024
In-Context Learning with Long-Context Models: An In-Depth Exploration Amanda Bertsch Maor Ivgi Uri Alon Jonathan Berant Matthew R. Gormley Matthew R. Gormley Graham Neubig ReLM AIMat 65 64 0 30 Apr 2024
LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing Zeyang Ma A. Chen Dong Jae Kim Tse-Husn Chen Shaowei Wang 21 41 0 27 Apr 2024
Evaluation of Few-Shot Learning for Classification Tasks in the Polish Language Tsimur Hadeliya D. Kajtoch 25 0 0 27 Apr 2024
ColA: Collaborative Adaptation with Gradient Learning Enmao Diao Qi Le Suya Wu Xinran Wang Ali Anwar Jie Ding Vahid Tarokh 14 1 0 22 Apr 2024
MM-PhyRLHF: Reinforcement Learning Framework for Multimodal Physics Question-Answering Avinash Anand Janak Kapuriya Chhavi Kirtani Apoorv Singh Jay Saraf Naman Lal Jatin Kumar A. Shivam Astha Verma R. Shah OffRL 34 9 0 19 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 17 7 0 13 Apr 2024
AnchorAL: Computationally Efficient Active Learning for Large and Imbalanced Datasets Pietro Lesci Andreas Vlachos 18 2 0 08 Apr 2024
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation Tong Su Xin Peng Sarubi Thillainathan David Guzmán Surangika Ranathunga En-Shiun Annie Lee 14 2 0 05 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 28 6 0 05 Apr 2024
Dual-path Frequency Discriminators for Few-shot Anomaly Detection Yuhu Bai Jiangning Zhang Yuhang Dong Guanzhong Tian Liang Liu Yunkang Cao Yabiao Wang Chengjie Wang 14 2 0 07 Mar 2024
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning Xupeng Miao Gabriele Oliaro Xinhao Cheng Vineeth Kada Ruohan Gao ... April Yang Yingcheng Wang Mengdi Wu Colin Unger Zhihao Jia MoE 85 8 0 29 Feb 2024
How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation? Rheeya Uppaal Yixuan Li Junjie Hu 25 4 0 31 Jan 2024
Low-Resource Vision Challenges for Foundation Models Yunhua Zhang Hazel Doughty Cees G. M. Snoek VLM 22 5 0 09 Jan 2024
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 16 5 0 25 Dec 2023
Towards Better Serialization of Tabular Data for Few-shot Classification with Large Language Models Sukriti Jaitly Tanay Shah Ashish Shugani Razik Singh Grewal LMTD 8 5 0 18 Dec 2023