LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention

28 March 2023

Yu Qiao

Papers citing "LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention"

50 / 52 papers shown

Title
AdCare-VLM: Leveraging Large Vision Language Model (LVLM) to Monitor Long-Term Medication Adherence and Care Md Asaduzzaman Jabin Hanqi Jiang Y. Li Patrick Kaggwa Eugene Douglass Juliet N. Sekandi Tianming Liu LM&MA 60 0 0 01 May 2025
Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models Minh-Hao Van Xintao Wu VLM 79 0 0 30 Apr 2025
E-InMeMo: Enhanced Prompting for Visual In-Context Learning Jiahao Zhang Bowen Wang Hong Liu Liangzhi Li Yuta Nakashima Hajime Nagahara VLM 94 0 0 25 Apr 2025
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Jiaming Liu Hao Chen Pengju An Zhuoyang Liu Renrui Zhang ... Chengkai Hou Mengdi Zhao KC alex Zhou Pheng-Ann Heng S. Zhang 58 5 0 13 Mar 2025
PersonaBooth: Personalized Text-to-Motion Generation Boeun Kim Hea In Jeong JungHoon Sung Yihua Cheng Jeongmin Lee ... Sang-Il Choi Younggeun Choi Saim Shin Jungho Kim Hyung Jin Chang DiffM VGen 53 0 0 10 Mar 2025
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting Chen Cai Zheng Wang J. Gao Wenyang Liu Ye Lu Runzhong Zhang Kim-Hui Yap CLL 42 0 0 20 Jan 2025
Transfer Learning Strategies for Pathological Foundation Models: A Systematic Evaluation in Brain Tumor Classification Ken Enda Yoshitaka Oda Zen-ichi Tanei Wang Lei Masumi Tsuda ... Shinya Tanaka Takahiro Ogawa Wang Lei Masumi Tsuda Shinya Tanaka 28 0 0 19 Jan 2025
DriveLM: Driving with Graph Visual Question Answering Chonghao Sima Katrin Renz Kashyap Chitta L. Chen Hanxue Zhang Chengen Xie Jens Beißwenger Ping Luo Andreas Geiger Hongyang Li 62 159 0 17 Jan 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang D. Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 87 102 0 10 Jan 2025
Efficient Fine-Tuning of Single-Cell Foundation Models Enables Zero-Shot Molecular Perturbation Prediction Sepideh Maleki Jan-Christian Huetter Kangway V Chuang Gabriele Scalia Tommaso Biancalani Tommaso Biancalani AI4CE 75 2 0 18 Dec 2024
EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios Lu Qiu Yuying Ge Yi Chen Yixiao Ge Ying Shan Xihui Liu LLMAG LRM 78 5 0 05 Dec 2024
IterIS: Iterative Inference-Solving Alignment for LoRA Merging Hongxu Chen Runshi Li Bowei Zhu Zhen Wang Long Chen MoMe 82 0 0 21 Nov 2024
Parameter Efficient Mamba Tuning via Projector-targeted Diagonal-centric Linear Transformation Seokil Ham H. Kim Sangmin Woo Changick Kim Mamba 83 0 0 21 Nov 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 35 3 0 24 Oct 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 21 2 0 23 Oct 2024
Opportunities and Challenges of Generative-AI in Finance Akshar Prabhu Desai Ganesh Satish Mallya Mohammad Luqman Tejasvi Ravi Nithya Kota Pranjul Yadav AIFin 23 2 0 21 Oct 2024
Frame-Voyager: Learning to Query Frames for Video Large Language Models Sicheng Yu Chengkai Jin Huanyu Wang Zhenghao Chen Sheng Jin ... Zhenbang Sun Bingni Zhang Jiawei Wu Hao Zhang Qianru Sun 39 5 0 04 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 53 25 0 04 Oct 2024
AudioComposer: Towards Fine-grained Audio Generation with Natural Language Descriptions Y. Wang Hangting Chen Dongchao Yang Zhiyong Wu Xixin Wu DiffM 33 2 0 19 Sep 2024
Uncertainty-Guided Self-Questioning and Answering for Video-Language Alignment Jin Chen Kaijing Ma Haojian Huang Jiayu Shen Han Fang Xianghao Zang Chao Ban 73 2 0 17 Sep 2024
Question-Answering Dense Video Events Hangyu Qin Junbin Xiao Angela Yao VLM 65 1 0 06 Sep 2024
Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understanding Yunze Man Shuhong Zheng Zhipeng Bao M. Hebert Liang-Yan Gui Yu-xiong Wang 67 15 0 05 Sep 2024
ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding Yijia Xiao Edward Sun Yiqiao Jin Qifan Wang Wei Wang 25 9 0 21 Aug 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 38 0 0 31 Jul 2024
ViLLa: Video Reasoning Segmentation with Large Language Model Rongkun Zheng Lu Qi Xi Chen Yi Wang Kun Wang Yu Qiao Hengshuang Zhao VOS LRM 37 2 0 18 Jul 2024
Evaluating Zero-Shot Long-Context LLM Compression Chenyu Wang Yihan Wang Kai Li 39 0 0 10 Jun 2024
Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification Weizhen He Yiheng Deng Yunfeng Yan Feng Zhu Yizhou Wang Lei Bai Qingsong Xie Donglian Qi Wanli Ouyang Shixiang Tang 84 2 0 28 May 2024
Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentation Ya Lu Jishnu Jaykumar Yunhui Guo Nicholas Ruozzi Yu Xiang VLM ISeg 28 3 0 28 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 49 38 0 23 May 2024
DEGAP: Dual Event-Guided Adaptive Prefixes for Templated-Based Event Argument Extraction with Slot Querying Guanghui Wang Dexi Liu Jian-Yun Nie Qizhi Wan Rong Hu Xiping Liu Wanlong Liu Jiaming Liu 47 0 0 22 May 2024
NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning Bingqian Lin Yunshuang Nie Ziming Wei Jiaqi Chen Shikui Ma Jianhua Han Hang Xu Xiaojun Chang Xiaodan Liang LM&Ro LRM 50 17 0 12 Mar 2024
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation Sunghyeon Woo Baeseong Park Byeongwook Kim Minjung Jo S. Kwon Dongsuk Jeon Dongsoo Lee 32 2 0 27 Feb 2024
JudgeLM: Fine-tuned Large Language Models are Scalable Judges Lianghui Zhu Xinggang Wang Xinlong Wang ELM ALM 54 103 0 26 Oct 2023
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models Jung Hwan Heo Jeonghoon Kim Beomseok Kwon Byeongwook Kim Se Jung Kwon Dongsoo Lee MQ 29 9 0 27 Sep 2023
Linear Alignment of Vision-language Models for Image Captioning Fabian Paischer M. Hofmarcher Sepp Hochreiter Thomas Adler CLIP VLM 24 0 0 10 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 62 222 0 07 Jul 2023
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions Weizhen He Yihe Deng Shixiang Tang Qihao Chen Qingsong Xie ... Feng Zhu Rui Zhao Wanli Ouyang Donglian Qi Yunfeng Yan 62 18 0 13 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 16 185 0 12 Jun 2023
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering Zhou Yu Xuecheng Ouyang Zhenwei Shao Mei Wang Jun Yu MLLM 86 11 0 03 Mar 2023
Retrieval Augmented Visual Question Answering with Outside Knowledge Weizhe Lin Bill Byrne RALM 74 68 0 07 Oct 2022
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan F. Khan VPVLM VLM 178 521 0 06 Oct 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 212 335 0 02 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 380 4,010 0 28 Jan 2022
PointCLIP: Point Cloud Understanding by CLIP Renrui Zhang Ziyu Guo Wei Zhang Kunchang Li Xupeng Miao Bin Cui Yu Qiao Peng Gao Hongsheng Li VLM 3DPC 158 428 0 04 Dec 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 170 281 0 06 Nov 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 228 780 0 14 Oct 2021
FILM: Following Instructions in Language with Modular Methods So Yeon Min Devendra Singh Chaplot Pradeep Ravikumar Yonatan Bisk Ruslan Salakhutdinov LM&Ro 187 159 0 12 Oct 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 319 2,108 0 02 Sep 2021