Multimodal Few-Shot Learning with Frozen Language Models

25 June 2021

Papers citing "Multimodal Few-Shot Learning with Frozen Language Models"

50 / 532 papers shown

Title
Topological Data Analysis Guided Segment Anything Model Prompt Optimization for Zero-Shot Segmentation in Biological Imaging Ruben Glatt Shusen Liu 19 3 0 30 Jun 2023
Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language William Berrios Gautam Mittal Tristan Thrush Douwe Kiela Amanpreet Singh MLLM VLM 13 60 0 28 Jun 2023
DesCo: Learning Object Recognition with Rich Language Descriptions Liunian Harold Li Zi-Yi Dou Nanyun Peng Kai-Wei Chang ObjD VLM 24 20 0 24 Jun 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Bill Xu Enhong Chen MLLM LRM 45 553 0 23 Jun 2023
Generative Multimodal Entity Linking Senbao Shi Zhenran Xu Baotian Hu M. Zhang MLLM VLM 19 5 0 22 Jun 2023
Learning to Summarize and Answer Questions about a Virtual Robot's Past Actions Chad DeChant Iretiayo Akinola Daniel Bauer 16 7 0 16 Jun 2023
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 38 12 0 16 Jun 2023
Tell Me Where to Go: A Composable Framework for Context-Aware Embodied Robot Navigation Harel Biggie Ajay Narasimha Mopidevi Dusty Woods Christoffer Heckman LM&Ro 11 11 0 15 Jun 2023
Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models Junting Pan Ziyi Lin Yuying Ge Xiatian Zhu Renrui Zhang Yi Wang Yu Qiao Hongsheng Li MLLM 24 26 0 15 Jun 2023
Generating Language Corrections for Teaching Physical Control Tasks Megha Srivastava Noah D. Goodman Dorsa Sadigh 26 5 0 12 Jun 2023
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding Hang Zhang Xin Li Lidong Bing MLLM 53 948 0 05 Jun 2023
Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models Shuo Chen Jindong Gu Zhen Han Yunpu Ma Philip H. S. Torr Volker Tresp VPVLM VLM 27 17 0 03 Jun 2023
Prompt Tuning Large Language Models on Personalized Aspect Extraction for Recommendations Pan Li Yuyan Wang Ed H. Chi Minmin Chen 16 12 0 02 Jun 2023
MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models Masoud Monajatipoor Liunian Harold Li Mozhdeh Rouhsedaghat Lin F. Yang Kai-Wei Chang MLLM LRM 19 12 0 02 Jun 2023
MEWL: Few-shot multimodal word learning with referential uncertainty Guangyuan Jiang Manjie Xu Shiji Xin Weihan Liang Yujia Peng Chi Zhang Yixin Zhu OffRL 21 16 0 01 Jun 2023
Adapting Pre-trained Language Models to Vision-Language Tasks via Dynamic Visual Prompting Shubin Huang Qiong Wu Yiyi Zhou Weijie Chen Rongsheng Zhang Xiaoshuai Sun Rongrong Ji VLM VPVLM LRM 16 0 0 01 Jun 2023
GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Task Ning Ding Yehui Tang Zhongqian Fu Chaoting Xu Kai Han Yunhe Wang MLLM VLM 29 0 0 01 Jun 2023
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting R. Ramos Bruno Martins Desmond Elliott VLM 13 16 0 31 May 2023
Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning Xiaoxin He Xavier Bresson T. Laurent Adam Perold Yann LeCun Bryan Hooi 31 71 0 31 May 2023
Contextual Object Detection with Multimodal Large Language Models Yuhang Zang Wei Li Jun Han Kaiyang Zhou Chen Change Loy ObjD VLM MLLM 27 77 0 29 May 2023
Modularized Zero-shot VQA with Pre-trained Models Rui Cao Jing Jiang LRM 23 2 0 27 May 2023
Generating Images with Multimodal Language Models Jing Yu Koh Daniel Fried Ruslan Salakhutdinov MLLM 28 241 0 26 May 2023
Zero-shot Visual Question Answering with Language Model Feedback Yifan Du Junyi Li Tianyi Tang Wayne Xin Zhao Ji-Rong Wen 16 13 0 26 May 2023
On Evaluating Adversarial Robustness of Large Vision-Language Models Yunqing Zhao Tianyu Pang Chao Du Xiao Yang Chongxuan Li Ngai-man Cheung Min-Bin Lin VLM AAML MLLM 19 166 0 26 May 2023
ChatCAD+: Towards a Universal and Reliable Interactive CAD using LLMs Zihao Zhao Sheng Wang Jinchen Gu Yitao Zhu Lanzhuju Mei Zixu Zhuang Zhiming Cui Qian Wang Dinggang Shen LM&MA 29 36 0 25 May 2023
Visually-Situated Natural Language Understanding with Contrastive Reading Model and Frozen Large Language Models Geewook Kim Hodong Lee D. Kim Haeji Jung S. Park Yoon Kim Sangdoo Yun Taeho Kil Bado Lee Seunghyun Park VLM 35 4 0 24 May 2023
ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories Heming Xia Qingxiu Dong Lei Li Jingjing Xu Tianyu Liu Ziwei Qin Zhifang Sui MLLM VLM 16 3 0 24 May 2023
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought Yao Mu Qinglong Zhang Mengkang Hu Wen Wang Mingyu Ding Jun Jin Bin Wang Jifeng Dai Yu Qiao Ping Luo LM&Ro LRM 23 219 0 24 May 2023
Exploring Diverse In-Context Configurations for Image Captioning Xu Yang Yongliang Wu Mingzhuo Yang Haokun Chen Xin Geng MLLM 19 51 0 24 May 2023
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions Woojeong Jin Subhabrata Mukherjee Yu Cheng Yelong Shen Weizhu Chen Ahmed Hassan Awadallah Damien Jose Xiang Ren ObjD VLM 25 8 0 24 May 2023
S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions Sangwoo Mo Minkyu Kim Kyungmin Lee Jinwoo Shin VLM CLIP 36 21 0 23 May 2023
Images in Language Space: Exploring the Suitability of Large Language Models for Vision & Language Tasks Sherzod Hakimov David Schlangen VLM 34 5 0 23 May 2023
Album Storytelling with Iterative Story-aware Captioning and Large Language Models Munan Ning Yujia Xie Dongdong Chen Zeyin Song Lu Yuan Yonghong Tian QiXiang Ye Liuliang Yuan 19 8 0 22 May 2023
Pengi: An Audio Language Model for Audio Tasks Soham Deshmukh Benjamin Elizalde Rita Singh Huaming Wang MLLM AuLLM 30 157 0 19 May 2023
Few-Shot Learning with Visual Distribution Calibration and Cross-Modal Distribution Alignment Runqi Wang Hao Zheng Xiaoyue Duan Jianzhuang Liu Yuning Lu Tian Wang Songcen Xu Baochang Zhang VLM 24 12 0 19 May 2023
X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models Yixiong Chen Li Liu C. Ding 26 21 0 18 May 2023
MedBLIP: Bootstrapping Language-Image Pre-training from 3D Medical Images and Texts Qiuhui Chen Xinyue Hu Zirui Wang Yi Hong LM&MA MedIm 14 34 0 18 May 2023
Discffusion: Discriminative Diffusion Models as Few-shot Vision and Language Learners Xuehai He Weixi Feng Tsu-jui Fu Varun Jampani Arjun Reddy Akula P. Narayana Sugato Basu William Yang Wang X. Wang DiffM 39 7 0 18 May 2023
ArtGPT-4: Towards Artistic-understanding Large Vision-Language Models with Enhanced Adapter Zheng Yuan HU Xue Kun Wang Yongming Liu Kun Wang VLM MLLM 21 5 0 12 May 2023
MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognition Xinyu Gong S. Mohan Naina Dhingra Jean-Charles Bazin Yilei Li Zhangyang Wang Rakesh Ranjan EgoV 54 17 0 12 May 2023
Masked Audio Text Encoders are Effective Multi-Modal Rescorers Jason (Jinglun) Cai Monica Sunkara Xilai Li Anshu Bhatia Xiao Pan S. Bodapati 26 3 0 11 May 2023
Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts Zhaoyang Zhang Yantao Shen Kunyu Shi Zhaowei Cai Jun Fang Siqi Deng Hao-Yu Yang Davide Modolo Z. Tu Stefano Soatto VLM 25 2 0 11 May 2023
Text-To-Concept (and Back) via Cross-Model Alignment Mazda Moayeri Keivan Rezaei Maziar Sanjabi S. Feizi CLIP 31 39 0 10 May 2023
Read, Diagnose and Chat: Towards Explainable and Interactive LLMs-Augmented Depression Detection in Social Media Wei Qin Zetong Chen Lei Wang Yunshi Lan Wei Ren Richang Hong AI4MH 25 18 0 09 May 2023
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues Yunxin Li Baotian Hu Xinyu Chen Yuxin Ding Lin Ma Min Zhang LRM 48 14 0 08 May 2023
Few-shot Domain-Adaptive Visually-fused Event Detection from Text Farhad Moghimifar Fatemeh Shiri Van Nguyen Gholamreza Haffari Yuanyou Li VLM 30 2 0 04 May 2023
Making the Most of What You Have: Adapting Pre-trained Visual Language Models in the Low-data Regime Chuhan Zhang Antoine Miech Jiajun Shen Jean-Baptiste Alayrac Pauline Luc VLM VPVLM 39 2 0 03 May 2023
Multimodal Procedural Planning via Dual Text-Image Prompting Yujie Lu Pan Lu Zhiyu Zoey Chen Wanrong Zhu X. Wang William Yang Wang LM&Ro 62 43 0 02 May 2023
VPGTrans: Transfer Visual Prompt Generator across LLMs Ao Zhang Hao Fei Yuan Yao Wei Ji Li Li Zhiyuan Liu Tat-Seng Chua MLLM VLM 27 85 0 02 May 2023
Framing the News:From Human Perception to Large Language Model Inferences David Alonso del Barrio D. Gática-Pérez 28 12 0 27 Apr 2023