BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

30 January 2023

Silvio Savarese

Papers citing "BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models"

50 / 52 papers shown

Title
JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers Kwon Byung-Ki Qi Dai Lee Hyoseok Chong Luo Tae-Hyun Oh 16 0 0 01 May 2025
SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models Wufei Ma Luoxin Ye Nessa McWeeney Celso M de Melo A. Yuille Jieneng Chen LRM 6 0 0 01 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Tianlong Chen Mohit Bansal AAML MU 18 0 0 01 May 2025
Improving Routing in Sparse Mixture of Experts with Graph of Tokens Tam Minh Nguyen Ngoc N. Tran Khai Nguyen Richard G. Baraniuk MoE 16 0 0 01 May 2025
RoboGround: Robotic Manipulation with Grounded Vision-Language Priors Haifeng Huang Xinyi Chen Y. Chen H. Li Xiaoshen Han Z. Wang Tai Wang Jiangmiao Pang Zhou Zhao LM&Ro 39 0 0 30 Apr 2025
Rethinking Visual Layer Selection in Multimodal LLMs H. Chen Junyan Lin Xinhao Chen Yue Fan Xin Jin Hui Su Jianfeng Dong Jinlan Fu Xiaoyu Shen VLM 51 0 0 30 Apr 2025
Eye2Eye: A Simple Approach for Monocular-to-Stereo Video Synthesis Michal Geyer Omer Tov Linyi Jin Richard Tucker Inbar Mosseri Tali Dekel Noah Snavely DiffM VGen 59 0 0 30 Apr 2025
AGHI-QA: A Subjective-Aligned Dataset and Metric for AI-Generated Human Images Yunhao Li Sijing Wu Wei Sun Zhichao Zhang Yucheng Zhu Zicheng Zhang Huiyu Duan Xiongkuo Min Guangtao Zhai EGVM 44 0 0 30 Apr 2025
An Evaluation of a Visual Question Answering Strategy for Zero-shot Facial Expression Recognition in Still Images Modesto Castrillón-Santana Oliverio J. Santana David Freire-Obregón Daniel Hernández-Sosa J. Lorenzo-Navarro 28 0 0 30 Apr 2025
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 61 57 0 30 Apr 2025
CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation Jianyu Wu Yizhou Wang Xiangyu Yue Xinzhu Ma J. Guo Dongzhan Zhou Wanli Ouyang Shixiang Tang 27 0 0 29 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 33 0 0 29 Apr 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 49 0 0 29 Apr 2025
MemeBLIP2: A novel lightweight multimodal system to detect harmful memes Jiaqi Liu Ran Tong Aowei Shen Shuzheng Li Changlin Yang Lisha Xu VLM 43 0 0 29 Apr 2025
VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuning Run Luo Renke Shan Longze Chen Z. Liu Lu Wang Min Yang Xiaobo Xia MLLM VLM 59 47 0 28 Apr 2025
EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipedia Valerie Zermatten J. Castillo-Navarro Pallavi Jain D. Tuia Diego Marcos 26 56 0 28 Apr 2025
VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs? Mohamed Gado Towhid Taliee Muhammad Memon D. Ignatov Radu Timofte 24 41 0 27 Apr 2025
CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis Alexander Baumann Leonardo Ayala S. Jan Sellner Alexander Studier-Fischer Berkin Özdemir Lena Maier-Hein Slobodan Ilic 24 37 0 27 Apr 2025
Anyprefer: An Agentic Framework for Preference Data Synthesis Yiyang Zhou Z. Wang Tianle Wang Shangyu Xing Peng Xia ... Chetan Bansal Weitong Zhang Ying Wei Mohit Bansal Huaxiu Yao 31 0 0 27 Apr 2025
Platonic Grounding for Efficient Multimodal Language Models Moulik Choraria Xinbo Wu Akhil Bhimaraju Nitesh Sekhar Yue Wu Xu Zhang Prateek Singhal L. Varshney 34 52 0 27 Apr 2025
Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation Shahad Albastaki Anabia Sohail I. I. Ganapathi B. Alawode Asim Khan Sajid Javed N. Werghi Mohammed Bennamoun Arif Mahmood 33 96 0 26 Apr 2025
ActionArt: Advancing Multimodal Large Models for Fine-Grained Human-Centric Video Understanding Yi-Xing Peng Q. Yang Yu-Ming Tang Shenghao Fu Kun-Yu Lin Xihan Wei Wei-Shi Zheng 14 0 0 25 Apr 2025
A Large Vision-Language Model based Environment Perception System for Visually Impaired People Zezhou Chen Zhaoxiang Liu Kai Wang Kohou Wang Shiguo Lian 26 0 0 25 Apr 2025
VideoMultiAgents: A Multi-Agent Framework for Video Question Answering Noriyuki Kugo Xiang Li Z. Li Ashish Gupta Arpandeep Khatua ... Yuta Kyuragi Yasunori Ishii Masamoto Tanabiki Kazuki Kozuka Ehsan Adeli 14 0 0 25 Apr 2025
Token Sequence Compression for Efficient Multimodal Computing Yasmine Omri Parth Shroff Thierry Tambe 16 0 0 24 Apr 2025
Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning Chris Yichen Wei Yi Peng X. Wang Weijie Qiu ... Jianhao Zhang Y. Hao Xuchen Song Yang Liu Yahui Zhou OffRL AI4TS SyDa LRM VLM 31 39 0 23 Apr 2025
FaceInsight: A Multimodal Large Language Model for Face Perception Jingzhi Li Changjiang Luo Ruoyu Chen Hua Zhang Wenqi Ren Jianhou Gan Xiaochun Cao CVBM LRM 30 0 0 22 Apr 2025
ForesightNav: Learning Scene Imagination for Efficient Exploration Hardik Shah Jiaxu Xing Nico Messikommer Boyang Sun Marc Pollefeys Davide Scaramuzza 40 0 0 22 Apr 2025
T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models Siyuan Liang Jiayang Liu Jiecheng Zhai Tianmeng Fang Rongcheng Tu A. Liu Xiaochun Cao Dacheng Tao VGen 28 0 0 22 Apr 2025
LGD: Leveraging Generative Descriptions for Zero-Shot Referring Image Segmentation Jiachen Li Qing Xie Xiaohan Yu Hongyun Wang Jinyu Xu Yongjian Liu ObjD 51 0 0 20 Apr 2025
Video-Bench: Human-Aligned Video Generation Benchmark Hui Han Siyuan Li Jiaqi Chen Yiwen Yuan Yuling Wu ... Y. Li J. Zhang Chi Zhang Li Li Yongxin Ni EGVM VGen 38 0 0 07 Apr 2025
Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding Xiangrui Liu Yan Shu Zheng Liu Ao Li Yang Tian Bo Zhao VGen VLM 56 0 0 24 Mar 2025
Vision-based 3D Semantic Scene Completion via Capture Dynamic Representations Meng Wang Fan Wu Yunchuan Qin Ruihui Li Zhuo Tang KenLi Li 3DPC 56 0 0 08 Mar 2025
WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation Dujun Nie Xianda Guo Yiqun Duan Ruijun Zhang Long Chen LM&Ro 67 0 0 04 Mar 2025
MatterChat: A Multi-Modal LLM for Material Science Yingheng Tang Wenbin Xu Jie Cao Jianzhu Ma Weilu Gao Steve Farrell Benjamin Erichson Michael W. Mahoney Andy Nonaka 65 3 0 18 Feb 2025
Progressive Compositionality in Text-to-Image Generative Models Xu Han Linghao Jin Xiaofeng Liu Paul Pu Liang CoGe 40 2 0 22 Oct 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip H. S. Torr Mor Geva David M. Krueger Fazl Barez 45 6 0 09 Oct 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 33 29 0 26 Sep 2024
Generative Semantic Communication via Textual Prompts: Latency Performance Tradeoffs Mengmeng Ren Li Qiao Long Yang Zhen Gao Jian Chen Mahdi Boloursaz Mashhadi Pei Xiao Rahim Tafazolli Mehdi Bennis VLM 24 0 0 15 Sep 2024
FungiTastic: A multi-modal dataset and benchmark for image categorization Lukás Picek Klara Janouskova Milan Šulc Jirí Matas 32 1 0 24 Aug 2024
GOFA: A Generative One-For-All Model for Joint Graph Language Modeling Lecheng Kong Jiarui Feng Hao Liu Chengsong Huang Jiaxin Huang Yixin Chen Muhan Zhang AI4CE 30 4 0 12 Jul 2024
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling Zeyue Tian Zhaoyang Liu Ruibin Yuan Jiahao Pan Xiaoqiang Huang Xu Tan Xu Tan Qifeng Chen Y. Guo VGen 37 2 0 06 Jun 2024
Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations? Letitia Parcalabescu Anette Frank MLLM CoGe VLM 43 3 0 29 Apr 2024
Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation Yutong He Alexander Robey Naoki Murata Yiding Jiang J. Williams George Pappas Hamed Hassani Yuki Mitsufuji Ruslan Salakhutdinov J. Zico Kolter DiffM 39 4 0 28 Mar 2024
Garment3DGen: 3D Garment Stylization and Texture Generation N. Sarafianos Tuur Stuyck Xiaoyu Xiang Yilei Li Jovan Popovic Rakesh Ranjan 3DH 43 16 0 27 Mar 2024
StoryGPT-V: Large Language Models as Consistent Story Visualizers Xiaoqian Shen Mohamed Elhoseiny VLM 50 9 0 04 Dec 2023
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions Weizhen He Yihe Deng Shixiang Tang Qihao Chen Qingsong Xie ... Feng Zhu Rui Zhao Wanli Ouyang Donglian Qi Yunfeng Yan 35 18 0 13 Jun 2023
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 359 2,713 0 28 Jan 2022
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 247 845 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 265 2,875 0 11 Feb 2021