MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models

20 April 2023

Papers citing "MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models"

50 / 295 papers shown

Title
Libra: Leveraging Temporal Images for Biomedical Radiology Analysis Xi Zhang Zaiqiao Meng Jake Lever Edmond S. L. Ho MedIm 96 0 0 28 Nov 2024
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning Di Zhang Jingdi Lei Junxian Li Xunzhi Wang Y. Liu ... S. M. I. Simon X. Yang Jianbo Wu Peng Ye Wanli Ouyang Dongzhan Zhou OffRL LRM 105 6 0 27 Nov 2024
SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentation Claudia Cuttano Gabriele Trivigno Gabriele Rosi Carlo Masone Giuseppe Averta VOS 101 2 0 26 Nov 2024
VideoOrion: Tokenizing Object Dynamics in Videos Yicheng Feng Yijiang Li Wanpeng Zhang Sipeng Zheng Zongqing Lu Sipeng Zheng Zongqing Lu 104 1 0 25 Nov 2024
GEMeX: A Large-Scale, Groundable, and Explainable Medical VQA Benchmark for Chest X-ray Diagnosis Bo Liu K. Zou Liming Zhan Zexin Lu Xiaoyu Dong Yidi Chen Chengqiang Xie Jiannong Cao Xiao-Ming Wu Huazhu Fu 120 0 0 25 Nov 2024
Is 'Right' Right? Enhancing Object Orientation Understanding in Multimodal Large Language Models through Egocentric Instruction Tuning Ji Hyeok Jung Eun Tae Kim S. Kim Joo Ho Lee Bumsoo Kim Buru Chang VLM 133 0 0 24 Nov 2024
Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts Qizhou Chen Chengyu Wang Dakan Wang Taolin Zhang Wangyue Li Xiaofeng He KELM 73 1 0 23 Nov 2024
On the Consistency of Video Large Language Models in Temporal Comprehension Minjoon Jung Junbin Xiao Byoung-Tak Zhang Angela Yao 83 2 0 20 Nov 2024
Teaching VLMs to Localize Specific Objects from In-context Examples Sivan Doveh Nimrod Shabtay Wei Lin Eli Schwartz Hilde Kuehne ... Leonid Karlinsky James Glass Assaf Arbelle S. Ullman Muhammad Jehanzeb Mirza VLM 96 1 0 20 Nov 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 56 2 0 14 Nov 2024
CAD-MLLM: Unifying Multimodality-Conditioned CAD Generation With MLLM Jingwei Xu Chenyu Wang Zibo Zhao Wen Liu Yi-An Ma Shenghua Gao 50 13 0 07 Nov 2024
Exploring Hierarchical Molecular Graph Representation in Multimodal LLMs Chengxin Hu Hao Li Yihe Yuan Jing Li Ivor Tsang 39 0 0 07 Nov 2024
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos Shehan Munasinghe Hanan Gani Wenqi Zhu Jiale Cao Eric P. Xing F. Khan Salman Khan MLLM VGen VLM 44 6 0 07 Nov 2024
Performance evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forward Shashi Kumar Iuliia Thorbecke Sergio Burdisso Esaú Villatoro-Tello M. Errecalde Kadri Hacioğlu Pradeep Rangappa P. Motlícek A. Ganapathiraju Andreas Stolcke 51 1 0 06 Nov 2024
One VLM to Keep it Learning: Generation and Balancing for Data-free Continual Visual Question Answering Deepayan Das Davide Talon Massimiliano Mancini Yiming Wang Elisa Ricci 39 0 0 04 Nov 2024
UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models Sejoon Oh Yiqiao Jin Megha Sharma Donghyun Kim Eric Ma Gaurav Verma Srijan Kumar 60 5 0 03 Nov 2024
On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection Xiufeng Song Xiao Guo J. Zhang Qirui Li Lei Bai Xiaoming Liu Guangtao Zhai Xiaohong Liu DiffM VGen 69 8 0 31 Oct 2024
AutoBench-V: Can Large Vision-Language Models Benchmark Themselves? Han Bao Yue Huang Yanbo Wang Jiayi Ye Xiangqi Wang Xiuying Chen Mohamed Elhoseiny X. Zhang Mohamed Elhoseiny Xiangliang Zhang 47 7 0 28 Oct 2024
Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs) Leander Girrbach Yiran Huang Stephan Alaniz Trevor Darrell Zeynep Akata VLM 40 2 0 25 Oct 2024
PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction Long Xing Qidong Huang Xiaoyi Dong Jiajie Lu Pan Zhang ... Yuhang Cao Conghui He Jiaqi Wang Feng Wu Dahua Lin VLM 45 26 0 22 Oct 2024
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation Hyungjoo Chae Namyoung Kim Kai Tzu-iunn Ong Minju Gwak Gwanwoo Song Jihoon Kim S. Kim Dongha Lee Jinyoung Yeo LLMAG 33 14 0 17 Oct 2024
TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration Yiwei Guo Shaobin Zhuang Kunchang Li Yu Qiao Yali Wang VLM CLIP 21 0 0 16 Oct 2024
Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modeling Guiyu Zhang Huan-ang Gao Zijian Jiang Hao Zhao Zhedong Zheng EGVM 36 6 0 15 Oct 2024
3DArticCyclists: Generating Synthetic Articulated 8D Pose-Controllable Cyclist Data for Computer Vision Applications Eduardo R. Corral-Soto Yang Liu Tongtong Cao Y. Ren Liu Bingbing 44 5 0 14 Oct 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 61 4 0 10 Oct 2024
MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA Hanrong Ye Haotian Zhang Erik Daxberger Lin Chen Zongyu Lin ... Haoxuan You Dan Xu Zhe Gan Jiasen Lu Yinfei Yang EgoV MLLM 70 12 0 09 Oct 2024
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models Muhammad Jehanzeb Mirza Mengjie Zhao Zhuoyuan Mao Sivan Doveh Wei Lin ... Yuki Mitsufuji Horst Possegger Rogerio Feris Leonid Karlinsky James Glass VLM 76 1 0 08 Oct 2024
TRACE: Temporal Grounding Video LLM via Causal Event Modeling Yongxin Guo Jingyu Liu Mingda Li Xiaoying Tang Qingbin Liu Xiaoying Tang 33 14 0 08 Oct 2024
Geometric Analysis of Reasoning Trajectories: A Phase Space Approach to Understanding Valid and Invalid Multi-Hop Reasoning in LLMs Javier Marin LRM 72 0 0 06 Oct 2024
Enhancing Screen Time Identification in Children with a Multi-View Vision Language Model and Screen Time Tracker Xinlong Hou Sen Shen Xueshen Li Xinran Gao Ziyi Huang Steven J. Holiday Matthew R. Cribbet Susan W. White Edward Sazonov Yu Gan 34 0 0 02 Oct 2024
Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining Jie Cheng Ruixi Qiao Gang Xiong Binhua Li Yingwei Ma Binhua Li Yongbin Li Yisheng Lv OffRL OnRL LM&Ro 40 3 0 01 Oct 2024
Image-guided topic modeling for interpretable privacy classification Alina Elena Baia Andrea Cavallaro 32 0 0 27 Sep 2024
VLMine: Long-Tail Data Mining with Vision Language Models Mao Ye Gregory P. Meyer Zaiwei Zhang Dennis Park Siva Karthik Mustikovela Yuning Chai Eric M. Wolff VLM 21 1 0 23 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 59 2 0 19 Sep 2024
TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation Junjie Wen Y. X. Zhu Jinming Li Minjie Zhu Kun Wu ... Ran Cheng Chaomin Shen Yaxin Peng Feifei Feng Jian Tang LM&Ro 56 41 0 19 Sep 2024
Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attacks Md Zarif Hossain Ahmed Imteaj AAML VLM 38 3 0 11 Sep 2024
Deep Computer Vision for Solar Physics Big Data: Opportunities and Challenges Bo Shen Marco Marena Chenyang Li Qin Li Haodi Jiang Mengnan Du Jiajun Xu Haimin Wang 31 0 0 07 Sep 2024
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation Yi-Chia Chen Wei-Hua Li Cheng Sun Yu-Chiang Frank Wang Chu-Song Chen VLM 35 10 0 01 Sep 2024
StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models Y. Guo Faizan Siddiqui Yang Zhao Rama Chellappa Shao-Yuan Lo LRM 24 2 0 31 Aug 2024
RSTeller: Scaling Up Visual Language Modeling in Remote Sensing with Rich Linguistic Semantics from Openly Available Data and Large Language Models Junyao Ge Yang Zheng Kaitai Guo Jimin Liang Jimin Liang 29 1 0 27 Aug 2024
LLM-3D Print: Large Language Models To Monitor and Control 3D Printing Yayati Jadhav P. Pak Amir Barati Farimani AI4CE 84 8 0 26 Aug 2024
K-Sort Arena: Efficient and Reliable Benchmarking for Generative Models via K-wise Human Preferences Zhikai Li Xuewen Liu Dongrong Fu Jianquan Li Qingyi Gu Kurt Keutzer Zhen Dong EGVM VGen DiffM 81 1 0 26 Aug 2024
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model Feipeng Ma Yizhou Zhou Hebei Li Zilong He Siying Wu Fengyun Rao Siying Wu Fengyun Rao Yueyi Zhang Xiaoyan Sun 29 3 0 21 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 77 12 0 16 Aug 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 38 9 0 10 Aug 2024
AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning Xin Wang Kai-xiang Chen Xingjun Ma Zhineng Chen Jingjing Chen Yu-Gang Jiang AAML 34 3 0 04 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Haozhao Wang Zhicheng Chen Peilin Zhao VLM MLLM 61 18 0 04 Aug 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 41 5 0 31 Jul 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 62 0 0 31 Jul 2024
HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning Zhecan Wang Garrett Bingham Adams Wei Yu Quoc V. Le Thang Luong Golnaz Ghiasi MLLM LRM 35 9 0 22 Jul 2024