Title
PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction Long Xing Qidong Huang Xiaoyi Dong Jiajie Lu Pan Zhang ... Yuhang Cao Conghui He Jiaqi Wang Feng Wu Dahua Lin VLM 33 25 0 22 Oct 2024
Beyond Filtering: Adaptive Image-Text Quality Enhancement for MLLM Pretraining Han Huang Yuqi Huo Zijia Zhao Haoyu Lu Shu Wu B. Wang Qiang Liu Weipeng Chen Liang Wang VLM 20 0 0 21 Oct 2024
PUMA: Empowering Unified MLLM with Multi-granular Visual Generation Rongyao Fang Chengqi Duan Kun Wang Hao Li H. Tian Xingyu Zeng Rui Zhao Jifeng Dai Hongsheng Li Xihui Liu MLLM 26 11 0 17 Oct 2024
RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models Haoran Hao Jiaming Han Changsheng Li Yu-Feng Li Xiangyu Yue RALM 37 1 0 17 Oct 2024
The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio Sicong Leng Yun Xing Zesen Cheng Yang Zhou Hang Zhang Xin Li Deli Zhao Shijian Lu Chunyan Miao Lidong Bing 20 5 0 16 Oct 2024
VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI Sijie Cheng Kechen Fang Yangyang Yu Sicheng Zhou B. Li Ye Tian Tingguang Li Lei Han Yang Janet Liu 28 8 0 15 Oct 2024
When Does Perceptual Alignment Benefit Vision Representations? Shobhita Sundaram Stephanie Fu Lukas Muttenthaler Netanel Y. Tamir Lucy Chai Simon Kornblith Trevor Darrell Phillip Isola 41 8 1 14 Oct 2024
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models Hang Hua Yunlong Tang Ziyun Zeng Liangliang Cao Zhengyuan Yang Hangfeng He Chenliang Xu Jiebo Luo VLM CoGe 18 9 0 13 Oct 2024
ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time Yi Ding Bolian Li Ruqi Zhang MLLM 49 4 0 09 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 53 25 0 04 Oct 2024
Vinoground: Scrutinizing LMMs over Dense Temporal Reasoning with Short Videos Jianrui Zhang Mu Cai Yong Jae Lee 18 2 0 03 Oct 2024
Video Instruction Tuning With Synthetic Data Yuanhan Zhang Jinming Wu Wei Li Bo Li Zejun Ma Ziwei Liu Chunyuan Li SyDa VGen 34 1 0 03 Oct 2024
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 36 32 1 30 Sep 2024
MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines Dongzhi Jiang Renrui Zhang Ziyu Guo Yanmin Wu Jiayi Lei ... Guanglu Song Peng Gao Yu Liu Chunyuan Li Hongsheng Li MLLM 27 16 0 19 Sep 2024
Explore the Hallucination on Low-level Perception for MLLMs Yinan Sun Zicheng Zhang H. Wu Xiaohong Liu Weisi Lin Guangtao Zhai Xiongkuo Min 27 2 0 15 Sep 2024
Rhythmic Foley: A Framework For Seamless Audio-Visual Alignment In Video-to-Audio Synthesis Zhiqi Huang Dan Luo Jun Wang Huan Liao Zhiheng Li Zhiyong Wu VGen 27 4 0 13 Sep 2024
POINTS: Improving Your Vision-language Model with Affordable Strategies Yuan Liu Zhongyin Zhao Ziyuan Zhuang Le Tian Xiao Zhou Jie Zhou VLM 22 2 0 07 Sep 2024
Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videos Jiajun Fei Dian Li Zhidong Deng Zekun Wang Gang Liu Hui Wang VLM 25 33 0 26 Aug 2024
Building and better understanding vision-language models: insights and future directions Hugo Laurençon Andrés Marafioti Victor Sanh Léo Tronchon VLM 29 60 0 22 Aug 2024
mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Models Jiabo Ye Haiyang Xu Haowei Liu Anwen Hu Ming Yan Qi Qian Ji Zhang Fei Huang Jingren Zhou MLLM VLM 34 9 0 09 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 43 538 0 06 Aug 2024
MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding Xinyu Fang Kangrui Mao Haodong Duan Xiangyu Zhao Yining Li Dahua Lin Kai Chen VLM 49 61 0 20 Jun 2024
ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools Team GLM : Aohan Zeng Bin Xu Bowen Wang ... Zhaoyu Wang Zhen Yang Zhengxiao Du Zhenyu Hou Zihan Wang ALM 50 52 0 18 Jun 2024
Many-Shot In-Context Learning in Multimodal Foundation Models Yixing Jiang Jeremy Irvin Ji Hun Wang Muhammad Ahmed Chaudhry Jonathan H. Chen Andrew Y. Ng VLM 33 19 0 16 May 2024
From Image to Video, what do we need in multimodal LLMs? Suyuan Huang Haoxin Zhang Yan Gao Yao Hu Zengchang Qin VLM 26 8 0 18 Apr 2024
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding Bo He Hengduo Li Young Kyun Jang Menglin Jia Xuefei Cao Ashish Shah Abhinav Shrivastava Ser-Nam Lim MLLM 77 87 0 08 Apr 2024
LongVLM: Efficient Long Video Understanding via Large Language Models Yuetian Weng Mingfei Han Haoyu He Xiaojun Chang Bohan Zhuang VLM 34 56 0 04 Apr 2024
Are We on the Right Way for Evaluating Large Vision-Language Models? Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Yuhang Zang ... Haodong Duan Jiaqi Wang Yu Qiao Dahua Lin Feng Zhao VLM 49 208 0 29 Mar 2024
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM Wonkyun Kim Changin Choi Wonseok Lee Wonjong Rhee VLM 40 50 0 27 Mar 2024
Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination Dingchen Yang Bowen Cao Guang Chen Changjun Jiang 35 7 0 21 Mar 2024
Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset Hugo Laurençon Léo Tronchon Victor Sanh VLM 47 13 0 14 Mar 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 150 437 0 02 Feb 2024
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Conghui He Xingcheng Zhang Yu Qiao Dahua Lin Jiaqi Wang VLM MLLM 70 89 0 29 Jan 2024
A Simple LLM Framework for Long-Range Video Question-Answering Ce Zhang Taixi Lu Md. Mohaiminul Islam Ziyang Wang Shoubin Yu Mohit Bansal Gedas Bertasius 95 80 0 28 Dec 2023
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 126 895 0 21 Dec 2023
Silkie: Preference Distillation for Large Visual Language Models Lei Li Zhihui Xie Mukai Li Shunian Chen Peiyi Wang Liang Chen Yazheng Yang Benyou Wang Lingpeng Kong MLLM 96 67 0 17 Dec 2023
CogAgent: A Visual Language Model for GUI Agents Wenyi Hong Weihan Wang Qingsong Lv Jiazheng Xu Wenmeng Yu ... Juanzi Li Bin Xu Yuxiao Dong Ming Ding Jie Tang MLLM 132 310 0 14 Dec 2023
Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models Haoran Wei Lingyu Kong Jinyue Chen Liang Zhao Zheng Ge Jinrong Yang Jian‐Yuan Sun Chunrui Han Xiangyu Zhang MLLM VLM 61 39 0 11 Dec 2023
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback M. Steyvers Yuan Yao Haoye Zhang Taiwen He Yifeng Han ... Xinyue Hu Zhiyuan Liu Hai-Tao Zheng Maosong Sun Tat-Seng Chua MLLM VLM 125 176 0 01 Dec 2023
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection Bin Lin Yang Ye Bin Zhu Jiaxi Cui Munan Ning Peng Jin Li-ming Yuan VLM MLLM 182 576 0 16 Nov 2023
UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Model Jiabo Ye Anwen Hu Haiyang Xu Qinghao Ye Mingshi Yan ... Ji Zhang Qin Jin Liang He Xin Lin Feiyan Huang VLM MLLM 118 83 0 08 Oct 2023
WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models Conghui He Zhenjiang Jin Chaoxi Xu Jiantao Qiu Bin Wang Wei Li Hang Yan Jiaqi Wang Da Lin 56 32 0 21 Aug 2023
On the Hidden Mystery of OCR in Large Multimodal Models Yuliang Liu Zhang Li Mingxin Huang Chunyuan Li Dezhi Peng Mingyu Liu Lianwen Jin Xiang Bai VLM MLLM 16 47 0 13 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 198 883 0 27 Apr 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 237 840 0 05 Oct 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022