Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning

1 October 2023

Papers citing "Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning"

25 / 25 papers shown

Title
A Data-Centric Approach for Safe and Secure Large Language Models against Threatening and Toxic Content Chaima Njeh Haïfa Nakouri Fehmi Jaafar 24 0 0 19 Apr 2025
Mimic In-Context Learning for Multimodal Tasks Yuchu Jiang Jiale Fu Chenduo Hao Xinting Hu Yingzhe Peng Xin Geng Xu Yang 27 0 0 11 Apr 2025
Analyzing Fine-tuning Representation Shift for Multimodal LLMs Steering alignment Pegah Khayatan Mustafa Shukor Jayneel Parekh Matthieu Cord LLMSV 41 1 0 06 Jan 2025
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future Shilin Sun Wenbin An Feng Tian Fang Nan Qidong Liu Xiaozhong Liu N. Shah Ping Chen 96 2 0 18 Dec 2024
What Factors Affect Multi-Modal In-Context Learning? An In-Depth Exploration L. Qin Qiguang Chen Hao Fei Zhi Chen Min Li Wanxiang Che 41 5 0 27 Oct 2024
Evaluating Linguistic Capabilities of Multimodal LLMs in the Lens of Few-Shot Learning Mustafa Dogan .Ilker Kesen Iacer Calixto Aykut Erdem Erkut Erdem LRM 37 1 0 17 Jul 2024
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models Feng Li Renrui Zhang Hao Zhang Yuanhan Zhang Bo Li Wei Li Zejun Ma Chunyuan Li MLLM VLM 52 198 0 10 Jul 2024
From Efficient Multimodal Models to World Models: A Survey Xinji Mai Zeng Tao Junxiong Lin Haoran Wang Yang Chang Yanlan Kang Yan Wang Wenqiang Zhang 32 5 0 27 Jun 2024
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding Fei Wang Xingyu Fu James Y. Huang Zekun Li Qin Liu ... Kai-Wei Chang Dan Roth Sheng Zhang Hoifung Poon Muhao Chen VLM 50 47 0 13 Jun 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 68 5 0 26 May 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks Jacob Russin Sam Whitman McGrath Danielle J. Williams Lotem Elber-Dorozko AI4CE 75 3 0 24 May 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 40 18 0 24 Apr 2024
Assessment of Multimodal Large Language Models in Alignment with Human Values Zhelun Shi Zhipin Wang Hongxing Fan Zaibin Zhang Lijun Li Yongting Zhang Zhen-fei Yin Lu Sheng Yu Qiao Jing Shao 42 15 0 26 Mar 2024
Improved Baselines for Data-efficient Perceptual Augmentation of LLMs Théophane Vallaeys Mustafa Shukor Matthieu Cord Jakob Verbeek 56 12 0 20 Mar 2024
Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning Tejas Srinivasan Jack Hessel Tanmay Gupta Bill Yuchen Lin Yejin Choi Jesse Thomason Khyathi Raghavi Chandu 24 7 0 23 Feb 2024
Browse and Concentrate: Comprehending Multimodal Content via prior-LLM Context Fusion Ziyue Wang Chi Chen Yiqi Zhu Fuwen Luo Peng Li Ming Yan Ji Zhang Fei Huang Maosong Sun Yang Liu 46 5 0 19 Feb 2024
MERA: A Comprehensive LLM Evaluation in Russian Alena Fenogenova Artem Chervyakov Nikita Martynov Anastasia Kozlova Maria Tikhonova ... Nikita Savushkin Polina Mikhailova Denis Dimitrov Alexander Panchenko Sergey Markov ELM 39 10 0 09 Jan 2024
LightHouse: A Survey of AGI Hallucination Feng Wang LRM HILM VLM 32 3 0 08 Jan 2024
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 61 42 0 30 Jul 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 281 4,244 0 30 Jan 2023
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 392 4,137 0 28 Jan 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 395 8,495 0 28 Jan 2022
Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning Ali Furkan Biten L. G. I. Bigorda Dimosthenis Karatzas 97 57 0 04 Oct 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 279 1,124 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,312 0 17 Jan 2021