Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models

8 September 2023

Heng Ji

Papers citing "Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models"

34 / 34 papers shown

Title
VideoAds for Fast-Paced Video Understanding: Where Opensource Foundation Models Beat GPT-4o & Gemini-1.5 Pro Zheyuan Zhang Monica Dou Linkai Peng Hongyi Pan Ulas Bagci Boqing Gong VLM 56 0 0 12 Apr 2025
Evolutionary Prompt Optimization Discovers Emergent Multimodal Reasoning Strategies in Vision-Language Models Sid Bharthulwar John Rho Katrina Brown ReLM VLM LRM 50 0 0 30 Mar 2025
Exploring the Reliability of Self-explanation and its Relationship with Classification in Language Model-driven Financial Analysis Han Yuan Li Zhang Zheng Ma 77 0 0 20 Mar 2025
V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning Zixu Cheng Jian Hu Ziquan Liu Chenyang Si Wei Li Shaogang Gong LRM 68 2 0 14 Mar 2025
Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering Zeqing Wang Wentao Wan Qiqing Lao Runmeng Chen Minjie Lang Keze Wang Liang Lin Liang Lin LRM 96 3 0 17 Feb 2025
VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoning Xueqing Wu Yuheng Ding Bingxuan Li Pan Lu Da Yin Kai-Wei Chang Nanyun Peng LRM 100 3 0 03 Dec 2024
Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent Yangning Li Yinghui Li Xinyu Wang Yong-feng Jiang Zhen Zhang ... Hui Wang Hai-Tao Zheng Pengjun Xie Philip S. Yu Fei Huang 60 15 0 05 Nov 2024
Scaling Laws for Predicting Downstream Performance in LLMs Yangyi Chen Binxuan Huang Yifan Gao Zhengyang Wang Jingfeng Yang Heng Ji LRM 43 7 0 11 Oct 2024
VHELM: A Holistic Evaluation of Vision Language Models Tony Lee Haoqin Tu Chi Heem Wong Wenhao Zheng Yiyang Zhou ... Josselin Somerville Roberts Michihiro Yasunaga Huaxiu Yao Cihang Xie Percy Liang VLM 37 10 0 09 Oct 2024
Beyond Captioning: Task-Specific Prompting for Improved VLM Performance in Mathematical Reasoning Ayush Singh Mansi Gupta Shivank Garg Abhinav Kumar Vansh Agrawal ReLM LRM 24 0 0 08 Oct 2024
What Makes a Maze Look Like a Maze? Joy Hsu Jiayuan Mao J. Tenenbaum Noah D. Goodman Jiajun Wu OCL 52 6 0 12 Sep 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 40 13 0 08 Jul 2024
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales Tianyang Xu Shujin Wu Shizhe Diao Xiaoze Liu Xingyao Wang Yangyi Chen Jing Gao LRM 29 27 0 31 May 2024
A Theory for Length Generalization in Learning to Reason Changnan Xiao Bing Liu LRM 26 8 0 31 Mar 2024
CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual Prompt Tuning Ziyang Gong Fuhao Li Yupeng Deng Deblina Bhattacharjee Xianzheng Ma Xiangwei Zhu Zhenming Ji 66 9 0 26 Mar 2024
A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models Xiujie Song Mengyue Wu Ke Zhu Chunhao Zhang Yanyi Chen LRM ELM 29 3 0 28 Feb 2024
Executable Code Actions Elicit Better LLM Agents Xingyao Wang Yangyi Chen Lifan Yuan Yizhe Zhang Yunzhu Li Hao Peng Heng Ji ELM LLMAG LM&Ro 24 127 0 01 Feb 2024
Conditions for Length Generalization in Learning Reasoning Skills Changnan Xiao Bing Liu LRM 29 7 0 22 Nov 2023
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran 24 56 0 16 Nov 2023
Trustworthy Large Models in Vision: A Survey Ziyan Guo Li Xu Jun Liu MU 56 0 0 16 Nov 2023
CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets Lifan Yuan Yangyi Chen Xingyao Wang Yi Ren Fung Hao Peng Heng Ji LLMAG KELM 18 56 0 29 Sep 2023
ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT S. Ubani S. Polat Rodney D. Nielsen 84 51 0 27 Apr 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 206 2,232 0 22 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought Abulhair Saparov He He ELM LRM ReLM 116 270 0 03 Oct 2022
Unpacking Large Language Models with Conceptual Consistency Pritish Sahu Michael Cogswell Yunye Gong Ajay Divakaran LRM 79 16 0 29 Sep 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 207 1,089 0 20 Sep 2022
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango Aman Madaan Amir Yazdanbakhsh LRM 136 115 0 16 Sep 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning Jack Hessel Jena D. Hwang J. Park Rowan Zellers Chandra Bhagavatula Anna Rohrbach Kate Saenko Yejin Choi ReLM 139 48 0 10 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Mor Geva Yoav Goldberg Jonathan Berant 235 319 0 21 Aug 2019