SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality

26 June 2023

Papers citing "SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality"

24 / 24 papers shown

Title
MINERVA: Evaluating Complex Video Reasoning Arsha Nagrani Sachit Menon Ahmet Iscen Shyamal Buch Ramin Mehran ... Yukun Zhu Carl Vondrick Mikhail Sirotenko Cordelia Schmid Tobias Weyand 56 0 0 01 May 2025
Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers Quentin Guimard Moreno DÍncà Massimiliano Mancini Elisa Ricci SSL 72 0 0 29 Apr 2025
Decoupled Global-Local Alignment for Improving Compositional Understanding Xiaoxing Hu Kaicheng Yang J. Z. Wang Haoran Xu Ziyong Feng Y. Wang VLM 89 0 0 23 Apr 2025
Enhancing Vision-Language Compositional Understanding with Multimodal Synthetic Data Haoxin Li Boyang Li CoGe 69 0 0 03 Mar 2025
CLIP Under the Microscope: A Fine-Grained Analysis of Multi-Object Representation Reza Abbasi Ali Nazari Aminreza Sefid Mohammadali Banayeeanzade M. Rohban M. Baghshah VLM 73 1 0 27 Feb 2025
From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models Mayank Vatsa Aparna Bharati S. Mittal Richa Singh 53 0 0 10 Feb 2025
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 70 0 0 02 Dec 2024
Sparse Attention Vectors: Generative Multimodal Model Features Are Discriminative Vision-Language Classifiers Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 121 4 0 28 Nov 2024
NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples Baiqi Li Zhiqiu Lin Wenxuan Peng Jean de Dieu Nyandwi Daniel Jiang Zixian Ma Simran Khanuja Ranjay Krishna Graham Neubig Deva Ramanan AAML CoGe VLM 61 20 0 18 Oct 2024
Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad VLM 16 2 0 16 Oct 2024
Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective Xiangru Zhu Penglei Sun Yaoxian Song Yanghua Xiao Zhixu Li Chengyu Wang Jun Huang Bei Yang Xiaoxiao Xu EGVM 120 1 0 14 Oct 2024
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension Junzhuo Liu X. Yang Weiwei Li Peng Wang ObjD 44 3 0 23 Sep 2024
No Detail Left Behind: Revisiting Self-Retrieval for Fine-Grained Image Captioning Manu Gaur Darshan Singh Makarand Tapaswi 80 1 0 04 Sep 2024
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions Yu-Guan Hsieh Cheng-Yu Hsieh Shih-Ying Yeh Louis Béthune Hadi Pour Ansari Pavan Kumar Anasosalu Vasu Chun-Liang Li Ranjay Krishna Oncel Tuzel Marco Cuturi 58 4 0 09 Jul 2024
Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Models Reza Abbasi M. Rohban M. Baghshah CoGe 38 5 0 08 Jul 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks Jacob Russin Sam Whitman McGrath Danielle J. Williams Lotem Elber-Dorozko AI4CE 61 3 0 24 May 2024
Pre-trained Vision-Language Models Learn Discoverable Visual Concepts Yuan Zang Tian Yun Hao Tan Trung Bui Chen Sun VLM CoGe 45 9 0 19 Apr 2024
Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement Zaid Khan B. Vijaykumar S. Schulter Yun Fu Manmohan Chandraker LRM ReLM 26 6 0 06 Apr 2024
Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationships Sebastian Koch Narunas Vaskevicius Mirco Colosi Pedro Hermosilla Timo Ropinski 3DPC 28 25 0 19 Feb 2024
An Examination of the Compositionality of Large Generative Vision-Language Models Teli Ma Rong Li Junwei Liang CoGe 19 2 0 21 Aug 2023
Compositional diversity in visual concept learning Yanli Zhou Reuben Feinman Brenden Lake CoGe OCL 24 8 0 30 May 2023
An Examination of the Robustness of Reference-Free Image Captioning Evaluation Metrics Saba Ahmadi Aishwarya Agrawal 17 6 0 24 May 2023
Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality Anuj Diwan Layne Berry Eunsol Choi David F. Harwath Kyle Mahowald CoGe 101 41 0 01 Nov 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 388 4,110 0 28 Jan 2022