Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP

27 August 2023

Papers citing "Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP"

23 / 23 papers shown

Title
Exploring the Effectiveness and Interpretability of Texts in LLM-based Time Series Models Zhengke Sun Hangwei Qian Ivor Tsang AI4TS 29 0 0 09 Apr 2025
Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Models Ruta Binkyte Ivaxi Sheth Zhijing Jin Mohammad Havaei Bernhard Schölkopf Mario Fritz 70 0 0 28 Feb 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 86 10 0 06 Jan 2025
Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models Ido Cohen Daniela Gottesman Mor Geva Raja Giryes VLM 82 0 1 18 Dec 2024
Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis Po-Hsuan Huang Jeng-Lin Li Chin-Po Chen Ming-Ching Chang Wei-Chao Chen LRM 72 1 0 04 Dec 2024
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey Yunkai Dang Kaichen Huang Jiahao Huo Yibo Yan S. Huang ... Kun Wang Yong Liu Jing Shao Hui Xiong Xuming Hu LRM 96 14 0 03 Dec 2024
Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens Zhangqi Jiang Junkai Chen Beier Zhu Tingjin Luo Yankun Shen Xu Yang 95 4 0 23 Nov 2024
Identifying Sub-networks in Neural Networks via Functionally Similar Representations Tian Gao Amit Dhurandhar K. Ramamurthy Dennis L. Wei 40 0 0 21 Oct 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip H. S. Torr Mor Geva David M. Krueger Fazl Barez 84 6 0 09 Oct 2024
Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations Nick Jiang Anish Kachinthaya Suzie Petryk Yossi Gandelsman VLM 32 14 0 03 Oct 2024
What Do VLMs NOTICE? A Mechanistic Interpretability Pipeline for Gaussian-Noise-free Text-Image Corruption and Evaluation Michal Golovanevsky William Rudman Vedant Palit Ritambhara Singh Carsten Eickhoff 31 1 0 24 Jun 2024
Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects Michael A. Lepori Alexa R. Tartaglini Wai Keen Vong Thomas Serre Brenden Lake Ellie Pavlick 34 2 0 22 Jun 2024
A Concept-Based Explainability Framework for Large Multimodal Models Jayneel Parekh Pegah Khayatan Mustafa Shukor A. Newson Matthieu Cord 29 16 0 12 Jun 2024
Mechanistic Interpretability for AI Safety -- A Review Leonard Bereska E. Gavves AI4CE 38 111 0 22 Apr 2024
Finding Visual Task Vectors Alberto Hojel Yutong Bai Trevor Darrell Amir Globerson Amir Bar 60 6 0 08 Apr 2024
Trustworthy Large Models in Vision: A Survey Ziyan Guo Li Xu Jun Liu MU 56 0 0 16 Nov 2023
Finding Neurons in a Haystack: Case Studies with Sparse Probing Wes Gurnee Neel Nanda Matthew Pauly Katherine Harvey Dmitrii Troitskii Dimitris Bertsimas MILM 153 186 0 02 May 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 210 486 0 01 Nov 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 453 0 24 Sep 2022
Toy Models of Superposition Nelson Elhage Tristan Hume Catherine Olsson Nicholas Schiefer T. Henighan ... Sam McCandlish Jared Kaplan Dario Amodei Martin Wattenberg C. Olah AAML MILM 120 314 0 21 Sep 2022
Linear Connectivity Reveals Generalization Strategies Jeevesh Juneja Rachit Bansal Kyunghyun Cho João Sedoc Naomi Saphra 232 45 0 24 May 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 388 4,010 0 28 Jan 2022
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 199 876 0 03 May 2018