GIT: A Generative Image-to-text Transformer for Vision and Language

27 May 2022

Zicheng Liu

Papers citing "GIT: A Generative Image-to-text Transformer for Vision and Language"

50 / 90 papers shown

Title
GIF: Generative Inspiration for Face Recognition at Scale Saeed Ebrahimi Sahar Rahimi Ali Dabouei Srinjoy Das Jeremy M. Dawson Nasser M. Nasrabadi CVBM 60 0 0 05 May 2025
T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT D. Jiang Ziyu Guo Renrui Zhang Zhuofan Zong Hao Li Le Zhuo Shilin Yan Pheng-Ann Heng H. Li LRM 57 0 0 01 May 2025
HoloDx: Knowledge- and Data-Driven Multimodal Diagnosis of Alzheimer's Disease Qiuhui Chen Jintao Wang Gang Wang Yi Hong 39 0 0 27 Apr 2025
LLM-based Semantic Augmentation for Harmful Content Detection Elyas Meguellati Assaad Zeghina S. Sadiq Gianluca Demartini 32 0 0 22 Apr 2025
Large-scale Pre-training for Grounded Video Caption Generation Evangelos Kazakos Cordelia Schmid Josef Sivic 52 0 0 13 Mar 2025
HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding Shehreen Azad Vibhav Vineet Y. S. Rawat VLM 58 1 0 11 Mar 2025
LangGas: Introducing Language in Selective Zero-Shot Background Subtraction for Semi-Transparent Gas Leak Detection with a New Dataset Wenqi Guo Yiyang Du Shan Du 67 1 0 04 Mar 2025
Quantifying Memorization and Retriever Performance in Retrieval-Augmented Vision-Language Models Peter Carragher Abhinand Jha R Raghav Kathleen M. Carley RALM 73 0 0 20 Feb 2025
Pretrained Image-Text Models are Secretly Video Captioners Chunhui Zhang Yiren Jian Z. Ouyang Soroush Vosoughi VLM 63 3 0 20 Feb 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang D. Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 104 102 0 10 Jan 2025
Altogether: Image Captioning via Re-aligning Alt-text Hu Xu Po-Yao (Bernie) Huang Xiaoqing Ellen Tan Ching-Feng Yeh Jacob Kahn ... Luke Zettlemoyer Wen-tau Yih Shang-Wen Li Saining Xie Christoph Feichtenhofer DiffM 36 6 0 31 Dec 2024
Neptune: The Long Orbit to Benchmarking Long Video Understanding Arsha Nagrani Mingda Zhang Ramin Mehran Rachel Hornung N. B. Gundavarapu ... Boqing Gong Cordelia Schmid Mikhail Sirotenko Yukun Zhu Tobias Weyand 98 4 0 12 Dec 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 52 2 0 14 Nov 2024
Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devices Kilian Pfeiffer Mohamed Aboelenien Ahmed R. Khalili J. Henkel 25 0 0 12 Nov 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 66 25 0 04 Oct 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 22 3 0 11 Sep 2024
Enhancing Long Video Understanding via Hierarchical Event-Based Memory Dingxin Cheng Mingda Li Jingyu Liu Yongxin Guo Bin Jiang Qingbin Liu Xi Chen Bo Zhao 22 4 0 10 Sep 2024
Modelling Visual Semantics via Image Captioning to extract Enhanced Multi-Level Cross-Modal Semantic Incongruity Representation with Attention for Multimodal Sarcasm Detection Sajal Aggarwal Ananya Pandey Dinesh Kumar Vishwakarma 36 1 0 05 Aug 2024
ExpertAF: Expert Actionable Feedback from Video Kumar Ashutosh Tushar Nagarajan Georgios Pavlakos Kris M. Kitani Kristen Grauman VGen 42 2 0 01 Aug 2024
Learning Visual Grounding from Generative Vision and Language Model Shijie Wang Dahun Kim A. Taalimi Chen Sun Weicheng Kuo ObjD 32 5 0 18 Jul 2024
Controllable Contextualized Image Captioning: Directing the Visual Narrative through User-Defined Highlights Shunqi Mao Chaoyi Zhang Hang Su Hwanjun Song Igor Shalyminov Weidong Cai 26 1 0 16 Jul 2024
Vision-Language Models under Cultural and Inclusive Considerations Antonia Karamolegkou Phillip Rust Yong Cao Ruixiang Cui Anders Søgaard Daniel Hershcovich VLM 45 7 0 08 Jul 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 29 52 0 30 Jun 2024
NarrativeBridge: Enhancing Video Captioning with Causal-Temporal Narrative Asmar Nadeem Faegheh Sardari R. Dawes Syed Sameed Husain Adrian Hilton Armin Mustafa 47 4 0 10 Jun 2024
Multi-Modal Generative Embedding Model Feipeng Ma Hongwei Xue Guangting Wang Yizhou Zhou Fengyun Rao Shilin Yan Yueyi Zhang Siying Wu Mike Zheng Shou Xiaoyan Sun VLM 26 3 0 29 May 2024
How Culturally Aware are Vision-Language Models? Olena Burda-Lassen Aman Chadha Shashank Goswami Vinija Jain VLM 31 0 0 24 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 62 38 0 23 May 2024
Iteratively Prompting Multimodal LLMs to Reproduce Natural and AI-Generated Images Ali Naseh Katherine Thai Mohit Iyyer Amir Houmansadr 20 5 0 21 Apr 2024
ECOR: Explainable CLIP for Object Recognition Ali Rasekh Sepehr Kazemi Ranjbar Milad Heidari Wolfgang Nejdl VLM 33 4 0 19 Apr 2024
ViTextVQA: A Large-Scale Visual Question Answering Dataset for Evaluating Vietnamese Text Comprehension in Images Quan Van Nguyen Dan Quang Tran Huy Quang Pham Thang Kien-Bao Nguyen Nghia Hieu Nguyen Kiet Van Nguyen N. Nguyen CoGe 35 3 0 16 Apr 2024
Embodied Understanding of Driving Scenarios Yunsong Zhou Linyan Huang Qingwen Bu Jia Zeng Tianyu Li Hang Qiu Hongzi Zhu Minyi Guo Yu Qiao Hongyang Li LM&Ro 47 30 0 07 Mar 2024
Polos: Multimodal Metric Learning from Human Feedback for Image Captioning Yuiga Wada Kanta Kaneda Daichi Saito Komei Sugiura 22 24 0 28 Feb 2024
SnapCap: Efficient Snapshot Compressive Video Captioning Jianqiao Sun Yudi Su Hao Zhang Ziheng Cheng Zequn Zeng Zhengjue Wang Bo Chen Xin Yuan 22 1 0 10 Jan 2024
AliFuse: Aligning and Fusing Multi-modal Medical Data for Computer-Aided Diagnosis Qiuhui Chen Yi Hong MedIm 13 1 0 02 Jan 2024
RCA-NOC: Relative Contrastive Alignment for Novel Object Captioning Jiashuo Fan Yaoyuan Liang Leyao Liu Shao-Lun Huang Lei Zhang 27 2 0 11 Dec 2023
Mitigating Open-Vocabulary Caption Hallucinations Assaf Ben-Kish Moran Yanuka Morris Alper Raja Giryes Hadar Averbuch-Elor MLLM VLM 11 6 0 06 Dec 2023
Uni3DL: Unified Model for 3D and Language Understanding Xiang Li Jian Ding Zhaoyang Chen Mohamed Elhoseiny 24 3 0 05 Dec 2023
Semantic and Expressive Variation in Image Captions Across Languages Andre Ye Sebastin Santy Jena D. Hwang Amy X. Zhang Ranjay Krishna VLM 41 3 0 22 Oct 2023
SILC: Improving Vision Language Pretraining with Self-Distillation Muhammad Ferjad Naeem Yongqin Xian Xiaohua Zhai Lukas Hoyer Luc Van Gool F. Tombari VLM 17 32 0 20 Oct 2023
Visual Grounding Helps Learn Word Meanings in Low-Data Regimes Chengxu Zhuang Evelina Fedorenko Jacob Andreas 18 10 0 20 Oct 2023
Few-shot Action Recognition with Captioning Foundation Models Xiang Wang Shiwei Zhang Hangjie Yuan Yingya Zhang Changxin Gao Deli Zhao Nong Sang VLM 16 7 0 16 Oct 2023
VLIS: Unimodal Language Models Guide Multimodal Language Generation Jiwan Chung Youngjae Yu VLM 22 1 0 15 Oct 2023
Towards Robust Multi-Modal Reasoning via Model Selection Xiangyan Liu Rongxue Li Wei Ji Tao Lin LLMAG LRM 22 3 0 12 Oct 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 58 222 0 26 Sep 2023
Object-Centric Open-Vocabulary Image-Retrieval with Aggregated Features Hila Levi Guy Heller Dan Levi Ethan Fetaya OCL VLM 14 3 0 26 Sep 2023
SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap Daehee Kim Yoon Kim Donghyun Kim Yumin Lim Geewook Kim Taeho Kil 21 3 0 21 Sep 2023
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning Bang-ju Yang Fenglin Liu X. Wu Yaowei Wang Xu Sun Yuexian Zou VLM CLIP 22 13 0 25 Aug 2023
EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE Junyi Chen Longteng Guo Jianxiang Sun Shuai Shao Zehuan Yuan Liang Lin Dongyu Zhang MLLM VLM MoE 35 9 0 23 Aug 2023
ViCo: Engaging Video Comment Generation with Human Preference Rewards Yuchong Sun Bei Liu Xu Chen Ruihua Song Jianlong Fu VGen 18 2 0 22 Aug 2023
SpeechX: Neural Codec Language Model as a Versatile Speech Transformer Xiaofei Wang Manthan Thakker Zhuo Chen Naoyuki Kanda Sefik Emre Eskimez Sanyuan Chen M. Tang Shujie Liu Jinyu Li Takuya Yoshioka 13 79 0 14 Aug 2023