GIT: A Generative Image-to-text Transformer for Vision and Language

27 May 2022

Zicheng Liu

Papers citing "GIT: A Generative Image-to-text Transformer for Vision and Language"

40 / 90 papers shown

Title
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang F. Khan VLM 13 116 0 25 Jul 2023
Towards a Visual-Language Foundation Model for Computational Pathology Ming Y. Lu Bowen Chen Drew F. K. Williamson Richard J. Chen Ivy Liang ... Andrew Zhang L. Le Georg Gerber Anil V. Parwani Faisal Mahmood VLM MedIm 20 46 0 24 Jul 2023
Multi-Similarity Contrastive Learning Emily Mu John Guttag Maggie Makar SSL 23 2 0 06 Jul 2023
VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution S. Hall F. G. Abrantes Hanwen Zhu Grace A. Sodunke Aleksandar Shtedritski Hannah Rose Kirk CoGe 11 38 0 21 Jun 2023
Improving Image Captioning Descriptiveness by Ranking and LLM-based Fusion Simone Bianco Luigi Celona Marco Donzella Paolo Napoletano 24 18 0 20 Jun 2023
A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics Hong-Yu Zhou Yizhou Yu Chengdi Wang Shu Zhen Zhang Yuanxu Gao Jia-Yu Pan Jun Shao Guangming Lu Kang Zhang Weimin Li MedIm 11 149 0 01 Jun 2023
Joint Adaptive Representations for Image-Language Learning A. Piergiovanni A. Angelova VLM 14 0 0 31 May 2023
PaLI-X: On Scaling up a Multilingual Vision and Language Model Xi Chen Josip Djolonga Piotr Padlewski Basil Mustafa Soravit Changpinyo ... Mojtaba Seyedhosseini A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut VLM 33 186 0 29 May 2023
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning Chia-Wen Kuo Z. Kira 25 21 0 25 May 2023
DiffCap: Exploring Continuous Diffusion on Image Captioning Yufeng He Zefan Cai Xu Gan Baobao Chang DiffM 19 5 0 20 May 2023
Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner Zikang Liu Sihan Chen Longteng Guo Handong Li Xingjian He J. Liu 6 1 0 19 May 2023
Brain Captioning: Decoding human brain activity into images and text Matteo Ferrante Furkan Ozcelik T. Boccato R. V. Rullen N. Toschi DiffM 13 26 0 19 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 13 113 0 18 May 2023
OneCAD: One Classifier for All image Datasets using multimodal learning S. Wadekar Eugenio Culurciello 19 0 0 11 May 2023
OpenAGI: When LLM Meets Domain Experts Yingqiang Ge Wenyue Hua Kai Mei Jianchao Ji Juntao Tan Shuyuan Xu Zelong Li Yongfeng Zhang VLM LRM 22 206 0 10 Apr 2023
ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules Zhi-Qi Cheng Qianwen Dai Siyao Li Jingdong Sun Teruko Mitamura Alexander G. Hauptmann 19 21 0 05 Apr 2023
Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA Yongxin Zhu Z. Liu Yukang Liang Xin Li Hao Liu Changcun Bao Linli Xu 16 6 0 04 Apr 2023
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks Weicheng Kuo A. Piergiovanni Dahun Kim Xiyang Luo Benjamin Caine ... Luowei Zhou Andrew M. Dai Zhifeng Chen Claire Cui A. Angelova MLLM VLM 12 23 0 29 Mar 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 23 736 0 28 Mar 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 12 917 0 27 Mar 2023
VideoXum: Cross-modal Visual and Textural Summarization of Videos Jingyang Lin Hang Hua Ming Chen Yikang Li Jenhao Hsiao C. Ho Jiebo Luo 23 30 0 21 Mar 2023
Prompt Stealing Attacks Against Text-to-Image Generation Models Xinyue Shen Y. Qu Michael Backes Yang Zhang 17 31 0 20 Feb 2023
Position-guided Text Prompt for Vision-Language Pre-training Alex Jinpeng Wang Pan Zhou Mike Zheng Shou Shuicheng Yan VLM 11 37 0 19 Dec 2022
ReCo: Region-Controlled Text-to-Image Generation Zhengyuan Yang Jianfeng Wang Zhe Gan Linjie Li Kevin Qinghong Lin ... Nan Duan Zicheng Liu Ce Liu Michael Zeng Lijuan Wang DiffM 17 139 0 23 Nov 2022
Feedback is Needed for Retakes: An Explainable Poor Image Notification Framework for the Visually Impaired Kazuya Ohata Shunsuke Kitada Hitoshi Iyatomi 14 0 0 17 Nov 2022
PromptCap: Prompt-Guided Task-Aware Image Captioning Yushi Hu Hang Hua Zhengyuan Yang Weijia Shi Noah A. Smith Jiebo Luo 28 101 0 15 Nov 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 31 391 0 17 Jun 2022
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners Zhenhailong Wang Manling Li Ruochen Xu Luowei Zhou Jie Lei ... Chenguang Zhu Derek Hoiem Shih-Fu Chang Mohit Bansal Heng Ji MLLM VLM 167 134 0 22 May 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 385 4,010 0 28 Jan 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 169 401 0 10 Sep 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Mohit Bansal Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 185 403 0 13 Jul 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 1,077 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Mohit Bansal MLLM 249 518 0 04 Feb 2021
VinVL: Revisiting Visual Representations in Vision-Language Models Pengchuan Zhang Xiujun Li Xiaowei Hu Jianwei Yang Lei Zhang Lijuan Wang Yejin Choi Jianfeng Gao ObjD VLM 252 157 0 02 Jan 2021
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval Jie Lei Licheng Yu Tamara L. Berg Mohit Bansal 106 268 0 24 Jan 2020
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 250 922 0 24 Sep 2019
Controllable Video Captioning with POS Sequence Guidance Based on Gated Fusion Network Bairui Wang Lin Ma Wei Zhang Wenhao Jiang Jingwen Wang Wei Liu 63 158 0 27 Aug 2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,724 0 26 Sep 2016