12-in-1: Multi-Task Vision and Language Representation Learning

5 December 2019

Devi Parikh

Papers citing "12-in-1: Multi-Task Vision and Language Representation Learning"

40 / 90 papers shown

Title
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 82 212 0 18 Feb 2022
Webly Supervised Concept Expansion for General Purpose Vision Models Amita Kamath Christopher Clark Tanmay Gupta Eric Kolve Derek Hoiem Aniruddha Kembhavi VLM 19 54 0 04 Feb 2022
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages Emanuele Bugliarello Fangyu Liu Jonas Pfeiffer Siva Reddy Desmond Elliott E. Ponti Ivan Vulić MLLM VLM ELM 35 62 0 27 Jan 2022
LaTr: Layout-Aware Transformer for Scene-Text VQA Ali Furkan Biten Ron Litman Yusheng Xie Srikar Appalaraju R. Manmatha ViT 22 100 0 23 Dec 2021
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling Yang Li Gang Li Xin Zhou Mostafa Dehghani A. Gritsenko MLLM 25 33 0 10 Dec 2021
Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation Tianyi Liu Zuxuan Wu Wenhan Xiong Jingjing Chen Yu-Gang Jiang VLM MLLM 32 10 0 10 Dec 2021
Injecting Semantic Concepts into End-to-End Image Captioning Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lin Liang Zhe Gan Lijuan Wang Yezhou Yang Zicheng Liu ViT VLM 19 86 0 09 Dec 2021
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 38 686 0 08 Dec 2021
VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition Changyao Tian Wenhai Wang Xizhou Zhu Jifeng Dai Yu Qiao VLM 30 69 0 26 Nov 2021
PolyViT: Co-training Vision Transformers on Images, Videos and Audio Valerii Likhosherstov Anurag Arnab K. Choromanski Mario Lucic Yi Tay Adrian Weller Mostafa Dehghani ViT 33 73 0 25 Nov 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin W. Wang Lijuan Wang Zicheng Liu VLM 34 216 0 24 Nov 2021
Achieving Human Parity on Visual Question Answering Ming Yan Haiyang Xu Chenliang Li Junfeng Tian Bin Bi ... Ji Zhang Songfang Huang Fei Huang Luo Si Rong Jin 24 12 0 17 Nov 2021
TriBERT: Full-body Human-centric Audio-visual Representation Learning for Visual Sound Separation Tanzila Rahman Mengyu Yang Leonid Sigal ViT 21 8 0 26 Oct 2021
Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers Stella Frank Emanuele Bugliarello Desmond Elliott 30 81 0 09 Sep 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq R. Joty Caiming Xiong S. Hoi FaML 45 1,881 0 16 Jul 2021
End-to-end Multi-modal Video Temporal Grounding Yi-Wen Chen Yi-Hsuan Tsai Ming-Hsuan Yang 11 51 0 12 Jul 2021
Adventurer's Treasure Hunt: A Transparent System for Visually Grounded Compositional Visual Question Answering based on Scene Graphs Daniel Reich F. Putze Tanja Schultz 22 2 0 28 Jun 2021
Core Challenges in Embodied Vision-Language Planning Jonathan M Francis Nariaki Kitamura Felix Labelle Xiaopeng Lu Ingrid Navarro Jean Oh LM&Ro 39 45 0 26 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 24 811 0 14 Jun 2021
Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations? Thierry Deruyttere Victor Milewski Marie-Francine Moens 28 15 0 08 Jun 2021
M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers Zhu Zhang Jianxin Ma Chang Zhou Rui Men Zhikang Li Ming Ding Jie Tang Jingren Zhou Hongxia Yang 25 46 0 29 May 2021
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding Aishwarya Kamath Mannat Singh Yann LeCun Gabriel Synnaeve Ishan Misra Nicolas Carion ObjD VLM 49 858 0 26 Apr 2021
Playing Lottery Tickets with Vision and Language Zhe Gan Yen-Chun Chen Linjie Li Tianlong Chen Yu Cheng Shuohang Wang Jingjing Liu Lijuan Wang Zicheng Liu VLM 101 53 0 23 Apr 2021
CLIPScore: A Reference-free Evaluation Metric for Image Captioning Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 13 1,434 0 18 Apr 2021
Compressing Visual-linguistic Model via Knowledge Distillation Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lijuan Wang Yezhou Yang Zicheng Liu VLM 28 96 0 05 Apr 2021
Scene-Intuitive Agent for Remote Embodied Visual Grounding Xiangru Lin Guanbin Li Yizhou Yu LM&Ro 16 52 0 24 Mar 2021
Few-Shot Visual Grounding for Natural Human-Robot Interaction Georgios Tziafas S. Kasaei 11 6 0 17 Mar 2021
On the Role of Images for Analyzing Claims in Social Media Gullal Singh Cheema Sherzod Hakimov Eric Müller-Budack Ralph Ewerth 11 10 0 17 Mar 2021
What is Multimodality? Letitia Parcalabescu Nils Trost Anette Frank 16 0 0 10 Mar 2021
Pretrained Transformers as Universal Computation Engines Kevin Lu Aditya Grover Pieter Abbeel Igor Mordatch 21 217 0 09 Mar 2021
Causal Attention for Vision-Language Tasks Xu Yang Hanwang Zhang Guojun Qi Jianfei Cai CML 23 148 0 05 Mar 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Mohit Bansal MLLM 249 525 0 04 Feb 2021
Detecting 32 Pedestrian Attributes for Autonomous Vehicles Taylor Mordan Matthieu Cord P. Pérez Alexandre Alahi 22 22 0 04 Dec 2020
Multimodal Pretraining Unmasked: A Meta-Analysis and a Unified Framework of Vision-and-Language BERTs Emanuele Bugliarello Ryan Cotterell Naoaki Okazaki Desmond Elliott 22 119 0 30 Nov 2020
Large-Scale Adversarial Training for Vision-and-Language Representation Learning Zhe Gan Yen-Chun Chen Linjie Li Chen Zhu Yu Cheng Jingjing Liu ObjD VLM 24 487 0 11 Jun 2020
Entity-Enriched Neural Models for Clinical Question Answering Bhanu Pratap Singh Rawat W. Weng So Yeon Min Preethi Raghavan Peter Szolovits 18 18 0 13 May 2020
Probing Contextual Language Models for Common Ground with Visual Representations Gabriel Ilharco Rowan Zellers Ali Farhadi Hannaneh Hajishirzi 22 14 0 01 May 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 41 491 0 01 May 2020
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 250 927 0 24 Sep 2019
Neural Baby Talk Jiasen Lu Jianwei Yang Dhruv Batra Devi Parikh VLM 189 434 0 27 Mar 2018