12-in-1: Multi-Task Vision and Language Representation Learning

5 December 2019

Devi Parikh

Papers citing "12-in-1: Multi-Task Vision and Language Representation Learning"

50 / 90 papers shown

Title
Hadamard product in deep learning: Introduction, Advances and Challenges Grigorios G. Chrysos Yongtao Wu Razvan Pascanu Philip Torr V. Cevher AAML 96 0 0 17 Apr 2025
MASS: Overcoming Language Bias in Image-Text Matching Jiwan Chung Seungwon Lim Sangkyu Lee Youngjae Yu VLM 30 0 0 20 Jan 2025
Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios Shantanu Jaiswal Debaditya Roy Basura Fernando Cheston Tan ReLM LRM 71 2 0 20 Nov 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 22 3 0 11 Sep 2024
UniVS: Unified and Universal Video Segmentation with Prompts as Queries Ming-hui Li Shuai Li Xindong Zhang Lei Zhang VOS 33 16 0 28 Feb 2024
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 38 29 0 19 Dec 2023
LEAP: LLM-Generation of Egocentric Action Programs Eadom Dessalene Michael Maynord Cornelia Fermuller Yiannis Aloimonos 18 3 0 29 Nov 2023
Multiscale Superpixel Structured Difference Graph Convolutional Network for VL Representation Siyu Zhang Ye-Ting Chen Fang Wang Yaoru Sun Jun Yang Lizhi Bai SSL 22 0 0 20 Oct 2023
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models Noriyuki Kojima Hadar Averbuch-Elor Yoav Artzi 21 2 0 06 Sep 2023
GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training Xi Deng Han Shi Runhu Huang Changlin Li Hang Xu Jianhua Han James T. Kwok Shen Zhao Wei Zhang Xiaodan Liang CLIP VLM 29 3 0 22 Aug 2023
MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entities Y. Li Tingwei Lu Yinghui Li Tianyu Yu Shulin Huang Haitao Zheng Rui Zhang Jun Yuan 37 11 0 27 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 83 224 0 07 Jul 2023
Joint Adaptive Representations for Image-Language Learning A. Piergiovanni A. Angelova VLM 21 0 0 31 May 2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning Antoine Yang Arsha Nagrani Paul Hongsuck Seo Antoine Miech Jordi Pont-Tuset Ivan Laptev Josef Sivic Cordelia Schmid AI4TS VLM 23 220 0 27 Feb 2023
MINOTAUR: Multi-task Video Grounding From Multimodal Queries Raghav Goyal E. Mavroudi Xitong Yang Sainbayar Sukhbaatar Leonid Sigal Matt Feiszli Lorenzo Torresani Du Tran 8 7 0 16 Feb 2023
Effective End-to-End Vision Language Pretraining with Semantic Visual Loss Xiaofeng Yang Fayao Liu Guosheng Lin VLM 19 7 0 18 Jan 2023
Enhancing Multi-modal and Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation Qian Yang Qian Chen Wen Wang Baotian Hu Min Zhang 17 24 0 16 Dec 2022
Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation Maha Elbayad Anna Y. Sun Shruti Bhosale MoE 41 8 0 15 Dec 2022
PUnifiedNER: A Prompting-based Unified NER System for Diverse Datasets Jinghui Lu Rui Zhao Brian Mac Namee Fei Tan 19 18 0 27 Nov 2022
YORO -- Lightweight End to End Visual Grounding Chih-Hui Ho Srikar Appalaraju Bhavan A. Jasani R. Manmatha Nuno Vasconcelos ObjD 21 21 0 15 Nov 2022
Cold Start Streaming Learning for Deep Networks Cameron R. Wolfe Anastasios Kyrillidis CLL 15 2 0 09 Nov 2022
Image-Text Retrieval with Binary and Continuous Label Supervision Zheng Li Caili Guo Zerun Feng Jenq-Neng Hwang Ying Jin Yufeng Zhang VLM 20 4 0 20 Oct 2022
VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang Agrim Gupta Zichen Zhang Guanzhi Wang Yongqiang Dou Yanjun Chen Li Fei-Fei Anima Anandkumar Yuke Zhu Linxi Fan LM&Ro 15 334 0 06 Oct 2022
Highly Scalable Task Grouping for Deep Multi-Task Learning in Prediction of Epigenetic Events Mohammad Shiri Jiangwen Sun 11 1 0 24 Sep 2022
Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings Yiren Jian Chongyang Gao Soroush Vosoughi SSL 18 15 0 20 Sep 2022
Belief Revision based Caption Re-ranker with Visual Semantic Information Ahmed Sabir Francesc Moreno-Noguer Pranava Madhyastha Lluís Padró BDL 14 2 0 16 Sep 2022
ImageArg: A Multi-modal Tweet Dataset for Image Persuasiveness Mining Zhexiong Liu M. Guo Y. Dai Diane Litman 16 15 0 14 Sep 2022
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining Xiaoyi Dong Jianmin Bao Yinglin Zheng Ting Zhang Dongdong Chen ... Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu CLIP VLM 35 157 0 25 Aug 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 37 3 0 24 Aug 2022
Learning More May Not Be Better: Knowledge Transferability in Vision and Language Tasks Tianwei Chen Noa Garcia Mayu Otani Chenhui Chu Yuta Nakashima Hajime Nagahara VLM 22 0 0 23 Aug 2022
Masked Vision and Language Modeling for Multi-modal Representation Learning Gukyeong Kwon Zhaowei Cai Avinash Ravichandran Erhan Bas Rahul Bhotika Stefano Soatto 22 67 0 03 Aug 2022
Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics Xiaoyuan Guo Jiali Duan C.-C. Jay Kuo J. Gichoya Imon Banerjee VLM 14 1 0 31 Jul 2022
Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training Haoxuan You Luowei Zhou Bin Xiao Noel Codella Yu Cheng Ruochen Xu Shih-Fu Chang Lu Yuan CLIP VLM 19 47 0 26 Jul 2022
Don't Stop Learning: Towards Continual Learning for the CLIP Model Yuxuan Ding Lingqiao Liu Chunna Tian Jingyuan Yang Haoxuan Ding CLL VLM KELM 19 50 0 19 Jul 2022
DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detection Yunhao Ge Jiashu Xu Brian Nlong Zhao Neel Joshi Laurent Itti Vibhav Vineet DiffM ObjD 25 16 0 20 Jun 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 45 391 0 17 Jun 2022
OmniMAE: Single Model Masked Pretraining on Images and Videos Rohit Girdhar Alaaeldin El-Nouby Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra ViT 27 97 0 16 Jun 2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid 34 226 0 16 Jun 2022
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling Linjie Li Zhe Gan Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Ce Liu Lijuan Wang MLLM VLM 18 81 0 14 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 41 522 0 13 Jun 2022
VLUE: A Multi-Task Benchmark for Evaluating Vision-Language Models Wangchunshu Zhou Yan Zeng Shizhe Diao Xinsong Zhang CoGe VLM 17 13 0 30 May 2022
Voxel-informed Language Grounding Rodolfo Corona Shizhan Zhu Dan Klein Trevor Darrell 133 11 0 19 May 2022
Learning to Answer Visual Questions from Web Videos Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 28 33 0 10 May 2022
All You May Need for VQA are Image Captions Soravit Changpinyo Doron Kukliansky Idan Szpektor Xi Chen Nan Ding Radu Soricut 30 70 0 04 May 2022
Answer-Me: Multi-Task Open-Vocabulary Visual Question Answering A. Piergiovanni Wei Li Weicheng Kuo M. Saffar Fred Bertsch A. Angelova 17 16 0 02 May 2022
Vision-and-Language Pretrained Models: A Survey Siqu Long Feiqi Cao S. Han Haiqing Yang VLM 16 63 0 15 Apr 2022
MultiMAE: Multi-modal Multi-task Masked Autoencoders Roman Bachmann David Mizrahi Andrei Atanov Amir Zamir 22 265 0 04 Apr 2022
TubeDETR: Spatio-Temporal Video Grounding with Transformers Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 28 94 0 30 Mar 2022
Single-Stream Multi-Level Alignment for Vision-Language Pretraining Zaid Khan B. Vijaykumar Xiang Yu S. Schulter Manmohan Chandraker Y. Fu CLIP VLM 20 16 0 27 Mar 2022
Multi-modal Alignment using Representation Codebook Jiali Duan Liqun Chen Son Tran Jinyu Yang Yi Xu Belinda Zeng Trishul M. Chilimbi 28 67 0 28 Feb 2022