Title
Language Is Not All You Need: Aligning Perception with Language Models Shaohan Huang Li Dong Wenhui Wang Y. Hao Saksham Singhal ... Johan Bjorck Vishrav Chaudhary Subhojit Som Xia Song Furu Wei VLM LRM MLLM 19 532 0 27 Feb 2023
Few-shot Multimodal Multitask Multilingual Learning Aman Chadha Vinija Jain 34 0 0 19 Feb 2023
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts Zhihong Chen Shizhe Diao Benyou Wang Guanbin Li Xiang Wan MedIm 17 29 0 17 Feb 2023
UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling Haoyu Lu Yuqi Huo Guoxing Yang Zhiwu Lu Wei Zhan M. Tomizuka Mingyu Ding 25 31 0 13 Feb 2023
Less is More: Selective Layer Finetuning with SubTuning Gal Kaplun Andrey Gurevich Tal Swisa Mazor David Shai Shalev-Shwartz Eran Malach 11 7 0 13 Feb 2023
Analyzing Multimodal Objectives Through the Lens of Generative Diffusion Guidance Chaerin Kong Nojun Kwak DiffM 26 2 0 10 Feb 2023
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning Zhuolin Yang Wei Ping Zihan Liu V. Korthikanti Weili Nie ... Yuke Zhu M. Shoeybi Bryan Catanzaro Chaowei Xiao Anima Anandkumar VLM RALM 34 38 0 09 Feb 2023
SimCon Loss with Multiple Views for Text Supervised Semantic Segmentation Yash J. Patel Yusheng Xie Yi Zhu Srikar Appalaraju R. Manmatha 19 4 0 07 Feb 2023
AIM: Adapting Image Models for Efficient Video Action Recognition Taojiannan Yang Yi Zhu Yusheng Xie Aston Zhang C. L. P. Chen Mu Li ViT 39 143 0 06 Feb 2023
ADAPT: Action-aware Driving Caption Transformer Bu Jin Xinyi Liu Yupeng Zheng Pengfei Li Hao Zhao Tong Zhang Yuhang Zheng Guyue Zhou Jingjing Liu 17 69 0 01 Feb 2023
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video Haiyang Xu Qinghao Ye Mingshi Yan Yaya Shi Jiabo Ye ... Guohai Xu Ji Zhang Songfang Huang Feiran Huang Jingren Zhou MLLM VLM MoE 25 158 0 01 Feb 2023
UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers Dachuan Shi Chaofan Tao Ying Jin Zhendong Yang Chun Yuan Jiaqi Wang VLM ViT 18 38 0 31 Jan 2023
Advancing Radiograph Representation Learning with Masked Record Modeling Hong-Yu Zhou Chenyu Lian Lian-cheng Wang Yizhou Yu MedIm 20 55 0 30 Jan 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 265 4,223 0 30 Jan 2023
ClimaX: A foundation model for weather and climate Tung Nguyen Johannes Brandstetter Ashish Kapoor Jayesh K. Gupta Aditya Grover AI4Cl AI4CE 6 238 0 24 Jan 2023
Masked Autoencoding Does Not Help Natural Language Supervision at Scale Floris Weers Vaishaal Shankar Angelos Katharopoulos Yinfei Yang Tom Gunter CLIP 18 4 0 19 Jan 2023
PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets Shuo Sun Molei Qin Xinrun Wang Bo An FaML OffRL AIFin 19 4 0 14 Jan 2023
A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends Jie Gui Tuo Chen Jing Zhang Qiong Cao Zhe Sun Haoran Luo Dacheng Tao 29 120 0 13 Jan 2023
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Xinsong Zhang Yan Zeng Jipeng Zhang Hang Li VLM AI4CE LRM 14 17 0 12 Jan 2023
Vision Transformers Are Good Mask Auto-Labelers Shiyi Lan Xitong Yang Zhiding Yu Zuxuan Wu J. Álvarez Anima Anandkumar ISeg ViT MedIm 24 19 0 10 Jan 2023
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token Jia Ning Chen Li Zheng-Wei Zhang Zigang Geng Qi Dai Kun He Han Hu 33 43 0 05 Jan 2023
Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling Peng Wu Li Chen Hongyang Li Xiaosong Jia Junchi Yan Yu Qiao 84 28 0 03 Jan 2023
Disjoint Masking with Joint Distillation for Efficient Masked Image Modeling Xin Ma Chang-Shu Liu Chunyu Xie Long Ye Yafeng Deng Xiang Ji 20 9 0 31 Dec 2022
On Realization of Intelligent Decision-Making in the Real World: A Foundation Decision Model Perspective Ying Wen Ziyu Wan M. Zhou Shufang Hou Zhe Cao Chenyang Le Jingxiao Chen Zheng Tian Weinan Zhang J. Wang AI4CE 16 10 0 24 Dec 2022
Do DALL-E and Flamingo Understand Each Other? Hang Li Jindong Gu Rajat Koner Sahand Sharifzadeh Volker Tresp MLLM 16 12 0 23 Dec 2022
Reversible Column Networks Yuxuan Cai Yi Zhou Qi Han Jianjian Sun Xiangwen Kong Jun Yu Li Xiangyu Zhang VLM 29 53 0 22 Dec 2022
Generalized Decoding for Pixel, Image, and Language Xueyan Zou Zi-Yi Dou Jianwei Yang Zhe Gan Linjie Li ... Lu Yuan Nanyun Peng Lijuan Wang Yong Jae Lee Jianfeng Gao VLM MLLM ObjD 13 238 0 21 Dec 2022
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning Zhiyang Xu Ying Shen Lifu Huang MLLM 19 110 0 21 Dec 2022
A Length-Extrapolatable Transformer Yutao Sun Li Dong Barun Patra Shuming Ma Shaohan Huang Alon Benhaim Vishrav Chaudhary Xia Song Furu Wei 24 115 0 20 Dec 2022
Position-guided Text Prompt for Vision-Language Pre-training Alex Jinpeng Wang Pan Zhou Mike Zheng Shou Shuicheng Yan VLM 19 37 0 19 Dec 2022
Transferring General Multimodal Pretrained Models to Text Recognition Junyang Lin Xuancheng Ren Yichang Zhang Gao Liu Peng Wang An Yang Chang Zhou 32 4 0 19 Dec 2022
BEATs: Audio Pre-Training with Acoustic Tokenizers Sanyuan Chen Yu-Huan Wu Chengyi Wang Shujie Liu Daniel C. Tompkins Zhuo Chen Furu Wei 30 253 0 18 Dec 2022
Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning? Runpei Dong Zekun Qi Linfeng Zhang Junbo Zhang Jian‐Yuan Sun Zheng Ge Li Yi Kaisheng Ma ViT 3DPC 19 83 0 16 Dec 2022
Objaverse: A Universe of Annotated 3D Objects Matt Deitke Dustin Schwenk Jordi Salvador Luca Weihs Oscar Michel Eli VanderBilt Ludwig Schmidt Kiana Ehsani Aniruddha Kembhavi Ali Farhadi 22 882 0 15 Dec 2022
CLIPPO: Image-and-Language Understanding from Pixels Only Michael Tschannen Basil Mustafa N. Houlsby CLIP VLM 24 47 0 15 Dec 2022
Quant 4.0: Engineering Quantitative Investment with Automated, Explainable and Knowledge-driven Artificial Intelligence Jian Guo Sai Wang L. Ni H. Shum AIFin 13 7 0 13 Dec 2022
REVEAL: Retrieval-Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowledge Memory Ziniu Hu Ahmet Iscen Chen Sun Zirui Wang Kai-Wei Chang Yizhou Sun Cordelia Schmid David A. Ross Alireza Fathi RALM VLM 30 88 0 10 Dec 2022
MAGVIT: Masked Generative Video Transformer Lijun Yu Yong Cheng Kihyuk Sohn José Lezama Han Zhang ... Alexander G. Hauptmann Ming-Hsuan Yang Yuan Hao Irfan Essa Lu Jiang DiffM VGen 22 223 0 10 Dec 2022
VindLU: A Recipe for Effective Video-and-Language Pretraining Feng Cheng Xizi Wang Jie Lei David J. Crandall Mohit Bansal Gedas Bertasius VLM 27 78 0 09 Dec 2022
Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learning Jishnu Mukhoti Tsung-Yu Lin Omid Poursaeed Rui Wang Ashish Shah Philip H. S. Torr Ser-Nam Lim VLM 30 79 0 09 Dec 2022
VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners Shen Yan Tao Zhu Zirui Wang Yuan Cao Mi Zhang Soham Ghosh Yonghui Wu Jiahui Yu VLM VGen 19 45 0 09 Dec 2022
Learning Video Representations from Large Language Models Yue Zhao Ishan Misra Philipp Krahenbuhl Rohit Girdhar VLM AI4TS 20 164 0 08 Dec 2022
OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models Jinze Bai Rui Men Han Yang Xuancheng Ren Kai Dang ... Wenhang Ge Jianxin Ma Junyang Lin Jingren Zhou Chang Zhou 37 15 0 08 Dec 2022
Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval Mustafa Shukor Nicolas Thome Matthieu Cord CLIP CoGe 19 8 0 08 Dec 2022
Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priors Zhentao Yu Zixin Yin Deyu Zhou Duomin Wang Finn Wong Baoyuan Wang DiffM 22 35 0 07 Dec 2022
InternVideo: General Video Foundation Models via Generative and Discriminative Learning Yi Wang Kunchang Li Yizhuo Li Yinan He Bingkun Huang ... Junting Pan Jiashuo Yu Yali Wang Limin Wang Yu Qiao VLM VGen 38 309 0 06 Dec 2022
Unifying Vision, Text, and Layout for Universal Document Processing Zineng Tang Ziyi Yang Guoxin Wang Yuwei Fang Yang Liu Chenguang Zhu Michael Zeng Chao-Yue Zhang Mohit Bansal VLM 28 105 0 05 Dec 2022
Compound Tokens: Channel Fusion for Vision-Language Representation Learning Maxwell Mbabilla Aladago A. Piergiovanni 19 1 0 02 Dec 2022
Masked Contrastive Pre-Training for Efficient Video-Text Retrieval Fangxun Shu Biaolong Chen Yue Liao Shuwen Xiao Wenyu Sun Xiaobo Li Yousong Zhu Jinqiao Wang Si Liu CLIP 25 11 0 02 Dec 2022
Synaptic Dynamics Realize First-order Adaptive Learning and Weight Symmetry Yukun Yang Peng Li ODL 28 1 0 01 Dec 2022