EVA-CLIP: Improved Training Techniques for CLIP at Scale

27 March 2023

Papers citing "EVA-CLIP: Improved Training Techniques for CLIP at Scale"

50 / 357 papers shown

Title
Beyond Literal Descriptions: Understanding and Locating Open-World Objects Aligned with Human Intentions Wenxuan Wang Yisi Zhang Xingjian He Yichen Yan Zijia Zhao Xinlong Wang Jing Liu LM&Ro 20 4 0 17 Feb 2024
Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment Angelos Zavras Dimitrios Michail Begum Demir Ioannis Papoutsis VLM 17 11 0 15 Feb 2024
An Interactive Agent Foundation Model Zane Durante Bidipta Sarkar Ran Gong Rohan Taori Yusuke Noda ... Katsushi Ikeuchi Fei-Fei Li Jianfeng Gao Naoki Wake Qiuyuan Huang LM&Ro AI4CE LLMAG 89 14 0 08 Feb 2024
CREMA: Generalizable and Efficient Video-Language Reasoning via Multimodal Modular Fusion Shoubin Yu Jaehong Yoon Mohit Bansal 77 4 0 08 Feb 2024
EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters Quan-Sen Sun Jinsheng Wang Qiying Yu Yufeng Cui Fan Zhang Xiaosong Zhang Xinlong Wang VLM CLIP MLLM 81 38 0 06 Feb 2024
V-IRL: Grounding Virtual Intelligence in Real Life Jihan Yang Runyu Ding Ellis L Brown Xiaojuan Qi Saining Xie LM&Ro 48 18 0 05 Feb 2024
VIALM: A Survey and Benchmark of Visually Impaired Assistance with Large Models Yi Zhao Yilin Zhang Rong Xiang Jing Li Hillming Li 26 16 0 29 Jan 2024
M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining Qingpei Guo Furong Xu Hanxiao Zhang Wang Ren Ziping Ma Lin Ju Jian Wang Jingdong Chen Ming Yang VLM MLLM 25 2 0 29 Jan 2024
Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities Yiyuan Zhang Xiaohan Ding Kaixiong Gong Yixiao Ge Ying Shan Xiangyu Yue ViT 16 7 0 25 Jan 2024
Zoom-shot: Fast and Efficient Unsupervised Zero-Shot Transfer of CLIP to Vision Encoders with Multimodal Loss Jordan Shipard Arnold Wiliem Kien Nguyen Thanh Wei Xiang Clinton Fookes VLM CLIP 38 2 0 22 Jan 2024
Supervised Fine-tuning in turn Improves Visual Foundation Models Xiaohu Jiang Yixiao Ge Yuying Ge Dachuan Shi Chun Yuan Ying Shan VLM CLIP 38 8 0 18 Jan 2024
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer Changyao Tian Xizhou Zhu Yuwen Xiong Weiyun Wang Zhe Chen ... Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai AuLLM 83 40 0 18 Jan 2024
Instance Brownian Bridge as Texts for Open-vocabulary Video Instance Segmentation Ze-Long Cheng Kehan Li Hao Li Peng Jin Chang Liu Xiawu Zheng Rongrong Ji Jie Chen VOS 28 2 0 18 Jan 2024
Seeing the Unseen: Visual Common Sense for Semantic Placement Ram Ramrakhya Aniruddha Kembhavi Dhruv Batra Z. Kira Kuo-Hao Zeng Luca Weihs VLM 33 4 0 15 Jan 2024
UMG-CLIP: A Unified Multi-Granularity Vision Generalist for Open-World Understanding Bowen Shi Peisen Zhao Zichen Wang Yuhang Zhang Yaoming Wang ... Wenrui Dai Junni Zou Hongkai Xiong Qi Tian Xiaopeng Zhang VLM 33 6 0 12 Jan 2024
ModaVerse: Efficiently Transforming Modalities with LLMs Xinyu Wang Bohan Zhuang Qi Wu 11 11 0 12 Jan 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 18 281 0 11 Jan 2024
Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models Xin He Longhui Wei Lingxi Xie Qi Tian 43 8 0 06 Jan 2024
Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactively Haobo Yuan Xiangtai Li Chong Zhou Yining Li Kai Chen Chen Change Loy VLM 18 51 0 05 Jan 2024
SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment Ziping Ma Furong Xu Jian Liu Ming Yang Qingpei Guo VLM 34 3 0 04 Jan 2024
Few-shot Adaptation of Multi-modal Foundation Models: A Survey Fan Liu Tianshu Zhang Wenwen Dai Wenwen Cai Wenwen Cai Xiaocong Zhou Delong Chen VLM OffRL 20 22 0 03 Jan 2024
Morphing Tokens Draw Strong Masked Image Models Taekyung Kim Byeongho Heo Dongyoon Han 38 3 0 30 Dec 2023
MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices Xiangxiang Chu Limeng Qiao Xinyang Lin Shuang Xu Yang Yang ... Fei Wei Xinyu Zhang Bo-Wen Zhang Xiaolin Wei Chunhua Shen MLLM 26 32 0 28 Dec 2023
ChartBench: A Benchmark for Complex Visual Reasoning in Charts Zhengzhuo Xu Sinan Du Yiyan Qi Chengjin Xu Chun Yuan Jian Guo 33 33 0 26 Dec 2023
WebVLN: Vision-and-Language Navigation on Websites Qi Chen D. Pitawela Chongyang Zhao Gengze Zhou Hsiang-Ting Chen Qi Wu 21 8 0 25 Dec 2023
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 156 895 0 21 Dec 2023
Generative Multimodal Models are In-Context Learners Quan-Sen Sun Yufeng Cui Xiaosong Zhang Fan Zhang Qiying Yu ... Yueze Wang Yongming Rao Jingjing Liu Tiejun Huang Xinlong Wang MLLM LRM 45 244 0 20 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 38 29 0 19 Dec 2023
Text-Conditioned Resampler For Long Form Video Understanding Bruno Korbar Yongqin Xian A. Tonioni Andrew Zisserman Federico Tombari 25 12 0 19 Dec 2023
Tokenize Anything via Prompting Ting Pan Lulu Tang Xinlong Wang Shiguang Shan VLM 18 22 0 14 Dec 2023
SceneWiz3D: Towards Text-guided 3D Scene Composition Qihang Zhang Chaoyang Wang Aliaksandr Siarohin Peiye Zhuang Yinghao Xu Ceyuan Yang Dahua Lin Bolei Zhou Sergey Tulyakov Hsin-Ying Lee 20 31 0 13 Dec 2023
CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor Shuyang Sun Runjia Li Philip H. S. Torr Xiuye Gu Siyang Li VLM CLIP 20 32 0 12 Dec 2023
CLIP in Medical Imaging: A Comprehensive Survey Zihao Zhao Yuxiao Liu Han Wu Yonghao Li Sheng Wang L. Teng Disheng Liu Zhiming Cui Qian Wang Dinggang Shen CLIP MedIm LM&MA VLM 21 43 0 12 Dec 2023
Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokens Fan Ma Xiaojie Jin Heng Wang Yuchen Xian Jiashi Feng Yi Yang 13 47 0 12 Dec 2023
Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generator Henry Hengyuan Zhao Pan Zhou Mike Zheng Shou MLLM SyDa 33 7 0 11 Dec 2023
Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding Talfan Evans Shreya Pathak Hamza Merzic Jonathan Schwarz Ryutaro Tanno Olivier J. Hénaff 8 16 0 08 Dec 2023
User-Aware Prefix-Tuning is a Good Learner for Personalized Image Captioning Xuan Wang Guanhong Wang Wenhao Chai Jiayu Zhou Gaoang Wang 27 4 0 08 Dec 2023
Alpha-CLIP: A CLIP Model Focusing on Wherever You Want Zeyi Sun Ye Fang Tong Wu Pan Zhang Yuhang Zang Shu Kong Yuanjun Xiong Dahua Lin Jiaqi Wang VLM CLIP 25 81 0 06 Dec 2023
Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving Ming-Jun Nie Renyuan Peng Chunwei Wang Xinyue Cai Jianhua Han Hang Xu Li Zhang LRM 27 44 0 06 Dec 2023
Aligning and Prompting Everything All at Once for Universal Visual Perception Yunhang Shen Chaoyou Fu Peixian Chen Mengdan Zhang Ke Li Xing Sun Yunsheng Wu Shaohui Lin Rongrong Ji VLM ObjD 41 32 0 04 Dec 2023
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding Shuhuai Ren Linli Yao Shicheng Li Xu Sun Lu Hou VLM MLLM 10 174 0 04 Dec 2023
Towards Learning a Generalist Model for Embodied Navigation Duo Zheng Shijia Huang Lin Zhao Yiwu Zhong Liwei Wang LM&Ro 27 40 0 04 Dec 2023
Bootstrapping SparseFormers from Vision Foundation Models Ziteng Gao Zhan Tong K. Lin Joya Chen Mike Zheng Shou 23 0 0 04 Dec 2023
InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models Xunguang Wang Zhenlan Ji Pingchuan Ma Zongjie Li Shuai Wang MLLM 30 11 0 04 Dec 2023
Segment and Caption Anything Xiaoke Huang Jianfeng Wang Yansong Tang Zheng Zhang Han Hu Jiwen Lu Lijuan Wang Zicheng Liu MLLM VLM 21 17 0 01 Dec 2023
CAST: Cross-Attention in Space and Time for Video Action Recognition Dongho Lee Jongseo Lee Jinwoo Choi EgoV 25 10 0 30 Nov 2023
Zooming Out on Zooming In: Advancing Super-Resolution for Remote Sensing Piper Wolters F. Bastani Aniruddha Kembhavi 22 2 0 29 Nov 2023
VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models Shicheng Li Lei Li Shuhuai Ren Yuanxin Liu Yi Liu Rundong Gao Xu Sun Lu Hou 27 28 0 29 Nov 2023
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training Pavan Kumar Anasosalu Vasu Hadi Pouransari Fartash Faghri Raviteja Vemulapalli Oncel Tuzel CLIP VLM 11 43 0 28 Nov 2023
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang ... Jilan Xu Guo Chen Ping Luo Limin Wang Yu Qiao VLM MLLM 46 398 0 28 Nov 2023