ImageBind: One Embedding Space To Bind Them All

9 May 2023

Kalyan Vasudev Alwala

Papers citing "ImageBind: One Embedding Space To Bind Them All"

50 / 151 papers shown

Title
MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation T. Pham Tri Ton Chang D. Yoo 36 3 0 03 Oct 2024
Anchors Aweigh! Sail for Optimal Unified Multi-Modal Representations Minoh Jeong Min Namgung Zae Myung Kim Dongyeop Kang Yao-Yi Chiang Alfred Hero 25 0 0 02 Oct 2024
Image-guided topic modeling for interpretable privacy classification Alina Elena Baia Andrea Cavallaro 32 0 0 27 Sep 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 58 11 0 26 Sep 2024
Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa Kei Okada Masayuki Inaba LM&Ro 28 0 0 26 Sep 2024
A Simple but Strong Baseline for Sounding Video Generation: Effective Adaptation of Audio and Video Diffusion Models for Joint Generation Masato Ishii Akio Hayakawa Takashi Shibuya Yuki Mitsufuji VGen DiffM 63 4 0 26 Sep 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 67 21 0 26 Sep 2024
Measuring Sound Symbolism in Audio-visual Models Wei-Cheng Tseng Yi-Jen Shih David Harwath Raymond Mooney 32 0 0 18 Sep 2024
Enhancing Long Video Understanding via Hierarchical Event-Based Memory Dingxin Cheng Mingda Li Jingyu Liu Yongxin Guo Bin Jiang Qingbin Liu Xi Chen Bo Zhao 27 4 0 10 Sep 2024
Multi-Modal Adapter for Vision-Language Models Dominykas Seputis Serghei Mihailov Soham Chatterjee Zehao Xiao VLM 24 1 0 03 Sep 2024
YOLOO: You Only Learn from Others Once Lipeng Gu Mingqiang Wei Xuefeng Yan Dingkun Zhu Wei Zhao H. Xie Y. Liu 3DPC 29 0 0 01 Sep 2024
From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Space Andrew Hamara Pablo Rivas 16 1 0 30 Aug 2024
D&M: Enriching E-commerce Videos with Sound Effects by Key Moment Detection and SFX Matching Jingyu Liu Minquan Wang Ye Ma Bo Wang Aozhu Chen Quan Chen Peng Jiang Xirong Li 38 1 0 23 Aug 2024
PhishAgent: A Robust Multimodal Agent for Phishing Webpage Detection Tri Cao Chengyu Huang Yuexin Li Huilin Wang Amy He Nay Oo Bryan Hooi LLMAG OffRL 75 4 0 20 Aug 2024
A Markov Random Field Multi-Modal Variational AutoEncoder Fouad Oubari M. Baha Raphael Meunier Rodrigue Décatoire Mathilde Mougeot 36 0 0 18 Aug 2024
End-to-end Semantic-centric Video-based Multimodal Affective Computing Ronghao Lin Ying Zeng Sijie Mai Haifeng Hu VGen 40 0 0 14 Aug 2024
How Does Audio Influence Visual Attention in Omnidirectional Videos? Database and Model Yuxin Zhu Huiyu Duan Kaiwei Zhang Yucheng Zhu Xilei Zhu Long Teng Xiongkuo Min Guangtao Zhai 67 2 0 10 Aug 2024
Audio-visual training for improved grounding in video-text LLMs Shivprasad Sagare Hemachandran S Kinshuk Sarabhai Prashant Ullegaddi SA Rajeshkumar 27 0 0 21 Jul 2024
Large Visual-Language Models Are Also Good Classifiers: A Study of In-Context Multimodal Fake News Detection Ye Jiang Yimin Wang MLLM 38 1 0 16 Jul 2024
FabGPT: An Efficient Large Multimodal Model for Complex Wafer Defect Knowledge Queries Yuqi Jiang Xudong Lu Qian Jin Qi Sun Hanming Wu Cheng Zhuo 36 5 0 15 Jul 2024
Read, Watch and Scream! Sound Generation from Text and Video Yujin Jeong Yunji Kim Sanghyuk Chun Jiyoung Lee VGen DiffM 29 11 0 08 Jul 2024
Sequential Contrastive Audio-Visual Learning Ioannis Tsiamas Santiago Pascual Chunghsin Yeh Joan Serra 33 2 0 08 Jul 2024
GUIDE: A Guideline-Guided Dataset for Instructional Video Comprehension Jiafeng Liang Shixin Jiang Zekun Wang Haojie Pan Zerui Chen Zheng Chu Ming Liu Ruiji Fu Zhongyuan Wang Bing Qin 29 2 0 26 Jun 2024
Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Video Zhengbang Yang Haotian Xia Jingxi Li Zezhi Chen Zhuangdi Zhu Weining Shen ELM LRM 39 1 0 21 Jun 2024
AnoPatch: Towards Better Consistency in Machine Anomalous Sound Detection Anbai Jiang Bing Han Zhiqiang Lv Yufeng Deng Wei-Qiang Zhang Xie Chen Yanmin Qian Jia Liu Pingyi Fan 32 3 0 17 Jun 2024
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning Yunxin Li Xinyu Chen Baotian Hu Longyue Wang Haoyuan Shi Min-Ling Zhang MLLM LRM 42 25 0 17 Jun 2024
Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations Bulat Khaertdinov Pedro Jeuris Annanda Sousa Enrique Hortal 25 1 0 12 Jun 2024
Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning Shuvendu Roy Yasaman Parhizkar Franklin Ogidi Vahid Reza Khazaie Michael Colacci Ali Etemad Elham Dolatabadi Arash Afkanpour VLM 44 1 0 11 Jun 2024
FaceGPT: Self-supervised Learning to Chat about 3D Human Faces Haoran Wang Mohit Mendiratta Christian Theobalt Adam Kortylewski 3DH CVBM 31 3 0 11 Jun 2024
RU-AI: A Large Multimodal Dataset for Machine Generated Content Detection Liting Huang Zhihao Zhang Yiran Zhang Xiyue Zhou Shoujin Wang NoLa 38 2 0 07 Jun 2024
MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition Stefan Gerd Fritsch Cennet Oğuz Vitor Fortes Rey L. Ray Maximilian Kiefer-Emmanouilidis P. Lukowicz HAI 43 0 0 06 Jun 2024
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling Zeyue Tian Zhaoyang Liu Ruibin Yuan Jiahao Pan Xiaoqiang Huang Xu Tan Xu Tan Qifeng Chen Y. Guo VGen 100 16 0 06 Jun 2024
BeFA: A General Behavior-driven Feature Adapter for Multimedia Recommendation Qile Fan Penghang Yu Zhiyi Tan Bing-Kun Bao Guanming Lu 37 1 0 01 Jun 2024
X-VILA: Cross-Modality Alignment for Large Language Model Hanrong Ye De-An Huang Yao Lu Zhiding Yu Wei Ping ... Jan Kautz Song Han Dan Xu Pavlo Molchanov Hongxu Yin MLLM VLM 40 29 0 29 May 2024
OmniBind: Teach to Build Unequal-Scale Modality Interaction for Omni-Bind of All Yuanhuiyi Lyu Xueye Zheng Dahun Kim Lin Wang 44 10 0 25 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 71 41 0 23 May 2024
Images that Sound: Composing Images and Sounds on a Single Canvas Ziyang Chen Daniel Geng Andrew Owens DiffM 48 9 0 20 May 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 34 28 0 18 May 2024
Approximate Nearest Neighbour Search on Dynamic Datasets: An Investigation Ben Harwood Amir Dezfouli Iadine Chadès Conrad Sanderson 31 0 0 30 Apr 2024
Aligning Knowledge Graphs Provided by Humans and Generated from Neural Networks in Specific Tasks Tangrui Li Jun Zhou 35 0 0 23 Apr 2024
Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering Jie Ma Min Hu Pinghui Wang Wangchun Sun Lingyun Song Hongbin Pei Jun Liu Youtian Du 35 4 0 18 Apr 2024
OmniSat: Self-Supervised Modality Fusion for Earth Observation Guillaume Astruc Nicolas Gonthier Clement Mallet Loic Landrieu 32 25 0 12 Apr 2024
MoReVQA: Exploring Modular Reasoning Models for Video Question Answering Juhong Min Shyamal Buch Arsha Nagrani Minsu Cho Cordelia Schmid LRM 36 20 0 09 Apr 2024
Segment Any 3D Object with Language Seungjun Lee Yuyang Zhao Gim Hee Lee 38 1 0 02 Apr 2024
MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models Zebang Cheng Fuqiang Niu Yuxiang Lin Zhi-Qi Cheng Bowen Zhang Xiaojiang Peng 23 7 0 31 Mar 2024
Long-Tailed Anomaly Detection with Learnable Class Names Chih-Hui Ho Kuan-Chuan Peng Nuno Vasconcelos OODD 33 6 0 29 Mar 2024
RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method Ming Yan Yan Zhang Shuqiang Cai Shuqi Fan Xincheng Lin ... Siqi Shen Chenglu Wen Lan Xu Yuexin Ma Cheng-Yu Wang 43 6 0 28 Mar 2024
Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models Elaine Sui Xiaohan Wang Serena Yeung-Levy VLM 28 5 0 19 Mar 2024
Continuous Object State Recognition for Cooking Robots Using Pre-Trained Vision-Language Models and Black-box Optimization Kento Kawaharazuka Naoaki Kanazawa Yoshiki Obinata K. Okada Masayuki Inaba 17 5 0 13 Mar 2024
Diffusion Model-Based Image Editing: A Survey Yi Huang Jiancheng Huang Yifan Liu Mingfu Yan Jiaxi Lv Jianzhuang Liu Wei Xiong He Zhang Liangliang Cao Liangliang Cao EGVM 66 85 0 27 Feb 2024