Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

5 June 2023

Xin Li

Papers citing "Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding"

44 / 694 papers shown

Title
LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples Jia-Yu Yao Kun-Peng Ning Zhen-Hui Liu Munan Ning Li Yuan HILM LRM AAML 15 168 0 02 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 24 92 0 27 Sep 2023
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning Ruyang Liu Chen Li Yixiao Ge Ying Shan Thomas H. Li Ge Li 25 29 0 27 Sep 2023
MSG-BART: Multi-granularity Scene Graph-Enhanced Encoder-Decoder Language Model for Video-grounded Dialogue Generation Hongcheng Liu Zhe Chen Hui Li Pingjie Wang Yanfeng Wang Yu Wang VGen 38 1 0 26 Sep 2023
Connecting Speech Encoder and Large Language Model for ASR Wenyi Yu Changli Tang Guangzhi Sun Xianzhao Chen T. Tan Wei Li Lu Lu Zejun Ma Chao Zhang AuLLM 6 64 0 25 Sep 2023
Investigating the Catastrophic Forgetting in Multimodal Large Language Models Yuexiang Zhai Shengbang Tong Xiao Li Mu Cai Qing Qu Yong Jae Lee Y. Ma VLM MLLM CLL 75 77 0 19 Sep 2023
MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response Zihao Deng Yi Ma Yudong Liu Rongchen Guo Ge Zhang Wenhu Chen Wenhao Huang Emmanouil Benetos MLLM AuLLM 26 17 0 15 Sep 2023
Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos Sarthak Bhagat Simon Stepputtis Joseph Campbell Katia P. Sycara 18 4 0 12 Sep 2023
NExT-GPT: Any-to-Any Multimodal LLM Shengqiong Wu Hao Fei Leigang Qu Wei Ji Tat-Seng Chua MLLM 46 448 0 11 Sep 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng-Tao Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 25 115 0 07 Sep 2023
Can I Trust Your Answer? Visually Grounded Video Question Answering Junbin Xiao Angela Yao Yicong Li Tat-Seng Chua 28 46 0 04 Sep 2023
Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior Ashmit Khandelwal Aditya Agrawal Aanisha Bhattacharyya Yaman Kumar Singla Somesh Singh ... Ishita Dasgupta Stefano Petrangeli R. Shah Changyou Chen Balaji Krishnamurthy 11 8 0 01 Sep 2023
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond Jinze Bai Shuai Bai Shusheng Yang Shijie Wang Sinan Tan Peng Wang Junyang Lin Chang Zhou Jingren Zhou MLLM VLM ObjD 43 780 0 24 Aug 2023
Instruction Tuning for Large Language Models: A Survey Shengyu Zhang Linfeng Dong Xiaoya Li Sen Zhang Xiaofei Sun ... Jiwei Li Runyi Hu Tianwei Zhang Fei Wu Guoyin Wang LM&MA 21 524 0 21 Aug 2023
FashionLOGO: Prompting Multimodal Large Language Models for Fashion Logo Embeddings Yulin Su Min Yang Minghui Qiu Jing Wang Tao Wang VLM 25 0 0 17 Aug 2023
Chat-3D: Data-efficiently Tuning Large Language Model for Universal Dialogue of 3D Scenes Zehan Wang Haifeng Huang Yang Zhao Ziang Zhang Zhou Zhao 19 58 0 17 Aug 2023
OctoPack: Instruction Tuning Code Large Language Models Niklas Muennighoff Qian Liu A. Zebaze Qinkai Zheng Binyuan Hui Terry Yue Zhuo Swayam Singh Xiangru Tang Leandro von Werra Shayne Longpre VLM ALM 55 116 0 14 Aug 2023
Fine-Tune Language Models as Multi-Modal Differential Equation Solvers Liu Yang Siting Liu Stanley J. Osher 16 0 0 09 Aug 2023
RegionBLIP: A Unified Multi-modal Pre-training Framework for Holistic and Regional Comprehension Qiang-feng Zhou Chaohui Yu Shaofeng Zhang Sitong Wu Zhibin Wang Fan Wang 26 26 0 03 Aug 2023
NLLG Quarterly arXiv Report 06/23: What are the most influential current AI Papers? Steffen Eger Christoph Leiter Jonas Belouadi Ran Zhang Aida Kostikova Daniil Larionov Yanran Chen Vivian Fresen AI4CE 13 4 0 31 Jul 2023
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding Enxin Song Wenhao Chai Guanhong Wang Yucheng Zhang Haoyang Zhou ... Tianbo Ye Yanting Zhang Yang Lu Jenq-Neng Hwang Gaoang Wang VLM MLLM 17 259 0 31 Jul 2023
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao Zhijie Lin Daquan Zhou Zilong Huang Jiashi Feng Bingyi Kang MLLM 33 106 0 17 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 46 499 0 12 Jul 2023
SVIT: Scaling up Visual Instruction Tuning Bo-Lu Zhao Boya Wu Muyang He Tiejun Huang MLLM 28 120 0 09 Jul 2023
Exploring and Characterizing Large Language Models For Embedded System Development and Debugging Zachary Englhardt R. Li Dilini Nissanka Zhihan Zhang Girish Narayanswamy Joseph Breda Xin Liu Shwetak N. Patel Vikram Iyer 22 15 0 07 Jul 2023
What Matters in Training a GPT4-Style Language Model with Multimodal Inputs? Yan Zeng Hanbo Zhang Jiani Zheng Jiangnan Xia Guoqiang Wei Yang Wei Yuchen Zhang Tao Kong MLLM 19 71 0 05 Jul 2023
LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding Yanzhe Zhang Ruiyi Zhang Jiuxiang Gu Yufan Zhou Nedim Lipka Diyi Yang Tongfei Sun VLM MLLM 25 217 0 29 Jun 2023
Explainable Multimodal Emotion Recognition Zheng Lian Haiyang Sun Licai Sun Hao Gu Zhuofan Wen ... Shan Liang Ya Li Jiangyan Yi B. Liu Jianhua Tao MLLM 8 6 0 27 Jun 2023
FunQA: Towards Surprising Video Comprehension Binzhu Xie Sicheng Zhang Zitang Zhou Bo-wen Li Yuanhan Zhang Jack Hessel Jingkang Yang Ziwei Liu 26 20 0 26 Jun 2023
Large Multimodal Models: Notes on CVPR 2023 Tutorial Chunyuan Li MLLM VLM 14 20 0 26 Jun 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Bill Xu Enhong Chen MLLM LRM 33 551 0 23 Jun 2023
Path to Medical AGI: Unify Domain-specific Medical LLMs with the Lowest Cost Juexiao Zhou Xiuying Chen Xin Gao LM&MA AI4CE 85 12 0 19 Jun 2023
LLMVA-GEBC: Large Language Model with Video Adapter for Generic Event Boundary Captioning Yunlong Tang Jinrui Zhang Xiangchen Wang Teng Wang Feng Zheng VLM 64 9 0 17 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 38 188 0 12 Jun 2023
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models Muhammad Maaz H. Rasheed Salman Khan F. Khan MLLM 15 578 0 08 Jun 2023
Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Dataset for Pre-training and Benchmarks Haiyang Xu Qinghao Ye Xuan-Wei Wu Mingshi Yan Yuan Miao ... Qingfang Qian Maofei Que Ji Zhang Xiaoyan Zeng Feiyan Huang VLM MLLM 33 21 0 07 Jun 2023
Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models Oana Ignat Zhijing Jin Artem Abzaliev Laura Biester Santiago Castro ... Verónica Pérez-Rosas Siqi Shen Zekun Wang Winston Wu Rada Mihalcea LRM 24 6 0 21 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 206 883 0 27 Apr 2023
Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models Jiashuo Sun Yi Luo Yeyun Gong Chen Lin Yelong Shen Jian Guo Nan Duan LRM 30 19 0 23 Apr 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 20 1,896 0 20 Apr 2023
LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervision Jiani Huang Ziyang Li Mayur Naik Ser-Nam Lim 30 3 0 15 Apr 2023
Retrieving Multimodal Information for Augmented Generation: A Survey Ruochen Zhao Hailin Chen Weishi Wang Fangkai Jiao Do Xuan Long ... Bosheng Ding Xiaobao Guo Minzhi Li Xingxuan Li Shafiq R. Joty 13 80 0 20 Mar 2023
Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities Hammad A. Ayyubi Christopher Thomas Lovish Chum R. Lokesh Long Chen ... Xudong Lin Xuande Feng Jaywon Koo Sounak Ray Shih-Fu Chang AI4TS 10 0 0 14 Jun 2022
Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions Shahin Atakishiyev Mohammad Salameh Hengshuai Yao Randy Goebel 19 127 0 21 Dec 2021