LMDrive: Closed-Loop End-to-End Driving with Large Language Models

12 December 2023

Steven L. Waslander

Papers citing "LMDrive: Closed-Loop End-to-End Driving with Large Language Models"

36 / 86 papers shown

Title
CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Driving Hidehisa Arai Keita Miwa Kento Sasaki Yu Yamaguchi Kohei Watanabe Shunsuke Aoki Issei Yamamoto 29 9 0 19 Aug 2024
VLM-MPC: Vision Language Foundation Model (VLM)-Guided Model Predictive Controller (MPC) for Autonomous Driving Keke Long Haotian Shi Jiaxi Liu Xiaopeng Li 30 9 0 09 Aug 2024
SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Driving Peiru Zheng Yun Zhao Zhan Gong Hong Zhu Shaohua Wu MLLM 14 5 0 31 Jul 2024
Large Language Models for Human-like Autonomous Driving: A Survey Yun Li Kai Katsumata Ehsan Javanmardi Manabu Tsukada LM&MA 27 5 0 27 Jul 2024
KoMA: Knowledge-driven Multi-agent Framework for Autonomous Driving with Large Language Models Kemou Jiang Xuan Cai Zhiyong Cui Aoyong Li Yilong Ren Haiyang Yu Hao Yang Daocheng Fu Licheng Wen Pinlong Cai LLMAG 32 7 0 19 Jul 2024
Asynchronous Large Language Model Enhanced Planner for Autonomous Driving Yuan-Hsin Chen Zi-han Ding Ziqin Wang Yan Wang Lijun Zhang Si Liu 51 9 0 20 Jun 2024
Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers Omer Sahin Tas Royden Wagner 33 1 0 17 Jun 2024
CarLLaVA: Vision language models for camera-only closed-loop driving Katrin Renz Long Chen Ana-Maria Marcu Jan Hünermann Benoît Hanotte Alice Karnsund Jamie Shotton Elahe Arani Oleg Sinavski VLM 26 16 0 14 Jun 2024
Instruct Large Language Models to Drive like Humans Ruijun Zhang Xianda Guo Wenzhao Zheng Chenming Zhang Kurt Keutzer Long Chen 21 3 0 11 Jun 2024
AD-H: Autonomous Driving with Hierarchical Agents Zaibin Zhang Shiyu Tang Yuanhang Zhang Talas Fu Yifan Wang Yang Liu Dong Wang Jing Shao Lijun Wang H. Lu 42 3 0 05 Jun 2024
DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences Yidong Huang Jacob Sansom Ziqiao Ma Felix Gervits Joyce Chai 30 17 0 05 Jun 2024
PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planning Yupeng Zheng Zebin Xing Qichao Zhang Bu Jin Pengfei Li ... Zhongpu Xia Kun Zhan Xianpeng Lang Yaran Chen Dongbin Zhao LM&Ro LRM LLMAG 33 14 0 03 Jun 2024
Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving? Yifan Bai Dongming Wu Yingfei Liu Fan Jia Weixin Mao ... Yucheng Zhao Jianbing Shen Xing Wei Tiancai Wang Xiangyu Zhang MLLM 16 9 0 28 May 2024
Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving Jianbiao Mei Yukai Ma Xuemeng Yang Licheng Wen Xinyu Cai ... Min Dou Botian Shi Liang He Yong-Jin Liu Yu Qiao 20 9 0 24 May 2024
Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving with Typography N. Chung Sensen Gao Tuan-Anh Vu Jie M. Zhang Aishan Liu Yun Lin Jin Song Dong Qi Guo AAML 22 9 0 23 May 2024
Co-driver: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenes Ziang Guo Artem Lykov Zakhar Yagudin Mikhail Konenkov Dzmitry Tsetserukou VLM 22 4 0 09 May 2024
Language-Image Models with 3D Understanding Jang Hyun Cho B. Ivanovic Yulong Cao Edward Schmerling Yue Wang ... Boyi Li Yurong You Philipp Krahenbuhl Yan Wang Marco Pavone LRM 40 15 0 06 May 2024
MoVA: Adapting Mixture of Vision Experts to Multimodal Context Zhuofan Zong Bingqi Ma Dazhong Shen Guanglu Song Hao Shao Dongzhi Jiang Hongsheng Li Yu Liu MoE 37 40 0 19 Apr 2024
Physical Backdoor Attack can Jeopardize Driving with Vision-Large-Language Models Zhenyang Ni Rui Ye Yuxian Wei Zhen Xiang Yanfeng Wang Siheng Chen AAML 25 9 0 19 Apr 2024
LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving Pranjal Paul Anant Garg Tushar Choudhary Arun Kumar Singh K. M. Krishna 31 3 0 29 Mar 2024
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning Hao Shao Shengju Qian Han Xiao Guanglu Song Zhuofan Zong Letian Wang Yu Liu Hongsheng Li VGen LRM MLLM 47 35 0 25 Mar 2024
DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving Tianqi Wang Enze Xie Ruihang Chu Zhenguo Li Ping Luo LRM 27 12 0 25 Mar 2024
SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Prediction Yang Zhou Hao Shao Letian Wang Steven L. Waslander Hongsheng Li Yu Liu 26 13 0 18 Mar 2024
Driving Style Alignment for LLM-powered Driver Agent Ruoxuan Yang Xinyue Zhang Anais Fernandez-Laaksonen Xin Ding Jiangtao Gong 22 10 0 17 Mar 2024
V-IRL: Grounding Virtual Intelligence in Real Life Jihan Yang Runyu Ding Ellis L Brown Xiaojuan Qi Saining Xie LM&Ro 44 18 0 05 Feb 2024
Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives Sheng Luo Wei-Neng Chen Wanxin Tian Rui Liu Luanxuan Hou ... Ling Shao Yi Yang Bojun Gao Qun Li Guobin Wu 47 3 0 05 Feb 2024
LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving Daocheng Fu Wenjie Lei Licheng Wen Pinlong Cai Song Mao Min Dou Botian Shi Yu Qiao 31 6 0 02 Feb 2024
A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook Mingyu Liu Ekim Yurtsever Jonathan Fossaert Xingcheng Zhou Walter Zimmer Yuning Cui B. L. Žagar Alois C. Knoll 40 34 0 02 Jan 2024
Prospective Role of Foundation Models in Advancing Autonomous Vehicles Jianhua Wu B. Gao Jincheng Gao Jianhao Yu Hongqing Chu ... Xun Gong Yi Chang H. E. Tseng Hong Chen Jie Chen 26 3 0 08 Dec 2023
LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs Yunsheng Ma Can Cui Xu Cao Wenqian Ye Peiran Liu ... Rohit Gupta Kyungtae Han Aniket Bera James M. Rehg Ziran Wang 10 42 0 07 Dec 2023
LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving Hao Sha Yao Mu Yuxuan Jiang Li Chen Chenfeng Xu Ping Luo Shengbo Eben Li Masayoshi Tomizuka Wei Zhan Mingyu Ding 96 154 0 04 Oct 2023
End-to-end Autonomous Driving: Challenges and Frontiers Li Chen Peng Wu Kashyap Chitta Bernhard Jaeger Andreas Geiger Hongyang Li 3DV 17 160 0 29 Jun 2023
Car-Following Models: A Multidisciplinary Review T. Zhang Ph.D. Peter J. Jin Ph.D. Sean T. McQuade Ph.D. Alexandre M. Bayen Ph.D. B. Piccoli 14 3 0 14 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving Kashyap Chitta Aditya Prakash Bernhard Jaeger Zehao Yu Katrin Renz Andreas Geiger ViT 78 291 0 31 May 2022
Implicit Latent Variable Model for Scene-Consistent Motion Forecasting Sergio Casas Cole Gulino Simon Suo Katie Z Luo Renjie Liao R. Urtasun 141 135 0 23 Jul 2020