SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities

22 January 2024

Dorsa Sadigh

Papers citing "SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities"

42 / 42 papers shown

Title
PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes Ahmed Abdelreheem Filippo Aleotti Jamie Watson Z. Qureshi Abdelrahman Eldesokey Peter Wonka Gabriel J. Brostow Sara Vicente Guillermo Garcia-Hernando DiffM 50 0 0 08 May 2025
Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects Agnese Chiatti Sara Bernardini Lara Shibelski Godoy Piccolo Viola Schiaffonati Matteo Matteucci 58 0 0 08 May 2025
Seeking to Collide: Online Safety-Critical Scenario Generation for Autonomous Driving with Retrieval Augmented Large Language Models Yuewen Mei Tong Nie Jian-jun Sun Ye Tian 63 0 0 02 May 2025
SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models Wufei Ma Luoxin Ye Nessa McWeeney Celso M de Melo A. Yuille Jieneng Chen LRM 57 1 0 01 May 2025
Zoomer: Adaptive Image Focus Optimization for Black-box MLLM Jiaxu Qian Chendong Wang Y. Yang Chaoyun Zhang Huiqiang Jiang ... Saravan Rajmohan Dongmei Zhang Y. Yang Qi Zhang Lili Qiu VLM 70 0 0 30 Apr 2025
A Review of 3D Object Detection with Vision-Language Models Ranjan Sapkota Konstantinos I Roumeliotis Rahul Harsha Cheppally Marco Flores Calero Manoj Karkee VLM 74 1 0 25 Apr 2025
Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models Chen Wang Fei Xia Wenhao Yu Tingnan Zhang Ruohan Zhang Ce Liu Li Fei-Fei Jie Tan Jacky Liang 31 0 0 17 Apr 2025
NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving Kexin Tian Jingrui Mao Y. Zhang Jiwan Jiang Yang Zhou Zhengzhong Tu CoGe 60 0 0 04 Apr 2025
STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding? Y. Li Y. Zhang Tao Lin Xiangrui Liu Wenxiao Cai Zheng Liu Bo Zhao LRM 56 0 0 31 Mar 2025
From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D Jiahui Zhang Yurui Chen Yanpeng Zhou Yueming Xu Ze Huang ... Xinyue Cai G. Huang Xingyue Quan Hang Xu Li Zhang LRM 87 0 0 29 Mar 2025
AlphaSpace: Enabling Robotic Actions through Semantic Tokenization and Symbolic Reasoning Alan Dao Dinh Bach Vu Bui Quang Huy 55 0 0 24 Mar 2025
GraspCorrect: Robotic Grasp Correction via Vision-Language Model-Guided Feedback Sungjae Lee Yeonjoo Hong Kwang In KIm 44 0 0 19 Mar 2025
ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videos Peiran Wu Yunze Liu Chonghan Liu Miao Liu VGen LRM 57 1 0 16 Mar 2025
PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability Weijie Zhou Manli Tao Chaoyang Zhao Haiyun Guo Honghui Dong Ming Tang J. T. Wang 46 0 0 11 Mar 2025
AutoSpatial: Visual-Language Reasoning for Social Robot Navigation through Efficient Spatial Reasoning Learning Yangzhe Kong Daeun Song Jing Liang Dinesh Manocha Ziyu Yao Xuesu Xiao LRM 56 1 0 10 Mar 2025
Predicate Hierarchies Improve Few-Shot State Classification Emily Jin Joy Hsu Jiajun Wu OffRL 67 0 0 18 Feb 2025
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards Shivansh Patel Xinchen Yin Wenlong Huang Shubham Garg H. Nayyeri Li Fei-Fei Svetlana Lazebnik Y. Li 89 0 0 12 Feb 2025
Social-LLaVA: Enhancing Robot Navigation through Human-Language Reasoning in Social Spaces Amirreza Payandeh Daeun Song Mohammad Nazeri Jing Liang Praneel Mukherjee Amir Hossain Raj Yangzhe Kong Dinesh Manocha Xuesu Xiao LM&Ro LRM 70 5 0 17 Jan 2025
RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World Weixin Mao Weiheng Zhong Zhou Jiang Dong Fang Zhongyue Zhang ... Fan Jia Tiancai Wang Haoqiang Fan Osamu Yoshie Osamu Yoshie 114 4 0 29 Nov 2024
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics Chan Hee Song Valts Blukis Jonathan Tremblay Stephen Tyree Yu-Chuan Su Stan Birchfield 83 5 0 25 Nov 2024
TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigation Linqing Zhong Chen Gao Zihan Ding Yue Liao Si Liu Shifeng Zhang Xu Zhou Si Liu LRM 85 3 0 25 Nov 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 48 11 0 22 Oct 2024
Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards Lukas Brunke Yanni Zhang Ralf Romer Jack Naimer Nikola Staykov Siqi Zhou Angela P. Schoellig 52 3 0 19 Oct 2024
SPA: 3D Spatial-Awareness Enables Effective Embodied Representation Haoyi Zhu Honghui Yang Yating Wang Jiange Yang Limin Wang Tong He 3DH 43 5 0 10 Oct 2024
Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progress Christopher Agia Rohan Sinha Jingyun Yang Zi-ang Cao Rika Antonova Marco Pavone Jeannette Bohg 26 6 0 06 Oct 2024
ET-Plan-Bench: Embodied Task-level Planning Benchmark Towards Spatial-Temporal Cognition with Foundation Models Lingfeng Zhang Yuening Wang Hongjian Gu Atia Hamidizadeh Zhanguang Zhang ... Tongtong Cao Yuzheng Zhuang Yingxue Zhang Jianye Hao Jianye Hao LM&Ro 34 0 0 02 Oct 2024
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension Junzhuo Liu X. Yang Weiwei Li Peng Wang ObjD 39 3 0 23 Sep 2024
MotIF: Motion Instruction Fine-tuning Minyoung Hwang Joey Hejna Dorsa Sadigh Yonatan Bisk 45 1 0 16 Sep 2024
Game On: Towards Language Models as RL Experimenters Jingwei Zhang Thomas Lampe A. Abdolmaleki Jost Tobias Springenberg Martin Riedmiller LM&Ro 29 0 0 05 Sep 2024
Affordance-Guided Reinforcement Learning via Visual Prompting Olivia Y. Lee Annie Xie Kuan Fang Karl Pertsch Chelsea Finn OffRL LM&Ro 64 7 0 14 Jul 2024
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy Xiang Li Cristina Mata J. Park Kumara Kahatapitiya Yoo Sung Jang ... Kanchana Ranasinghe R. Burgert Mu Cai Yong Jae Lee Michael S. Ryoo LM&Ro 59 25 0 28 Jun 2024
Neuro-symbolic Training for Reasoning over Spatial Language Tanawan Premsri Parisa Kordjamshidi NAI LRM 35 5 0 19 Jun 2024
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models Byung-Kwan Lee Chae Won Kim Beomchan Park Yonghyun Ro MLLM LRM 22 17 0 24 May 2024
FlexCap: Describe Anything in Images in Controllable Detail Debidatta Dwibedi Vidhi Jain Jonathan Tompson Andrew Zisserman Y. Aytar CLIP VLM 40 2 0 18 Mar 2024
RePLan: Robotic Replanning with Perception and Language Models Marta Skreta Zihan Zhou Jia Lin Yuan Kourosh Darvish Alán Aspuru-Guzik Animesh Garg LM&Ro LRM 27 26 0 08 Jan 2024
Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning Juan Rocamonde Victoriano Montesinos Elvis Nava Ethan Perez David Lindner VLM 31 73 0 19 Oct 2023
Vision-Language Models as Success Detectors Yuqing Du Ksenia Konyushkova Misha Denil A. Raju Jessica Landon Felix Hill Nando de Freitas Serkan Cabi MLLM LRM 84 76 0 13 Mar 2023
Mind's Eye: Grounded Language Model Reasoning through Simulation Ruibo Liu Jason W. Wei S. Gu Te-Yen Wu Soroush Vosoughi Claire Cui Denny Zhou Andrew M. Dai ReLM LRM 109 78 0 11 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 233 341 0 22 Sep 2021
Grounding Natural Language Instructions: Can Large Language Models Capture Spatial Information? Julia Rozanova Deborah Ferreira K. Dubba Weiwei Cheng Dell Zhang André Freitas LM&Ro 28 8 0 17 Sep 2021
Deep Ordinal Regression Network for Monocular Depth Estimation Huan Fu Mingming Gong Chaohui Wang Kayhan Batmanghelich Dacheng Tao MDE 180 1,687 0 06 Jun 2018