VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

12 July 2023

Wenlong Huang

Chen Wang

Ruohan Zhang

Yunzhu Li

Jiajun Wu

Li Fei-Fei

LM&Ro

ArXiv PDF HTML

Papers citing "VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models"

50 / 106 papers shown

Title
Meta-Optimization and Program Search using Language Models for Task and Motion Planning Denis Shcherba Eckart Cobo-Briesewitz Cornelius V. Braun Marc Toussaint LM&Ro LRM 29 0 0 06 May 2025
CrayonRobo: Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation Xiaoqi Li Lingyun Xu M. Zhang Jiaming Liu Yan Shen ... Jiahui Xu Liang Heng Siyuan Huang S. Zhang Hao Dong LM&Ro 39 0 0 04 May 2025
Fast Flow-based Visuomotor Policies via Conditional Optimal Transport Couplings Andreas Sochopoulos Nikolay Malkin Nikolaos Tsagkas João Moura Michael Gienger S. Vijayakumar 37 1 0 02 May 2025
Dynamic Robot Tool Use with Vision Language Models Noah Trupin Zixing Wang A. H. Qureshi 35 0 0 02 May 2025
Robotic Visual Instruction Y. Li Ziyang Gong H. Li Xiaoqi Huang Haolan Kang Guangping Bai Xianzheng Ma LM&Ro 66 0 0 01 May 2025
A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI Lik Hang Kenny Wong Xueyang Kang Kaixin Bai Jianwei Zhang 45 0 0 01 May 2025
A Survey of Interactive Generative Video Jiwen Yu Yiran Qin Haoxuan Che Quande Liu X. Wang Pengfei Wan Di Zhang Kun Gai Hao Chen Xihui Liu VGen 53 0 0 30 Apr 2025
LLM-based Interactive Imitation Learning for Robotic Manipulation Jonas Werner Kun-Mo Chu C. Weber S. Wermter 71 0 0 30 Apr 2025
Cracking the Code of Action: a Generative Approach to Affordances for Reinforcement Learning Lynn Cherif Flemming Kondrup David Venuto Ankit Anand Doina Precup Khimya Khetarpal LM&Ro 40 0 0 24 Apr 2025
Enhancing Product Search Interfaces with Sketch-Guided Diffusion and Language Agents Edward Sun DiffM 30 0 0 21 Mar 2025
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Jiaming Liu Hao Chen Pengju An Zhuoyang Liu Renrui Zhang ... Chengkai Hou Mengdi Zhao KC alex Zhou Pheng-Ann Heng S. Zhang 64 6 0 13 Mar 2025
Efficient Alignment of Unconditioned Action Prior for Language-conditioned Pick and Place in Clutter Kechun Xu Xunlong Xia Kaixuan Wang Yifei Yang Yunxuan Mao Bing Deng R. Xiong Y. Wang OffRL 64 0 0 12 Mar 2025
NVP-HRI: Zero Shot Natural Voice and Posture-based Human-Robot Interaction via Large Language Model Yuzhi Lai Shenghai Yuan Youssef Nassar Mingyu Fan T. Weber Matthias Rätsch LM&Ro 64 3 0 12 Mar 2025
PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability Weijie Zhou Manli Tao Chaoyang Zhao Haiyun Guo Honghui Dong Ming Tang J. T. Wang 46 0 0 11 Mar 2025
A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning Xin Wen Bingchen Zhao Yilun Chen Jiangmiao Pang Xiaojuan Qi LM&Ro 36 0 0 10 Mar 2025
GAT-Grasp: Gesture-Driven Affordance Transfer for Task-Aware Robotic Grasping Ruixiang Wang Huayi Zhou Xinyue Yao Guiliang Liu K. Jia 34 0 0 08 Mar 2025
Generative Artificial Intelligence in Robotic Manipulation: A Survey Kun Zhang Peng Yun Jun Cen Junhao Cai DiDi Zhu ... Qifeng Chen Jia Pan Wei K. Zhang Bo Yang Hua Chen 59 1 0 05 Mar 2025
NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains Wonje Choi Jinwoo Park Sanghyun Ahn Daehee Lee Honguk Woo 47 1 0 02 Mar 2025
RobotIQ: Empowering Mobile Robots with Human-Level Planning for Real-World Execution Emmanuel K. Raptis Athanasios Ch. Kapoutsis Elias B. Kosmatopoulos LM&Ro 72 0 0 18 Feb 2025
RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation Kun Wu Chengkai Hou Jiaming Liu Zhengping Che Xiaozhu Ju ... Zhenyu Wang Pengju An Siyuan Qian S. Zhang Jian Tang LM&Ro 105 15 0 17 Feb 2025
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards Shivansh Patel Xinchen Yin Wenlong Huang Shubham Garg H. Nayyeri Li Fei-Fei Svetlana Lazebnik Y. Li 89 0 0 12 Feb 2025
Bilevel Learning for Bilevel Planning Bowen Li Tom Silver Sebastian A. Scherer Alexander G. Gray 61 0 0 12 Feb 2025
Imit Diff: Semantics Guided Diffusion Transformer with Dual Resolution Fusion for Imitation Learning Yuhang Dong Haizhou Ge Yupei Zeng J. Zhang Beiwen Tian ... Yufei Jia Ruixiang Wang Ran Yi Guyue Zhou Longhua Ma 51 0 0 11 Feb 2025
HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation Yi Li Yuquan Deng J. Zhang Joel Jang Marius Memme ... Fabio Ramos Dieter Fox Anqi Li Abhishek Gupta Ankit Goyal LM&Ro 86 5 0 08 Feb 2025
Learn from the Past: Language-conditioned Object Rearrangement with Large Language Models Guanqun Cao Ryan Mckenna Erich Graf John Oyekan LM&Ro 114 0 0 30 Jan 2025
RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation Zixuan Chen Jing Huo Yangtao Chen Yang Gao 43 2 0 11 Jan 2025
CoDriveVLM: VLM-Enhanced Urban Cooperative Dispatching and Motion Planning for Future Autonomous Mobility on Demand Systems Haichao Liu Ruoyu Yao Wenru Liu Zhenmin Huang Shaojie Shen Jun Ma 40 1 0 10 Jan 2025
AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulation Guanxing Lu Tengbo Yu Haoyuan Deng Season Si Chen Yansong Tang Ziwei Wang 70 3 0 09 Dec 2024
LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences Hongyan Zhi Peihao Chen Junyan Li Shuailei Ma Xinyu Sun Tianhang Xiang Yinjie Lei Mingkui Tan Chuang Gan 67 3 0 02 Dec 2024
RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World Weixin Mao Weiheng Zhong Zhou Jiang Dong Fang Zhongyue Zhang ... Fan Jia Tiancai Wang Haoqiang Fan Osamu Yoshie Osamu Yoshie 114 4 0 29 Nov 2024
DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models Yongdong Wang Runze Xiao Jun Younes Louhi Kasahara Ryosuke Yajima Keiji Nagatani Atsushi Yamashita Hajime Asama 23 3 0 13 Nov 2024
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints Nishanth Kumar F. Ramos Dieter Fox Caelan Reed Garrett Tomás Lozano-Pérez Leslie Pack Kaelbling Caelan Reed Garrett LRM LM&Ro 63 3 0 13 Nov 2024
Local Policies Enable Zero-shot Long-horizon Manipulation Murtaza Dalal Min Liu Walter Talbott Chen Chen Deepak Pathak Jian Zhang Ruslan Salakhutdinov 36 3 0 29 Oct 2024
Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards Lukas Brunke Yanni Zhang Ralf Romer Jack Naimer Nikola Staykov Siqi Zhou Angela P. Schoellig 52 3 0 19 Oct 2024
In-Context Learning Enables Robot Action Prediction in LLMs Yida Yin Zekai Wang Yuvan Sharma Dantong Niu Trevor Darrell Roei Herzig LM&Ro 56 1 0 16 Oct 2024
ES-Gaussian: Gaussian Splatting Mapping via Error Space-Based Gaussian Completion Lu Chen Yingfu Zeng Haoang Li Zhitao Deng Jiafu Yan Zhenjun Zhao 3DGS 3DV 29 0 0 09 Oct 2024
Open-World Reinforcement Learning over Long Short-Term Imagination Jiajian Li Q. Wang Yunbo Wang Xin Jin Yang Li Wenjun Zeng Xiaokang Yang OCL VLM 47 1 0 04 Oct 2024
SEAL: SEmantic-Augmented Imitation Learning via Language Model Chengyang Gu Yuxin Pan Haotian Bai Hui Xiong Yize Chen 27 0 0 03 Oct 2024
UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models Qiaojun Yu Siyuan Huang Xibin Yuan Zhengkai Jiang Ce Hao ... Junbo Wang Liu Liu Hongsheng Li Peng Gao Cewu Lu 65 3 0 30 Sep 2024
GravMAD: Grounded Spatial Value Maps Guided Action Diffusion for Generalized 3D Manipulation Yangtao Chen Zixuan Chen Junhui Yin Jing Huo Pinzhuo Tian Jieqi Shi Yang Gao LM&Ro 42 2 0 30 Sep 2024
FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation Litao Liu Wentao Wang Yifan Han Zhuoli Xie Pengfei Yi Junyan Li Yi Qin Wenzhao Lian 32 2 0 29 Sep 2024
Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa Kei Okada Masayuki Inaba LM&Ro 16 0 0 26 Sep 2024
COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models Kehui Liu Zixin Tang Dong Wang Z. Wang Bin Zhao Bin Zhao 29 10 0 23 Sep 2024
TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation Junjie Wen Y. X. Zhu Jinming Li Minjie Zhu Kun Wu ... Ran Cheng Chaomin Shen Yaxin Peng Feifei Feng Jian Tang LM&Ro 56 41 0 19 Sep 2024
SDP: Spiking Diffusion Policy for Robotic Manipulation with Learnable Channel-Wise Membrane Thresholds Zhixing Hou Maoxu Gao Hang Yu Mengyu Yang Chio-in Ieong 33 1 0 17 Sep 2024
Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics Yixuan Huang Christopher Agia Jimmy Wu Tucker Hermans Jeannette Bohg 3DPC 44 1 0 27 Aug 2024
General-purpose Clothes Manipulation with Semantic Keypoints Yuhong Deng David Hsu 54 2 0 15 Aug 2024
A Backbone for Long-Horizon Robot Task Understanding Xiaoshuai Chen Wei Chen Dongmyoung Lee Yukun Ge Nicolás Rojas Petar Kormushev 41 3 0 02 Aug 2024
Affordance-Guided Reinforcement Learning via Visual Prompting Olivia Y. Lee Annie Xie Kuan Fang Karl Pertsch Chelsea Finn OffRL LM&Ro 64 7 0 14 Jul 2024
VLMPC: Vision-Language Model Predictive Control for Robotic Manipulation Wentao Zhao Jiaming Chen Ziyu Meng Donghui Mao Ran Song Wei Zhang 35 8 0 13 Jul 2024