LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning

30 November 2023

Papers citing "LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning"

50 / 61 papers shown

Title
SpatialPrompting: Keyframe-driven Zero-Shot Spatial Reasoning with Off-the-Shelf Multimodal Large Language Models Shun Taguchi Hideki Deguchi Takumi Hamazaki Hiroyuki Sakai ReLM LRM 27 0 0 08 May 2025
3D CoCa: Contrastive Learners are 3D Captioners Ting Huang Z. Zhang Y. Wang Hao Tang 20 0 0 13 Apr 2025
OmniSVG: A Unified Scalable Vector Graphics Generation Model Yiying Yang Wei Cheng Sijin Chen Xianfang Zeng Jiaxu Zhang Liao Wang Gang Yu Xingjun Ma Yu Jiang VLM 35 0 0 08 Apr 2025
The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? Weichen Zhang Ruiying Peng Chen Gao Jianjie Fang Xin Zeng ... Z. Wang Jinqiang Cui Xin Wang Xinlei Chen Y. Li LRM 71 0 0 06 Apr 2025
Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision Xiaofeng Han Shunpeng Chen Zenghuang Fu Zhe Feng Lue Fan ... Li Guo Weiliang Meng Xiaopeng Zhang Rongtao Xu Shibiao Xu 60 0 0 03 Apr 2025
Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awareness Haochen Wang Yucheng Zhao Tiancai Wang Haoqiang Fan X. Zhang Zhaoxiang Zhang 59 0 0 02 Apr 2025
Empowering Large Language Models with 3D Situation Awareness Zhihao Yuan Yibo Peng Jinke Ren Yinghong Liao Yatong Han Chun-Mei Feng Hengshuang Zhao G. Li Shuguang Cui Zhen Li 41 0 0 29 Mar 2025
From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D Jiahui Zhang Yurui Chen Yanpeng Zhou Yueming Xu Ze Huang ... Xinyue Cai G. Huang Xingyue Quan Hang Xu Li Zhang LRM 87 0 0 29 Mar 2025
Unveiling the Mist over 3D Vision-Language Understanding: Object-centric Evaluation with Chain-of-Analysis J. Huang Baoxiong Jia Y. Wang Ziyu Zhu Xiongkun Linghu Qing Li Song-Chun Zhu Siyuan Huang 75 3 0 28 Mar 2025
MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation Jiaxin Huang Runnan Chen Ziwen Li Zhengqing Gao Xiao He Yandong Guo M. Gong Tongliang Liu LRM 44 0 0 23 Mar 2025
ExCap3D: Expressive 3D Scene Understanding via Object Captioning with Varying Detail Chandan Yeshwanth Dávid Rozenberszki Angela Dai 69 0 0 21 Mar 2025
GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructions Xiaomeng Chu Jiajun Deng Guoliang You Wei Liu X. Li Jianmin Ji Y. Zhang 77 0 0 20 Mar 2025
HIS-GPT: Towards 3D Human-In-Scene Multimodal Understanding Jiahe Zhao Ruibing Hou Zejie Tian Hong Chang Shiguang Shan 36 0 0 17 Mar 2025
Exploring 3D Activity Reasoning and Planning: From Implicit Human Intentions to Route-Aware Planning Xueying Jiang Wenhao Li Xiaoqin Zhang Ling Shao Shijian Lu LRM 35 0 0 17 Mar 2025
Learning A Zero-shot Occupancy Network from Vision Foundation Models via Self-supervised Adaptation Sihao Lin Daqi Liu Ruochong Fu Dongrui Liu A. Song Hongwei Xie Zhihui Li Bing Wang Xiaojun Chang 67 0 0 10 Mar 2025
SplatTalk: 3D VQA with Gaussian Splatting Anh Thai Songyou Peng Kyle Genova Leonidas J. Guibas Thomas Funkhouser 3DGS 75 0 0 08 Mar 2025
Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning Hanxun Yu Wentong Li Song Wang J. Chen Jianke Zhu 3DV LRM 71 3 0 01 Mar 2025
3D-AffordanceLLM: Harnessing Large Language Models for Open-Vocabulary Affordance Detection in 3D Worlds Hengshuo Chu Xiang Deng Qi Lv Xiaoyang Chen Yinchuan Li Jianye Hao Liqiang Nie 56 2 0 27 Feb 2025
Robin3D: Improving 3D Large Language Model via Robust Instruction Tuning Weitai Kang Haifeng Huang Yuzhang Shang Mubarak Shah Yan Yan 43 7 0 21 Feb 2025
Understanding and Evaluating Hallucinations in 3D Visual Language Models Ruiying Peng Kaiyuan Li Weichen Zhang Chen Gao Xinlei Chen Y. Li 33 0 0 18 Feb 2025
3D-Grounded Vision-Language Framework for Robotic Task Planning: Automated Prompt Synthesis and Supervised Reasoning Guoqin Tang Qingxuan Jia Zeyuan Huang Gang Chen Ning Ji Zhipeng Yao 58 0 0 13 Feb 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 83 10 0 06 Jan 2025
GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models Zhangyang Qi Zhixiong Zhang Ye Fang Jiaqi Wang Hengshuang Zhao 81 6 0 02 Jan 2025
GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-training Renqiu Xia M. Li Hancheng Ye Wenjie Wu Hongbin Zhou ... Conghui He Botian Shi Tao Chen Junchi Yan Bo Zhang 74 7 0 16 Dec 2024
Chimera: Improving Generalist Model with Domain-Specific Experts Tianshuo Peng M. Li Hongbin Zhou Renqiu Xia Renrui Zhang ... Aojun Zhou Botian Shi Tao Chen Bo Zhang Xiangyu Yue 79 4 0 08 Dec 2024
LSceneLLM: Enhancing Large 3D Scene Understanding Using Adaptive Visual Preferences Hongyan Zhi Peihao Chen Junyan Li Shuailei Ma Xinyu Sun Tianhang Xiang Yinjie Lei Mingkui Tan Chuang Gan 67 3 0 02 Dec 2024
Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments Sangmim Song S. Kodagoda A. Gunatilake Marc G. Carmichael Karthick Thiyagarajan Jodi Martin LM&Ro 21 1 0 28 Oct 2024
Synergistic Dual Spatial-aware Generation of Image-to-Text and Text-to-Image Yu Zhao Hao Fei Xiangtai Li L. Qin Jiayi Ji Hongyuan Zhu Meishan Zhang M. Zhang Jianguo Wei DiffM 23 1 0 20 Oct 2024
Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis Bohan Zeng Ling Yang Siyu Li Jiaming Liu Zixiang Zhang ... Yongzhen Guo Fu-Yun Wang Minkai Xu Stefano Ermon Wentao Zhang VGen AI4CE 18 6 0 09 Oct 2024
IntrinsicVoice: Empowering LLMs with Intrinsic Real-time Voice Interaction Abilities Xin Zhang Xiang Lyu Zhihao Du Qian Chen Dong Zhang ... Yuxuan Wang Bin Zhang Heng Lu Yaqian Zhou Xipeng Qiu AuLLM 20 4 0 09 Oct 2024
OrionNav: Online Planning for Robot Autonomy with Context-Aware LLM and Open-Vocabulary Semantic Scene Graphs Venkata Naren Devarakonda Raktim Gautam Goswami Ali Umut Kaypak Naman Patel Rooholla Khorrambakht P. Krishnamurthy Farshad Khorrami LM&Ro 19 3 0 08 Oct 2024
SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Models Yue Zhang Zhiyang Xu Ying Shen Parisa Kordjamshidi Lifu Huang 19 6 0 04 Oct 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 84 29 0 26 Sep 2024
Multi-modal Situated Reasoning in 3D Scenes Xiongkun Linghu Jiangyong Huang Xuesong Niu Xiaojian Ma Baoxiong Jia Siyuan Huang 16 11 0 04 Sep 2024
More Text, Less Point: Towards 3D Data-Efficient Point-Language Understanding Yuan Tang Xu Han Xianzhi Li Qiao Yu Jinfeng Xu Yixue Hao Long Hu Min Chen 22 1 0 28 Aug 2024
Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model Wenqi Zhang Zhenglin Cheng Yuanyu He Mengna Wang Yongliang Shen ... Guiyang Hou Mingqian He Yanna Ma Weiming Lu Yueting Zhuang SyDa 43 9 0 09 Jul 2024
ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilities Chenming Zhu Tai Wang Wenwei Zhang Kai Chen Xihui Liu ReLM LRM 42 16 0 01 Jul 2024
LiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control Delin Qu Qizhi Chen Pingrui Zhang Xianqiang Gao Bin Zhao Bin Zhao Dong Wang Xuelong Li AI4CE 30 7 0 23 Jun 2024
Lightweight Model Pre-training via Language Guided Knowledge Distillation Mingsheng Li Lin Zhang Mingzhen Zhu Zilong Huang Gang Yu Jiayuan Fan Tao Chen 22 0 0 17 Jun 2024
MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations Ruiyuan Lyu Tai Wang Jingli Lin Shuai Yang Xiaohan Mao ... Runsen Xu Haifeng Huang Chenming Zhu Dahua Lin Jiangmiao Pang 3DV 28 9 0 13 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 53 11 0 07 Jun 2024
LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model Yixuan Yang Junru Lu Zixiang Zhao Zhen Luo James J.Q. Yu Victor Sanchez Feng Zheng 3DV 23 3 0 06 Jun 2024
Collaborative Novel Object Discovery and Box-Guided Cross-Modal Alignment for Open-Vocabulary 3D Object Detection Yang Cao Yihan Zeng Hang Xu Dan Xu 3DPC ObjD 20 6 0 02 Jun 2024
MeshXL: Neural Coordinate Field for Generative 3D Foundation Models Sijin Chen Xin Chen Anqi Pang Xianfang Zeng Wei Cheng ... C. Zhang Jingyi Yu Gang Yu Bin-Bin Fu Tao Chen AI4CE 44 35 0 31 May 2024
Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models Tianrun Chen Chunan Yu Jing Li Jianqi Zhang Lanyun Zhu Deyi Ji Yong Zhang Ying-Dong Zang Zejian Li Lingyun Sun LRM 30 9 0 29 May 2024
Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model Kuan-Chih Huang Xiangtai Li Lu Qi Shuicheng Yan Ming-Hsuan Yang LRM 41 9 0 27 May 2024
Unifying 3D Vision-Language Understanding via Promptable Queries Ziyu Zhu Zhuofan Zhang Xiaojian Ma Xuesong Niu Yixin Chen Baoxiong Jia Zhidong Deng Siyuan Huang Qing Li 37 21 0 19 May 2024
Grounded 3D-LLM with Referent Tokens Yilun Chen Shuai Yang Haifeng Huang Tai Wang Ruiyuan Lyu Runsen Xu Dahua Lin Jiangmiao Pang 42 22 0 16 May 2024
When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models Xianzheng Ma Yash Bhalgat Brandon Smart Shuai Chen Xinghui Li ... Matthias Nießner Ian D Reid Angel X. Chang Iro Laina V. Prisacariu LRM 29 11 0 16 May 2024
MiniGPT-3D: Efficiently Aligning 3D Point Clouds with Large Language Models using 2D Priors Yuan Tang Xu Han Xianzhi Li Qiao Yu Yixue Hao Long Hu Min Chen 16 12 0 02 May 2024