UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding

1 December 2022

Matthias Nießner

Papers citing "UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding"

43 / 43 papers shown

Title
3D CoCa: Contrastive Learners are 3D Captioners Ting Huang Z. Zhang Y. Wang Hao Tang 25 0 0 13 Apr 2025
ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning Zhenyang Liu Yikai Wang Sixiao Zheng Tongying Pan Longfei Liang Yanwei Fu Xiangyang Xue LRM 49 0 0 30 Mar 2025
ExCap3D: Expressive 3D Scene Understanding via Object Captioning with Varying Detail Chandan Yeshwanth Dávid Rozenberszki Angela Dai 71 0 0 21 Mar 2025
Exploring 3D Activity Reasoning and Planning: From Implicit Human Intentions to Route-Aware Planning Xueying Jiang Wenhao Li Xiaoqin Zhang Ling Shao Shijian Lu LRM 40 0 0 17 Mar 2025
Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuning Hanxun Yu Wentong Li Song Wang J. Chen Jianke Zhu 3DV LRM 71 3 0 01 Mar 2025
Multi-Object 3D Grounding with Dynamic Modules and Language-Informed Spatial Attention Haomeng Zhang Chiao-An Yang Raymond A. Yeh 29 1 0 29 Oct 2024
Grounding 3D Scene Affordance From Egocentric Interactions Cuiyu Liu Wei Zhai Yuhang Yang Hongchen Luo Sen Liang Yang Cao Zheng-Jun Zha 26 1 0 29 Sep 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 87 29 0 26 Sep 2024
Multi-modal Situated Reasoning in 3D Scenes Xiongkun Linghu Jiangyong Huang Xuesong Niu Xiaojian Ma Baoxiong Jia Siyuan Huang 26 11 0 04 Sep 2024
CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT Volumes Theo Di Piazza 14 0 0 21 Aug 2024
See It All: Contextualized Late Aggregation for 3D Dense Captioning Minjung Kim Hyung Suk Lim Seung Hwan Kim Soonyoung Lee Bumsoo Kim Gunhee Kim 39 0 0 14 Aug 2024
Bi-directional Contextual Attention for 3D Dense Captioning Minjung Kim Hyung Suk Lim Soonyoung Lee Bumsoo Kim Gunhee Kim 29 0 0 13 Aug 2024
ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilities Chenming Zhu Tai Wang Wenwei Zhang Kai Chen Xihui Liu ReLM LRM 45 16 0 01 Jul 2024
A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions Daizong Liu Yang Liu Wencan Huang Wei Hu LM&Ro 29 9 0 09 Jun 2024
Collaborative Novel Object Discovery and Box-Guided Cross-Modal Alignment for Open-Vocabulary 3D Object Detection Yang Cao Yihan Zeng Hang Xu Dan Xu 3DPC ObjD 28 6 0 02 Jun 2024
Unifying 3D Vision-Language Understanding via Promptable Queries Ziyu Zhu Zhuofan Zhang Xiaojian Ma Xuesong Niu Yixin Chen Baoxiong Jia Zhidong Deng Siyuan Huang Qing Li 40 21 0 19 May 2024
Grounded 3D-LLM with Referent Tokens Yilun Chen Shuai Yang Haifeng Huang Tai Wang Ruiyuan Lyu Runsen Xu Dahua Lin Jiangmiao Pang 45 22 0 16 May 2024
When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models Xianzheng Ma Yash Bhalgat Brandon Smart Shuai Chen Xinghui Li ... Matthias Nießner Ian D Reid Angel X. Chang Iro Laina V. Prisacariu LRM 29 11 0 16 May 2024
Unified Scene Representation and Reconstruction for 3D Large Language Models Tao Chu Pan Zhang Xiao-wen Dong Yuhang Zang Qiong Liu Jiaqi Wang 18 1 0 19 Apr 2024
Rethinking 3D Dense Caption and Visual Grounding in A Unified Framework through Prompt-based Localization Yongdong Luo Haojia Lin Xiawu Zheng Yigeng Jiang Fei Chao Jie Hu Guannan Jiang Songan Zhang Rongrong Ji 18 0 0 17 Apr 2024
TOD3Cap: Towards 3D Dense Captioning in Outdoor Scenes Bu Jin Yupeng Zheng Pengfei Li Weize Li Yuhang Zheng ... Kun Zhan Peng Jia Xiaoxiao Long Yilun Chen Hao Zhao 3DV 47 14 0 28 Mar 2024
Surface Normal Estimation with Transformers Barry Shichen Hu Siyun Liang Johannes Paetzold H. Nguyen Isao Echizen Jiapeng Tang ViT 3DPC 22 0 0 11 Jan 2024
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI Tai Wang Xiaohan Mao Chenming Zhu Runsen Xu Ruiyuan Lyu ... Tianfan Xue Xihui Liu Cewu Lu Dahua Lin Jiangmiao Pang LM&Ro 18 58 0 26 Dec 2023
LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding Senqiao Yang Jiaming Liu Ray Zhang Mingjie Pan Zoey Guo Xiaoqi Li Zehui Chen Peng Gao Yandong Guo Shanghang Zhang 3DV 6 58 0 21 Dec 2023
M3DBench: Let's Instruct Large Models with Multi-modal 3D Prompts Mingsheng Li Xin Chen C. Zhang Sijin Chen Hongyuan Zhu Fukun Yin Gang Yu Tao Chen 17 23 0 17 Dec 2023
Uni3DL: Unified Model for 3D and Language Understanding Xiang Li Jian Ding Zhaoyang Chen Mohamed Elhoseiny 24 3 0 05 Dec 2023
LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning Sijin Chen Xin Chen C. Zhang Mingsheng Li Gang Yu Hao Fei Hongyuan Zhu Jiayuan Fan Tao Chen MLLM 24 76 0 30 Nov 2023
SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priors Dave Zhenyu Chen Haoxuan Li Hsin-Ying Lee Sergey Tulyakov Matthias Nießner DiffM 14 28 0 28 Nov 2023
Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding Zhihao Yuan Jinke Ren Chun-Mei Feng Hengshuang Zhao Shuguang Cui Zhen Li 16 26 0 26 Nov 2023
Generating Context-Aware Natural Answers for Questions in 3D Scenes Mohammed Munzer Dwedari Matthias Niessner Dave Zhenyu Chen 19 1 0 30 Oct 2023
Recent Advances in Multi-modal 3D Scene Understanding: A Comprehensive Survey and Evaluation Yinjie Lei Zixuan Wang Feng Chen Guoqing Wang Peng Wang Yang Yang 27 8 0 24 Oct 2023
Multi3DRefer: Grounding Text Description to Multiple 3D Objects Yiming Zhang ZeMing Gong Angel X. Chang 45 63 0 11 Sep 2023
Vote2Cap-DETR++: Decoupling Localization and Describing for End-to-End 3D Dense Captioning Sijin Chen Hongyuan Zhu Mingsheng Li Xin Chen Peng Guo Yinjie Lei Gang Yu Taihao Li Tao Chen 11 5 0 06 Sep 2023
Dense Object Grounding in 3D Scenes Wencan Huang Daizong Liu Wei Hu 13 17 0 05 Sep 2023
Cross3DVG: Cross-Dataset 3D Visual Grounding on Different RGB-D Scans Taiki Miyanishi Daich Azuma Shuhei Kurita M. Kawanabe 25 2 0 23 May 2023
Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding Zhang Tao Su He D. Tao Bin Chen Zhi Wang Shutao Xia VLM 21 21 0 18 May 2023
WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language Zhe-nan Lin Xidong Peng Peishan Cong Ge Zheng Yujin Sun Yuenan Hou Xinge Zhu Sibei Yang Yuexin Ma VGen 82 4 0 12 Apr 2023
Text2Tex: Text-driven Texture Synthesis via Diffusion Models Dave Zhenyu Chen Yawar Siddiqui Hsin-Ying Lee Sergey Tulyakov Matthias Nießner DiffM 22 185 0 20 Mar 2023
ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes Ahmed Abdelreheem Kyle Olszewski Hsin-Ying Lee Peter Wonka Panos Achlioptas 3DPC 20 28 0 12 Dec 2022
PointCLIP: Point Cloud Understanding by CLIP Renrui Zhang Ziyu Guo Wei Zhang Kunchang Li Xupeng Miao Bin Cui Yu Qiao Peng Gao Hongsheng Li VLM 3DPC 161 428 0 04 Dec 2021
InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring Zhihao Yuan Xu Yan Yinghong Liao Ruimao Zhang Sheng Wang Zhen Li Shuguang Cui 59 128 0 01 Mar 2021
ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes C. Qi Xinlei Chen Or Litany Leonidas J. Guibas 3DPC 178 239 0 29 Jan 2020
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 250 922 0 24 Sep 2019