Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration

We introduce Dr. Splat, a novel approach for open-vocabulary 3D scene understanding leveraging 3D Gaussian Splatting. Unlike existing language-embedded 3DGS methods, which rely on a rendering process, our method directly associates language-aligned CLIP embeddings with 3D Gaussians for holistic 3D scene understanding. The key of our method is a language feature registration technique where CLIP embeddings are assigned to the dominant Gaussians intersected by each pixel-ray. Moreover, we integrate Product Quantization (PQ) trained on general large-scale image data to compactly represent embeddings without per-scene optimization. Experiments demonstrate that our approach significantly outperforms existing approaches in 3D perception benchmarks, such as open-vocabulary 3D semantic segmentation, 3D object localization, and 3D object selection tasks. For video results, please visit :this https URL

@article{jun-seong2025_2502.16652,
  title={ Dr. Splat: Directly Referring 3D Gaussian Splatting via Direct Language Embedding Registration },
  author={ Kim Jun-Seong and GeonU Kim and Kim Yu-Ji and Yu-Chiang Frank Wang and Jaesung Choe and Tae-Hyun Oh },
  journal={arXiv preprint arXiv:2502.16652},
  year={ 2025 }
}