IMFine: 3D Inpainting via Geometry-guided Multi-view Refinement

Current 3D inpainting and object removal methods are largely limited to front-facing scenes, facing substantial challenges when applied to diverse, "unconstrained" scenes where the camera orientation and trajectory are unrestricted. To bridge this gap, we introduce a novel approach that produces inpainted 3D scenes with consistent visual quality and coherent underlying geometry across both front-facing and unconstrained scenes. Specifically, we propose a robust 3D inpainting pipeline that incorporates geometric priors and a multi-view refinement network trained via test-time adaptation, building on a pre-trained image inpainting model. Additionally, we develop a novel inpainting mask detection technique to derive targeted inpainting masks from object masks, boosting the performance in handling unconstrained scenes. To validate the efficacy of our approach, we create a challenging and diverse benchmark that spans a wide range of scenes. Comprehensive experiments demonstrate that our proposed method substantially outperforms existing state-of-the-art approaches.

@article{shi2025_2503.04501,
  title={ IMFine: 3D Inpainting via Geometry-guided Multi-view Refinement },
  author={ Zhihao Shi and Dong Huo and Yuhongze Zhou and Kejia Yin and Yan Min and Juwei Lu and Xinxin Zuo },
  journal={arXiv preprint arXiv:2503.04501},
  year={ 2025 }
}