R3-Avatar: Record and Retrieve Temporal Codebook for Reconstructing Photorealistic Human Avatars

We present R3-Avatar, incorporating a temporal codebook, to overcome the inability of human avatars to be both animatable and of high-fidelity rendering quality. Existing video-based reconstruction of 3D human avatars either focuses solely on rendering, lacking animation support, or learns a pose-appearance mapping for animating, which degrades under limited training poses or complex clothing. In this paper, we adopt a "record-retrieve-reconstruct" strategy that ensures high-quality rendering from novel views while mitigating degradation in novel poses. Specifically, disambiguating timestamps record temporal appearance variations in a codebook, ensuring high-fidelity novel-view rendering, while novel poses retrieve corresponding timestamps by matching the most similar training poses for augmented appearance. Our R3-Avatar outperforms cutting-edge video-based human avatar reconstruction, particularly in overcoming visual quality degradation in extreme scenarios with limited training human poses and complex clothing.

@article{zhan2025_2503.12751,
  title={ R3-Avatar: Record and Retrieve Temporal Codebook for Reconstructing Photorealistic Human Avatars },
  author={ Yifan Zhan and Wangze Xu and Qingtian Zhu and Muyao Niu and Mingze Ma and Yifei Liu and Zhihang Zhong and Xiao Sun and Yinqiang Zheng },
  journal={arXiv preprint arXiv:2503.12751},
  year={ 2025 }
}