ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization

Human mesh recovery (HMR) from a single image is inherently ill-posed due to depth ambiguity and occlusions. Probabilistic methods have tried to solve this by generating numerous plausible 3D human mesh predictions, but they often exhibit misalignment with 2D image observations and weak robustness to in-the-wild images. To address these issues, we propose ADHMR, a framework that Aligns a Diffusion-based HMR model in a preference optimization manner. First, we train a human mesh prediction assessment model, HMR-Scorer, capable of evaluating predictions even for in-the-wild images without 3D annotations. We then use HMR-Scorer to create a preference dataset, where each input image has a pair of winner and loser mesh predictions. This dataset is used to finetune the base model using direct preference optimization. Moreover, HMR-Scorer also helps improve existing HMR models by data cleaning, even with fewer training samples. Extensive experiments show that ADHMR outperforms current state-of-the-art methods. Code is available at:this https URL.

@article{shen2025_2505.10250,
  title={ ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization },
  author={ Wenhao Shen and Wanqi Yin and Xiaofeng Yang and Cheng Chen and Chaoyue Song and Zhongang Cai and Lei Yang and Hao Wang and Guosheng Lin },
  journal={arXiv preprint arXiv:2505.10250},
  year={ 2025 }
}