Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Setting

The dyadic reaction generation task involves synthesizing responsive facial reactions that align closely with the behaviors of a conversational partner, enhancing the naturalness and effectiveness of human-like interaction simulations. This paper introduces a novel approach, the Latent Behavior Diffusion Model, comprising a context-aware autoencoder and a diffusion-based conditional generator that addresses the challenge of generating diverse and contextually relevant facial reactions from input speaker behaviors. The autoencoder compresses high-dimensional input features, capturing dynamic patterns in listener reactions while condensing complex input data into a concise latent representation, facilitating more expressive and contextually appropriate reaction synthesis. The diffusion-based conditional generator operates on the latent space generated by the autoencoder to predict realistic facial reactions in a non-autoregressive manner. This approach allows for generating diverse facial reactions that reflect subtle variations in conversational cues and emotional states. Experimental results demonstrate the effectiveness of our approach in achieving superior performance in dyadic reaction synthesis tasks compared to existing methods.

@article{nguyen2025_2505.07901,
  title={ Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Setting },
  author={ Minh-Duc Nguyen and Hyung-Jeong Yang and Soo-Hyung Kim and Ji-Eun Shin and Seung-Won Kim },
  journal={arXiv preprint arXiv:2505.07901},
  year={ 2025 }
}