Xpress: A System For Dynamic, Context-Aware Robot Facial Expressions using Language Models

Facial expressions are vital in human communication and significantly influence outcomes in human-robot interaction (HRI), such as likeability, trust, and companionship. However, current methods for generating robotic facial expressions are often labor-intensive, lack adaptability across contexts and platforms, and have limited expressive ranges--leading to repetitive behaviors that reduce interaction quality, particularly in long-term scenarios. We introduce Xpress, a system that leverages language models (LMs) to dynamically generate context-aware facial expressions for robots through a three-phase process: encoding temporal flow, conditioning expressions on context, and generating facial expression code. We demonstrated Xpress as a proof-of-concept through two user studies (n=15x2) and a case study with children and parents (n=13), in storytelling and conversational scenarios to assess the system's context-awareness, expressiveness, and dynamism. Results demonstrate Xpress's ability to dynamically produce expressive and contextually appropriate facial expressions, highlighting its versatility and potential in HRI applications.

@article{antony2025_2503.00283,
  title={ Xpress: A System For Dynamic, Context-Aware Robot Facial Expressions using Language Models },
  author={ Victor Nikhil Antony and Maia Stiber and Chien-Ming Huang },
  journal={arXiv preprint arXiv:2503.00283},
  year={ 2025 }
}