Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations

We present an evaluation framework for interactive dialogue assessment in the context of English as a Second Language (ESL) speakers. Our framework collects dialogue-level interactivity labels (e.g., topic management; 4 labels in total) and micro-level span features (e.g., backchannels; 17 features in total). Given our annotated data, we study how the micro-level features influence the (higher level) interactivity quality of ESL dialogues by constructing various machine learning-based models. Our results demonstrate that certain micro-level features strongly correlate with interactivity quality, like reference word (e.g., she, her, he), revealing new insights about the interaction between higher-level dialogue quality and lower-level linguistic signals. Our framework also provides a means to assess ESL communication, which is useful for language assessment.

@article{gao2025_2407.06479,
  title={ Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversations },
  author={ Rena Gao and Carsten Roever and Jey Han Lau },
  journal={arXiv preprint arXiv:2407.06479},
  year={ 2025 }
}