47

Users Mispredict Their Own Preferences for AI Writing Assistance

Vivian Lai
Zana Buçinca
Nil-Jana Akpinar
Mo Houtti
Hyeonsu B. Kang
Kevin Chian
Namjoon Suh
Alex C. Williams
13 Figures
5 Tables
Appendix:22 Pages
Abstract

Proactive AI writing assistants need to predict when users want drafting help, yet we lack empirical understanding of what drives preferences. Through a factorial vignette study with 50 participants making 750 pairwise comparisons, we find compositional effort dominates decisions (ρ=0.597\rho = 0.597) while urgency shows no predictive power (ρ0\rho \approx 0). More critically, users exhibit a striking perception-behavior gap: they rank urgency first in self-reports despite it being the weakest behavioral driver, representing a complete preference inversion. This misalignment has measurable consequences. Systems designed from users' stated preferences achieve only 57.7\% accuracy, underperforming even naive baselines, while systems using behavioral patterns reach significantly higher 61.3\% (p<0.05p < 0.05). These findings demonstrate that relying on user introspection for system design actively misleads optimization, with direct implications for proactive natural language generation (NLG) systems.

View on arXiv
Comments on this paper