Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlines

Even to a simple and short news headline, readers react in a multitude of ways: cognitively (e.g., inferring the writer's intent), emotionally (e.g., feeling distrust), and behaviorally (e.g., sharing the news with their friends). Such reactions are instantaneous and yet complex, as they rely on factors that go beyond interpreting the factual content of the news headline. Instead, understanding reactions requires pragmatic understanding of the news headline, including broader background knowledge about contentious news topics as well as commonsense reasoning about people's intents and emotional reactions. We propose Misinfo Reaction Frames, a pragmatic formalism for modeling how readers might react to a news headline cognitively, emotionally, and behaviorally. We also introduce a Misinfo Reaction Frames corpus, a dataset of over 200k news headline/annotated dimension pairs with crowdsourced reactions focusing on global crises: the Covid-19 pandemic, climate change, and cancer. Empirical results confirm that it is indeed possible to learn the prominent patterns of readers' reactions to news headlines. We also find a potentially positive use case of our model; When we present our model generated inferences to people, we find that the machine inferences can increase readers' trust in real news while decreasing their trust in misinformation. Our work demonstrates the feasibility and the importance of pragmatic inferences of news to help enhance AI-guided misinformation detection and mitigation.