Do LLMs exhibit human-like response biases? A case study in survey design

7 November 2023

Graham Neubig

Papers citing "Do LLMs exhibit human-like response biases? A case study in survey design"

24 / 24 papers shown

Title
Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications Wenhan Dong Yuemeng Zhao Zhen Sun Yule Liu Zifan Peng ... Jun Wu Ruiming Wang Shengmin Xu Xinyi Huang Xinlei He LLMAG 57 0 0 30 Apr 2025
Improving LLM Personas via Rationalization with Psychological Scaffolds Brihi Joshi Xiang Ren Swabha Swayamdipta Rik Koncel-Kedziorski Tim Paek 68 0 0 25 Apr 2025
Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions Saffron Huang Esin Durmus Miles McCain Kunal Handa Alex Tamkin Jerry Hong Michael Stern Arushi Somani Xiuruo Zhang Deep Ganguli VLM 42 1 0 21 Apr 2025
Cognitive Debiasing Large Language Models for Decision-Making Yougang Lyu Shijie Ren Yue Feng Zihan Wang Z. Chen Z. Z. Ren Maarten de Rijke 36 0 0 05 Apr 2025
Validating LLM-as-a-Judge Systems in the Absence of Gold Labels Luke M. Guerdan Solon Barocas Kenneth Holstein Hanna M. Wallach Zhiwei Steven Wu Alexandra Chouldechova ALM ELM 168 0 0 13 Mar 2025
Who Relies More on World Knowledge and Bias for Syntactic Ambiguity Resolution: Humans or LLMs? So Young Lee Russell Scheinberg Amber Shore Ameeta Agrawal 46 1 0 13 Mar 2025
From Idea to Implementation: Evaluating the Influence of Large Language Models in Software Development -- An Opinion Paper Sargam Yadav Asifa Mehmood Qureshi Abhishek Kaushik Shubham Sharma Roisin Loughran ... . Nikhil Singh Padraic O'Hara Pranay Jaiswal Roshan Chandru David Lillis 56 1 0 10 Mar 2025
Aligning Black-box Language Models with Human Judgments Gerrit J. J. van den Burg Gen Suzuki Wei Liu Murat Sensoy ALM 71 0 0 07 Feb 2025
A Dynamic and High-Precision Method for Scenario-Based HRA Synthetic Data Collection in Multi-Agent Collaborative Environments Driven by LLMs Xingyu Xiao Peng Chen Qianqian Jia Jiejuan Tong Jingang Liang Haitao Wang 67 0 0 16 Jan 2025
Extracting Affect Aggregates from Longitudinal Social Media Data with Temporal Adapters for Large Language Models Georg Ahnert Max Pellert David García M. Strohmaier 35 0 0 10 Jan 2025
Undesirable Memorization in Large Language Models: A Survey Ali Satvaty Suzan Verberne Fatih Turkmen ELM PILM 69 7 0 03 Oct 2024
Mitigating Selection Bias with Node Pruning and Auxiliary Options Hyeong Kyu Choi Weijie Xu Chi Xue Stephanie Eckman Chandan K. Reddy 29 1 0 27 Sep 2024
United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections Leah von der Heyde Anna Haensch Alexander Wenz Bolei Ma 52 2 0 29 Aug 2024
Can Model Uncertainty Function as a Proxy for Multiple-Choice Question Item Difficulty? Leonidas Zotos H. Rijn Malvina Nissim ELM 37 2 0 07 Jul 2024
Can Large Language Models Beat Wall Street? Unveiling the Potential of AI in Stock Selection G. Fatouros Konstantinos Metaxas John Soldatos D. Kyriazis AIFin 36 20 0 08 Jan 2024
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation Patrick Fernandes Aman Madaan Emmy Liu António Farinhas Pedro Henrique Martins ... José G. C. de Souza Shuyan Zhou Tongshuang Wu Graham Neubig André F. T. Martins ALM 113 56 0 01 May 2023
Generative Agents: Interactive Simulacra of Human Behavior J. Park Joseph C. O'Brien Carrie J. Cai Meredith Ringel Morris Percy Liang Michael S. Bernstein LM&Ro AI4CE 215 1,733 0 07 Apr 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 293 4,077 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,909 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,448 0 28 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 211 1,656 0 15 Oct 2021
Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations Arabella J. Sinclair Jaap Jumelet Willem H. Zuidema Raquel Fernández 54 38 0 30 Sep 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 277 1,117 0 18 Apr 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,916 0 31 Dec 2020