Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting

13 September 2023

Papers citing "Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting"

28 / 28 papers shown

Title
Can Third-parties Read Our Emotions? Jiayi Li Yingfan Zhou Pranav Narayanan Venkit Halima Binte Islam Sneha Arya Shomir Wilson Sarah Rajtmajer 59 0 0 25 Apr 2025
Tell Me What You Know About Sexism: Expert-LLM Interaction Strategies and Co-Created Definitions for Zero-Shot Sexism Detection Myrthe Reuver Indira Sen Matteo Melis Gabriella Lapesa 20 0 0 21 Apr 2025
LLM Generated Persona is a Promise with a Catch Ang Li Haozhe Chen Hongseok Namkoong Tianyi Peng 53 1 0 18 Mar 2025
Evaluation and Facilitation of Online Discussions in the LLM Era: A Survey Katerina Korre Dimitris Tsirmpas Nikos Gkoumas Emma Cabalé Dionysis Kontarinis Danai Myrtzani Theodoros Evgeniou Ion Androutsopoulos John Pavlopoulos 35 1 0 03 Mar 2025
Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions Matthias Orlikowski Jiaxin Pei Paul Röttger Philipp Cimiano David Jurgens Dirk Hovy 54 1 0 28 Feb 2025
Designing Role Vectors to Improve LLM Inference Behaviour Daniele Potertì Andrea Seveso Fabio Mercorio LLMSV 42 0 0 17 Feb 2025
Adaptive Prompting: Ad-hoc Prompt Composition for Social Bias Detection Maximilian Spliethover Tim Knebler Fabian Fumagalli Maximilian Muschalik Barbara Hammer Eyke Hüllermeier Henning Wachsmuth 97 1 0 10 Feb 2025
SubData: A Python Library to Collect and Combine Datasets for Evaluating LLM Alignment on Downstream Tasks Leon Fröhling Pietro Bernardelle Gianluca Demartini ALM 74 0 0 21 Dec 2024
Personas with Attitudes: Controlling LLMs for Diverse Data Annotation Leon Fröhling Gianluca Demartini Dennis Assenmacher 24 2 0 15 Oct 2024
Which Demographics do LLMs Default to During Annotation? Johannes Schäfer Aidan Combs Christopher Bagdon Jiahui Li Nadine Probol ... Yarik Menchaca Resendiz Aswathy Velutharambath Amelie Wuhrl Sabine Weber Roman Klinger 30 2 0 11 Oct 2024
Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets Tommaso Giorgi Lorenzo Cima T. Fagni M. Avvenuti S. Cresci 40 9 0 10 Oct 2024
Re-examining Sexism and Misogyny Classification with Annotator Attitudes Aiqi Jiang Nikolas Vitsakis Tanvi Dinkar Gavin Abercrombie Ioannis Konstas 37 1 0 04 Oct 2024
The Lou Dataset -- Exploring the Impact of Gender-Fair Language in German Text Classification Andreas Waldis Joel Birrer Anne Lauscher Iryna Gurevych 23 1 0 26 Sep 2024
Self-Alignment: Improving Alignment of Cultural Values in LLMs via In-Context Learning Rochelle Choenni Ekaterina Shutova 32 6 0 29 Aug 2024
Voices in a Crowd: Searching for Clusters of Unique Perspectives Nikolas Vitsakis Amit Parekh Ioannis Konstas 36 0 0 19 Jul 2024
LLM Roleplay: Simulating Human-Chatbot Interaction Hovhannes Tamoyan Hendrik Schuff Iryna Gurevych 39 8 0 04 Jul 2024
Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance Manon Reusens Philipp Borchert Jochen De Weerdt Bart Baesens 16 0 0 25 Jun 2024
Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting Sagnik Mukherjee Muhammad Farid Adilazuarda Sunayana Sitaram Kalika Bali Alham Fikri Aji Monojit Choudhury 33 5 0 17 Jun 2024
The Potential and Challenges of Evaluating Attitudes, Opinions, and Values in Large Language Models Bolei Ma Xinpeng Wang Tiancheng Hu Anna Haensch Michael A. Hedderich Barbara Plank Frauke Kreuter ALM 33 2 0 16 Jun 2024
Annotation alignment: Comparing LLM and human annotations of conversational safety Rajiv Movva Pang Wei Koh Emma Pierson ALM 37 3 0 10 Jun 2024
Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art Chen Cecilia Liu Iryna Gurevych Anna Korhonen 33 5 0 06 Jun 2024
Prompting Techniques for Reducing Social Bias in LLMs through System 1 and System 2 Cognitive Processes M. Kamruzzaman Gene Louis Kim 45 15 0 26 Apr 2024
Annotator-Centric Active Learning for Subjective NLP Tasks Michiel van der Meer Neele Falk P. Murukannaiah Enrico Liscio 27 8 0 24 Apr 2024
Towards Measuring and Modeling "Culture" in LLMs: A Survey Muhammad Farid Adilazuarda Sagnik Mukherjee Pradhyumna Lavania Siddhant Singh Alham Fikri Aji Jacki OÑeill Ashutosh Modi Monojit Choudhury 52 53 0 05 Mar 2024
Quantifying the Persona Effect in LLM Simulations Tiancheng Hu Nigel Collier 22 51 0 16 Feb 2024
Can Large Language Models Capture Dissenting Human Voices? Noah Lee Na Min An James Thorne ALM 37 30 0 23 May 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 277 1,117 0 18 Apr 2021