v1v2 (latest)

The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection

17 November 2024

Papers citing "The Promises and Pitfalls of LLM Annotations in Dataset Labeling: a Case Study on Media Bias Detection"

29 / 29 papers shown

Title
Hateful Person or Hateful Model? Investigating the Role of Personas in Hate Speech Detection by Large Language Models Shuzhou Yuan Ercong Nie Mario Tawfelis Helmut Schmid Hinrich Schütze Michael Färber 20 0 0 10 Jun 2025
Leveraging Large Language Models for Automated Definition Extraction with TaxoMatic A Case Study on Media Bias Timo Spinde Luyang Lin Smi Hinterreiter Isao Echizen 94 0 0 01 Apr 2025
Through the LLM Looking Glass: A Socratic Probing of Donkeys, Elephants, and Markets Molly Kennedy Ayyoob Imani Timo Spinde Hinrich Schütze 100 0 0 20 Mar 2025
Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent Zeyu He Saniya Naphade Ting-Hao 'Kenneth' Huang 81 0 0 16 Feb 2025
Sentence-level Media Bias Analysis with Event Relation Graph Yuanyuan Lei Ruihong Huang 89 3 0 02 Apr 2024
MAGPIE: Multi-Task Media-Bias Analysis Generalization for Pre-Trained Identification of Expressions Tomávs Horych Martin Wessel Jan Philip Wahle Terry Ruas Jerome Wassmuth André Greiner-Petter Akiko Aizawa Bela Gipp Timo Spinde 75 2 0 27 Feb 2024
IndiVec: An Exploration of Leveraging Large Language Models for Media Bias Detection with Fine-Grained Bias Indicators Luyang Lin Lingzhi Wang Xiaoyan Zhao Jing Li Kam-Fai Wong 85 15 0 01 Feb 2024
The Media Bias Taxonomy: A Systematic Literature Review on the Forms and Automated Detection of Media Bias Timo Spinde Smilla Hinterreiter Fabian Haak Terry Ruas Helge Giese Norman Meuschke Bela Gipp 87 13 0 26 Dec 2023
Zephyr: Direct Distillation of LM Alignment Lewis Tunstall E. Beeching Nathan Lambert Nazneen Rajani Kashif Rasul ... Nathan Habib Nathan Sarrazin Omar Sanseviero Alexander M. Rush Thomas Wolf ALM 114 401 0 25 Oct 2023
OpenChat: Advancing Open-source Language Models with Mixed-Quality Data Guan-Bo Wang Sijie Cheng Xianyuan Zhan Xiangang Li Sen Song Yang Liu ALM 135 254 0 20 Sep 2023
Active Learning Principles for In-Context Learning with Large Language Models Katerina Margatina Timo Schick Nikolaos Aletras Jane Dwivedi-Yu 113 44 0 23 May 2023
Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP Anya Belz Craig Thomson Ehud Reiter Gavin Abercrombie J. Alonso-Moral ... Antonio Toral Xiao-Yi Wan Leo Wanner Lewis J. Watson Diyi Yang 149 36 0 02 May 2023
Introducing MBIB -- the first Media Bias Identification Benchmark Task and Dataset Collection Martin Wessel Tomávs Horych Terry Ruas Akiko Aizawa Bela Gipp Timo Spinde 83 25 0 25 Apr 2023
AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators Xingwei He Zheng-Wen Lin Yeyun Gong Alex Jin Hang Zhang Chen Lin Jian Jiao Siu-Ming Yiu Nan Duan Weizhu Chen 119 201 0 29 Mar 2023
ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks Fabrizio Gilardi Meysam Alizadeh M. Kubli AI4MH 149 935 0 27 Mar 2023
Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech Fan Huang Haewoon Kwak Jisun An AI4MH 75 267 0 11 Feb 2023
Exploiting Transformer-based Multitask Learning for the Detection of Media Bias in News Articles Timo Spinde Jan-David Krieger Terry Ruas Jelena Mitrović Franz Götz-Hahn Akiko Aizawa Bela Gipp 90 28 0 07 Nov 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 292 3,177 0 20 Oct 2022
Neural Media Bias Detection Using Distant Supervision With BABE -- Bias Annotations By Experts Timo Spinde Manuel Plank Jan-David Krieger Terry Ruas Bela Gipp Akiko Aizawa 67 76 0 29 Sep 2022
A Domain-adaptive Pre-training Approach for Language Bias Detection in News Jan-David Krieger Timo Spinde Terry Ruas Juhi Kulshrestha Bela Gipp AI4CE 85 21 0 22 May 2022
Towards A Reliable Ground-Truth For Biased Language Detection Timo Spinde David Krieger Manuel Plank Bela Gipp 61 18 0 14 Dec 2021
Do You Think It's Biased? How To Ask For The Perception Of Media Bias Timo Spinde Christina Kreuter W. Gaissmaier Felix Hamborg Bela Gipp H. Giese 82 23 0 14 Dec 2021
Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Development M. Scheuerman Emily L. Denton A. Hanna 81 215 0 09 Aug 2021
Towards Understanding and Mitigating Social Biases in Language Models Paul Pu Liang Chiyu Wu Louis-Philippe Morency Ruslan Salakhutdinov 102 399 0 24 Jun 2021
MBIC -- A Media Bias Annotation Dataset Including Annotator Characteristics Timo Spinde L. Rudnitckaia K. Sinha Felix Hamborg Bela Gipp K. Donnay 58 46 0 20 May 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 398 1,400 0 17 Jan 2021
Beyond Accuracy: Behavioral Testing of NLP models with CheckList Marco Tulio Ribeiro Tongshuang Wu Carlos Guestrin Sameer Singh ELM 217 1,110 0 08 May 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 602 20,418 0 23 Oct 2019
In Plain Sight: Media Bias Through the Lens of Factual Reporting Lisa Fan M. White Eva Sharma Ruisi Su Prafulla Kumar Choubey Ruihong Huang Lu Wang 62 110 0 05 Sep 2019