ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

27 March 2023

Papers citing "ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks"

50 / 489 papers shown

Title
QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluation Mengze Hong Wailing Ng Di Jiang Chen Zhang ELM 51 0 0 08 May 2025
Synthline: A Product Line Approach for Synthetic Requirements Engineering Data Generation using Large Language Models Abdelkarim El-Hajjami Camille Salinesi SyDa 27 0 0 06 May 2025
A Simple Ensemble Strategy for LLM Inference: Towards More Stable Text Classification Junichiro Niimi 48 0 0 26 Apr 2025
The Influence of Text Variation on User Engagement in Cross-Platform Content Sharing Yibo Hu Yiqiao Jin Meng Ye Ajay Divakaran Srijan Kumar 10 0 0 26 Apr 2025
Can Third-parties Read Our Emotions? Jiayi Li Yingfan Zhou Pranav Narayanan Venkit Halima Binte Islam Sneha Arya Shomir Wilson Sarah Rajtmajer 33 0 0 25 Apr 2025
DataScout: Automatic Data Fact Retrieval for Statement Augmentation with an LLM-Based Agent Chuer Chen Yuqi Liu Danqing Shi Shixiong Cao Nan Cao 35 0 0 24 Apr 2025
Deep literature reviews: an application of fine-tuned language models to migration research Stefano M. Iacus Haodong Qi Jiyoung Han 18 0 0 17 Apr 2025
AI Safety Should Prioritize the Future of Work Sanchaita Hazra Bodhisattwa Prasad Majumder Tuhin Chakrabarty 27 0 0 16 Apr 2025
Large Language Model-Informed Feature Discovery Improves Prediction and Interpretation of Credibility Perceptions of Visual Content Yilang Peng Sijia Qian Yingdan Lu Cuihua Shen 19 0 0 15 Apr 2025
The Art of Audience Engagement: LLM-Based Thin-Slicing of Scientific Talks Ralf Schmälzle Sue Lim Yuetong Du Gary Bente 16 0 0 15 Apr 2025
A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future Jialun Zhong Wei Shen Yanzeng Li Songyang Gao Hua Lu Yicheng Chen Yang Zhang Wei Zhou Jinjie Gu Lei Zou LRM 38 1 0 12 Apr 2025
A Fully Automated Pipeline for Conversational Discourse Annotation: Tree Scheme Generation and Labeling with Large Language Models Kseniia Petukhova Ekaterina Kochmar 25 0 0 11 Apr 2025
Leveraging LLMs for Utility-Focused Annotation: Reducing Manual Effort for Retrieval and RAG Hengran Zhang Minghao Tang Keping Bi J. Guo Shihao Liu Daiting Shi Dawei Yin Xueqi Cheng 14 0 0 07 Apr 2025
ArxivBench: Can LLMs Assist Researchers in Conducting Research? Ning Li Jingran Zhang Justin Cui 19 0 0 06 Apr 2025
AI-induced sexual harassment: Investigating Contextual Characteristics and User Reactions of Sexual Harassment by a Companion Chatbot Mohammad Namvarpour Harrison Pauwels Afsaneh Razi 26 0 0 05 Apr 2025
The Lyme Disease Controversy: An AI-Driven Discourse Analysis of a Quarter Century of Academic Debate and Divides Teo Susnjak Cole Palffy Tatiana Zimina Nazgul Altynbekova Kunal Garg Leona Gilbert 31 0 0 04 Apr 2025
Hide and Seek in Noise Labels: Noise-Robust Collaborative Active Learning with LLM-Powered Assistance Bo Yuan Yulin Chen Yin Zhang Wei Jiang NoLa 30 5 0 03 Apr 2025
Mapping Geopolitical Bias in 11 Large Language Models: A Bilingual, Dual-Framing Analysis of U.S.-China Tensions William Guey Pierrick Bougault Vitor D. de Moura Wei Zhang Jose O. Gomes 27 0 0 31 Mar 2025
Towards Understanding How Knowledge Evolves in Large Vision-Language Models Sudong Wang Y. Zhang Yao Zhu Jianing Li Zizhe Wang Y. Liu Xiangyang Ji 40 0 0 31 Mar 2025
Elite Political Discourse has Become More Toxic in Western Countries Petter Törnberg Juliana Chueri 41 0 0 28 Mar 2025
The Risks of Using Large Language Models for Text Annotation in Social Science Research Hao Lin Yongjun Zhang 24 2 0 27 Mar 2025
Synthetic Data Augmentation for Cross-domain Implicit Discourse Relation Recognition Frances Yung Varsha Suresh Zaynab Reza Mansoor Ahmad Vera Demberg 75 0 0 26 Mar 2025
"Whose Side Are You On?" Estimating Ideology of Political and News Content Using Large Language Models and Few-shot Demonstration Selection Muhammad Haroon Magdalena Wojcieszak Anshuman Chhabra 41 0 0 23 Mar 2025
Assessing the Reliability and Validity of GPT-4 in Annotating Emotion Appraisal Ratings Deniss Ruder Andero Uusberg Kairit Sirts 37 0 0 21 Mar 2025
TreeSynth: Synthesizing Diverse Data from Scratch via Tree-Guided Subspace Partitioning Sheng Wang Pengan Chen Jingqi Zhou Qintong Li Jingwei Dong Jiahui Gao Boyang Xue Jiyue Jiang Lingpeng Kong Chuan Wu SyDa 61 0 0 21 Mar 2025
Agent-Enhanced Large Language Models for Researching Political Institutions Joseph R. Loffredo Suyeol Yun LLMAG 59 0 0 14 Mar 2025
Augmenting Image Annotation: A Human-LMM Collaborative Framework for Efficient Object Selection and Label Generation He Zhang Xinyi Fu John M. Carroll VLM 51 0 0 14 Mar 2025
Have LLMs Made Active Learning Obsolete? Surveying the NLP Community Julia Romberg Christopher Schröder Julius Gonsior Katrin Tomanek Fredrik Olsson 57 0 0 12 Mar 2025
How Do Hackathons Foster Creativity? Towards AI Collaborative Evaluation of Creativity at Scale Jeanette Falk Yiyi Chen Janet Rafner Mike Zhang Johannes Bjerva Alexander Nolte 53 1 0 06 Mar 2025
Dynamic-KGQA: A Scalable Framework for Generating Adaptive Question Answering Datasets Preetam Prabhu Srikar Dammu Himanshu Naidu Chirag Shah 42 0 0 06 Mar 2025
Biases in Large Language Model-Elicited Text: A Case Study in Natural Language Inference Grace Proebsting Adam Poliak 40 0 0 06 Mar 2025
Embracing Diversity: A Multi-Perspective Approach with Soft Labels Benedetta Muscato Praveen Bushipaka Gizem Gezici Lucia Passaro F. Giannotti Tommaso Cucinotta 34 0 0 01 Mar 2025
Towards Conditioning Clinical Text Generation for User Control Osman Alperen Koras Rabi Bahnan Jens Kleesiek Amin Dada 31 0 0 24 Feb 2025
Text-to-SQL Domain Adaptation via Human-LLM Collaborative Data Annotation Yuan Tian Daniel Lee Fei Wu Tung Mai Kun Qian Siddhartha Sahai Tianyi Zhang Yunyao Li SyDa 43 0 0 21 Feb 2025
SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation Song Duong Florian Le Bronnec Alexandre Allauzen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari HILM 38 0 0 20 Feb 2025
Escaping Collapse: The Strength of Weak Data for Large Language Model Training Kareem Amin Sara Babakniya Alex Bie Weiwei Kong Umar Syed Sergei Vassilvitskii 68 1 0 13 Feb 2025
Measuring Diversity in Synthetic Datasets Yuchang Zhu Huizhe Zhang Bingzhe Wu Jintang Li Zibin Zheng Peilin Zhao Liang Chen Yatao Bian 95 0 0 12 Feb 2025
AI Alignment at Your Discretion Maarten Buyl Hadi Khalaf C. M. Verdun Lucas Monteiro Paes Caio Vieira Machado Flavio du Pin Calmon 33 0 0 10 Feb 2025
Scaling Public Health Text Annotation: Zero-Shot Learning vs. Crowdsourcing for Improved Efficiency and Labeling Accuracy Kamyar Kazari Yong Chen Zahra Shakeri AI4MH 48 0 0 10 Feb 2025
VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data Thomas Zeng Shuibai Zhang Shutong Wu Christian Classen Daewon Chae ... Jungtaek Kim H. Koo K. Ramchandran Dimitris Papailiopoulos Kangwook Lee LRM 52 2 0 10 Feb 2025
Few-shot LLM Synthetic Data with Distribution Matching Jiyuan Ren Zhaocheng Du Zhihao Wen Qinglin Jia Sunhao Dai Chuhan Wu Zhenhua Dong SyDa 73 0 0 09 Feb 2025
Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generation Yin Wang Mu Li Jiapeng Liu Zhiying Leng Frederick W. B. Li Ziyao Zhang Xiaohui Liang 51 2 0 08 Feb 2025
Aligning Black-box Language Models with Human Judgments Gerrit J. J. van den Burg Gen Suzuki Wei Liu Murat Sensoy ALM 66 0 0 07 Feb 2025
Can LLMs Rank the Harmfulness of Smaller LLMs? We are Not There Yet Berk Atil Vipul Gupta Sarkar Snigdha Sarathi Das R. Passonneau 78 0 0 07 Feb 2025
Uncovering Latent Arguments in Social Media Messaging by Employing LLMs-in-the-Loop Strategy Tunazzina Islam Dan Goldwasser 76 2 0 28 Jan 2025
Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models Patrick Y. Wu Jonathan Nagler Joshua A. Tucker Solomon Messing LRM 46 2 0 28 Jan 2025
Automatic Labelling with Open-source LLMs using Dynamic Label Schema Integration Thomas Walshe S. Moon Chunyang Xiao Yawwani Gunawardana Fran Silavong 35 0 0 21 Jan 2025
LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs Tongshuang Wu Haiyi Zhu Maya Albayrak Alexis Axon Amanda Bertsch ... Ying-Jui Tseng Patricia Vaidos Zhijin Wu Wei Yu Wu Chenyang Yang 64 27 0 10 Jan 2025
Predictable Artificial Intelligence Lexin Zhou Pablo Antonio Moreno Casares Fernando Martínez-Plumed John Burden Ryan Burnell ... Seán Ó hÉigeartaigh Danaja Rutar Wout Schellaert Konstantinos Voudouris José Hernández Orallo 36 2 0 08 Jan 2025
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts Helia Hashemi J. Eisner Corby Rosset Benjamin Van Durme Chris Kedzie 63 1 0 03 Jan 2025