Papers citing 'Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks'

Title
Repurposing Annotation Guidelines to Instruct LLM Annotators: A Case StudyInternational Conference on Applications of Natural Language to Data Bases (NLDB), 2025 Kon Woo Kim Rezarta Islamaj Jin-Dong Kim Florian Boudin Akiko Aizawa 84 0 0 13 Oct 2025
Specializing General-purpose LLM Embeddings for Implicit Hate Speech Detection across Datasets Vassiliy Cheremetiev Quang Long Ho Ngo Chau Ying Kot Alina Elena Baia Andrea Cavallaro 80 0 0 28 Aug 2025
Neither Valid nor Reliable? Investigating the Use of LLMs as Judges Khaoula Chehbouni Mohammed Haddou Jackie CK Cheung G. Farnadi LLMAG 297 6 0 25 Aug 2025
Evaluating Large Language Models as Expert Annotators Yu-Min Tseng Wei-Lin Chen Chung-Chi Chen Hsin-Hsi Chen 116 1 0 11 Aug 2025
Journalism-Guided Agentic In-Context Learning for News Stance Detection Dahyun Lee Jonghyeon Choi Jiyoung Han Kunwoo Park 181 0 0 15 Jul 2025
Evaluating Multimodal Large Language Models on Video Captioning via Monte Carlo Tree SearchAnnual Meeting of the Association for Computational Linguistics (ACL), 2025 Linhao Yu Xinguang Ji Yahui Liu Fanheng Kong Chenxi Sun Jingyuan Zhang Hongzhi Zhang Victoria A. Webster-Wood Fuzheng Zhang Deyi Xiong 169 2 0 11 Jun 2025
Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement Meghaj Tarte Agam Shah Chao Zhang Sudheer Chava 277 1 0 26 May 2025
Think Together and Work Better: Combining Humans' and LLMs' Think-Aloud Outcomes for Effective Text EvaluationInternational Conference on Human Factors in Computing Systems (CHI), 2024 SeongYeub Chu JongWoo Kim MunYong Yi 310 12 0 21 Feb 2025
A Large-Scale Benchmark for Vietnamese Sentence ParaphrasesNorth American Chapter of the Association for Computational Linguistics (NAACL), 2025 Sang Quang Nguyen Kiet Van Nguyen 323 0 0 11 Feb 2025
Scaling Public Health Text Annotation: Zero-Shot Learning vs. Crowdsourcing for Improved Efficiency and Labeling Accuracy Kamyar Kazari Yong Chen Zahra Shakeri AI4MH 262 3 0 10 Feb 2025
Can LLMs Rank the Harmfulness of Smaller LLMs? We are Not There Yet Berk Atil Vipul Gupta Sarkar Snigdha Sarathi Das R. Passonneau 767 0 0 07 Feb 2025
SkillScope: A Tool to Predict Fine-Grained Skills Needed to Solve Issues on GitHub Benjamin C. Carter Jonathan Rivas Contreras Carlos A. Llanes Villegas Pawan Acharya Jack Utzerath ... Dylan Johnson Jacob Penney Igor Steinmacher M. Gerosa Fabio Santos 297 1 0 28 Jan 2025
Automated Collection of Evaluation Dataset for Semantic Search in Low-Resource Domain Language Anastasia Zhukova Christian E. Matt Bela Gipp 393 4 0 13 Dec 2024
MedHalu: Hallucinations in Responses to Healthcare Queries by Large Language ModelsSocial Science Research Network (SSRN), 2025 Vibhor Agarwal Yiqiao Jin Mohit Chandra M. D. Choudhury Srijan Kumar Nishanth R. Sastry HILM LM&MA 317 22 0 29 Sep 2024
Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AIInternational Conference on Web and Social Media (ICWSM), 2024 Nicholas Pangakis Samuel Wolken 275 13 0 14 Sep 2024
CodeMirage: Hallucinations in Code Generated by Large Language Models Vibhor Agarwal Yulong Pei Salwa Alamir Xiaomo Liu 267 16 0 14 Aug 2024
The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representation Maja Pavlovic Massimo Poesio 334 30 0 02 May 2024
An Experiment with the Use of ChatGPT for LCSH Subject Assignment on Electronic Theses and Dissertations Eric H. C. Chow TJ Kao Xiaoli Li 192 15 0 25 Mar 2024
Generating Hard-Negative Out-of-Scope Data with ChatGPT for Intent ClassificationInternational Conference on Language Resources and Evaluation (LREC), 2024 Zhijian Li Stefan Larson Kevin Leach OODD 161 3 0 08 Mar 2024
MEGAnno+: A Human-LLM Collaborative Annotation System H. Kim Kushan Mitra Rafael Li Chen Sajjadur Rahman Dan Zhang 269 49 0 28 Feb 2024
If in a Crowdsourced Data Annotation Pipeline, a GPT-4 Zeyu He Huang Chieh-Yang C. C. Ding Shaurya Rohatgi Ting-Hao 'Kenneth' Huang 314 49 0 26 Feb 2024
AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators Jingwei Ni Minjing Shi Dominik Stammbach Mrinmaya Sachan Elliott Ash Markus Leippold HILM 325 28 0 16 Feb 2024
APT-Pipe: A Prompt-Tuning Tool for Social Data Annotation using ChatGPTThe Web Conference (WWW), 2024 Yiming Zhu Zhizhuo Yin Gareth Tyson Ehsan-ul Haq Lik-Hang Lee Pan Hui ALM 297 12 0 24 Jan 2024
Adaptive Crowdsourcing Via Self-Supervised Learning Anmol Kagrecha Henrik Marklund Benjamin Van Roy Hong Jun Jeon Richard Zeckhauser FedML 287 0 0 24 Jan 2024
DepressionEmo: A novel dataset for multilabel classification of depression emotionsJournal of Affective Disorders (J Affect Disord), 2024 Abu Bakar Siddiqur Rahman Hoang-Thang Ta Lotfollah Najjar A. Azadmanesh A. Gonul AI4MH 248 20 0 09 Jan 2024
The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model PerformanceAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 A. Salinas Fred Morstatter 267 82 0 08 Jan 2024
Exploring news intent and its application: A theory-driven approach Zhengjia Wang Danding Wang Qiang Sheng Juan Cao Silong Su Yifan Sun 328 5 0 27 Dec 2023
Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM InteractionsWeb Search and Data Mining (WSDM), 2023 Zahra Abbasiantaeb Yifei Yuan Evangelos Kanoulas Mohammad Aliannejadi 270 107 0 05 Dec 2023
Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation Eun Cheol Choi Emilio Ferrara 185 29 0 13 Oct 2023
Multiclass Classification of Policy Documents with Large Language Models Erkan Gunes Christoffer Florczak 211 1 0 12 Oct 2023
AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial TopicsInternational Conference on Information and Knowledge Management (CIKM), 2023 V. Ghafouri Vibhor Agarwal Yong Zhang Nishanth R. Sastry Jose Such Guillermo Suarez-Tangil AI4MH 216 27 0 28 Aug 2023
Time Travel in LLMs: Tracing Data Contamination in Large Language ModelsInternational Conference on Learning Representations (ICLR), 2023 Shahriar Golchin Mihai Surdeanu 394 138 0 16 Aug 2023
Can Large Language Models Aid in Annotating Speech Emotional Data? Uncovering New FrontiersIEEE Computational Intelligence Magazine (IEEE CIM), 2023 S. Latif Muhammad Usama Mohammad Ibrahim Malik Björn W. Schuller 280 20 0 12 Jul 2023
Dream Content Discovery from Reddit with an Unsupervised Mixed-Method ApproachEPJ Data Science (EPJ Data Sci.), 2023 Anubhab Das Sanja vSćepanović L. Aiello Remington Mallett D. Barrett Daniele Quercia 97 1 0 09 Jul 2023
Automated Annotation with Generative AI Requires Validation Nicholas Pangakis Samuel Wolken Neil Fasching SyDa 279 93 0 31 May 2023
Using Natural Language Explanations to Rescale Human Judgments Manya Wadhwa Jifan Chen Junyi Jessy Li Greg Durrett 270 11 0 24 May 2023
ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model RobustnessConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Ján Cegin Jakub Simko Peter Brusilovsky 221 57 0 22 May 2023
SMILE: Single-turn to Multi-turn Inclusive Language Expansion via ChatGPT for Mental Health SupportConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Huachuan Qiu Hongliang He Shuai Zhang Anqi Li Zhenzhong Lan AI4MH 371 50 0 30 Apr 2023
The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification TasksConference of the European Chapter of the Association for Computational Linguistics (EACL), 2023 Anders Giovanni Møller Jacob Aarup Dalsgaard Arianna Pera L. Aiello 233 57 0 26 Apr 2023
AGI: Artificial General Intelligence for Education Ehsan Latif Gengchen Mai Matthew Nyaaba Xuansheng Wu Ninghao Liu Guoyu Lu Sheng Li Tianming Liu Xiaoming Zhai ELM AI4CE 304 30 0 24 Apr 2023
Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses? Jaromír Šavelka Arav Agarwal Chris Bogart Yifan Song M. Sakr ELM 139 114 0 16 Mar 2023
MiDe22: An Annotated Multi-Event Tweet Dataset for Misinformation DetectionInternational Conference on Language Resources and Evaluation (LREC), 2022 Cagri Toraman Oguzhan Ozcelik Furkan Şahinuç Fazli Can 129 15 0 11 Oct 2022
Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary DebatesConference on Empirical Methods in Natural Language Processing (EMNLP), 2022 Aida Kostikova Benjamin Paassen Dominik Beese Ole Putz Gregor Wiedemann Steffen Eger 159 6 0 09 Oct 2022