ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

27 March 2023

Papers citing "ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks"

50 / 489 papers shown

Title
Hate Personified: Investigating the role of LLMs in content moderation Sarah Masud Sahajpreet Singh Viktor Hangya Alexander M. Fraser Tanmoy Chakraborty 17 7 0 03 Oct 2024
Mixed-Session Conversation with Egocentric Memory Jihyoung Jang Taeyoung Kim Hyounghun Kim 18 0 0 03 Oct 2024
Comparing Criteria Development Across Domain Experts, Lay Users, and Models in Large Language Model Evaluation Annalisa Szymanski Simret Araya Gebreegziabher Oghenemaro Anuyah Ronald A Metoyer T. Li ALM ELM 27 6 0 02 Oct 2024
Evaluating Robustness of Reward Models for Mathematical Reasoning Sunghwan Kim Dongjin Kang Taeyoon Kwon Hyungjoo Chae Jungsoo Won Dongha Lee Jinyoung Yeo 23 4 0 02 Oct 2024
'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants Shivani Kapania William Agnew Motahhare Eslami Hoda Heidari Sarah E Fox 34 4 0 28 Sep 2024
Learning to Love Edge Cases in Formative Math Assessment: Using the AMMORE Dataset and Chain-of-Thought Prompting to Improve Grading Accuracy Owen Henkel Hannah Horne-Robinson Maria Dyshel Nabil Ch Baptiste Moreau-Pernet Ralph Abood 25 0 0 26 Sep 2024
Setting the AI Agenda -- Evidence from Sweden in the ChatGPT Era Bastiaan Bruinsma Annika Fredén Kajsa Hansson Moa Johansson Pasko Kisić-Merino Denitsa Saynova 31 0 0 25 Sep 2024
AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment Nuo Chen Jiqun Liu Xiaoyu Dong Qijiong Liu Tetsuya Sakai Xiao-Ming Wu 13 9 0 24 Sep 2024
Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment Yuxiao Chen K. Li Wentao Bao Deep Patel Yu Kong Martin Renqiang Min Dimitris N. Metaxas DiffM 31 1 0 22 Sep 2024
Collaborative Human-AI Risk Annotation: Co-Annotating Online Incivility with CHAIRA J. Park Rahul Dev Ellezhuthil Pamela J. Wisniewski Vivek Singh 27 1 0 21 Sep 2024
MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Models Sarfaroz Yunusov Hamza Sidat Ali Emami 72 0 0 20 Sep 2024
What Would You Ask When You First Saw $a^2+b^2=c^2$ ? Evaluating LLM on Curiosity-Driven Questioning Shashidhar Reddy Javaji Zining Zhu ELM ALM 34 0 0 19 Sep 2024
LLM-Measure: Generating Valid, Consistent, and Reproducible Text-Based Measures for Social Science Research Yi Yang Hanyu Duan Jiaxin Liu Kar Yan Tam 11 0 0 19 Sep 2024
Human Interest or Conflict? Leveraging LLMs for Automated Framing Analysis in TV Shows David Alonso del Barrio Max Tiel D. Gática-Pérez 31 3 0 19 Sep 2024
ARTICLE: Annotator Reliability Through In-Context Learning Sujan Dutta Deepak Pandita Tharindu Cyril Weerasooriya Marcos Zampieri Christopher M. Homan Ashiqur R. KhudaBukhsh 17 0 0 18 Sep 2024
Learning variant product relationship and variation attributes from e-commerce website structures Pedro Herrero-Vidal You-Lin Chen Cris Liu Prithviraj Sen Lichao Wang 24 0 0 17 Sep 2024
Model-in-the-Loop (MILO): Accelerating Multimodal AI Data Annotation with LLMs Yifan Wang David Stevens Pranay Shah Wenwen Jiang Miao Liu ... Boying Gong Daniel Lee Jiabo Hu Ning Zhang Bob Kamma 30 1 0 16 Sep 2024
Benchmarking LLMs in Political Content Text-Annotation: Proof-of-Concept with Toxicity and Incivility Data Bastián González-Bustamante 18 1 0 15 Sep 2024
Enhancing Text Annotation through Rationale-Driven Collaborative Few-Shot Prompting Jianfei Wu Xubin Wang Weijia Jia 22 1 0 15 Sep 2024
Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI Nicholas Pangakis Samuel Wolken 21 3 0 14 Sep 2024
Safeguarding Decentralized Social Media: LLM Agents for Automating Community Rule Compliance Lucio La Cava Andrea Tagarelli LLMAG 17 0 0 13 Sep 2024
Your Weak LLM is Secretly a Strong Teacher for Alignment Leitian Tao Yixuan Li 84 5 0 13 Sep 2024
HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Data Hossein Hajipour Lea Schönherr Thorsten Holz Mario Fritz AAML SyDa 21 0 0 10 Sep 2024
Column Vocabulary Association (CVA): semantic interpretation of dataless tables Margherita Martorana Xueli Pan Benno Kruit Tobias Kuhn Jacco van Ossenbruggen 20 1 0 06 Sep 2024
Content Moderation by LLM: From Accuracy to Legitimacy Tao Huang AILaw 32 3 0 05 Sep 2024
Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for Political Text Michael Burnham Kayla Kahn Ryan Yank Wang Rachel X. Peng 27 4 0 03 Sep 2024
It is Time to Develop an Auditing Framework to Promote Value Aware Chatbots Yanchen Wang Lisa Singh 24 1 0 03 Sep 2024
Towards Empathetic Conversational Recommender Systems Xiaoyu Zhang Ruobing Xie Yougang Lyu Xin Xin Pengjie Ren Mingfei Liang Bo Zhang Zhanhui Kang Maarten de Rijke Zhaochun Ren 23 6 0 30 Aug 2024
From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs Minxue Niu Mimansa Jaiswal Emily Mower Provost 31 5 0 30 Aug 2024
Can Unconfident LLM Annotations Be Used for Confident Conclusions? Kristina Gligorić Tijana Zrnic Cinoo Lee Emmanuel J. Candès Dan Jurafsky 63 4 0 27 Aug 2024
Utilizing Large Language Models for Named Entity Recognition in Traditional Chinese Medicine against COVID-19 Literature: Comparative Study Xu Tong N. Smirnova Sharmila Upadhyaya Ran Yu Jack H. Culbert Chao Sun Wolfgang Otto Philipp Mayr AI4MH 21 0 0 24 Aug 2024
Intelligent OPC Engineer Assistant for Semiconductor Manufacturing Guojin Chen Haoyu Yang Bei Yu Haoxing Ren 37 0 0 23 Aug 2024
Critique-out-Loud Reward Models Zachary Ankner Mansheej Paul Brandon Cui Jonathan D. Chang Prithviraj Ammanabrolu ALM LRM 25 25 0 21 Aug 2024
How Susceptible are LLMs to Influence in Prompts? Sotiris Anagnostidis Jannis Bulian LRM 25 16 0 17 Aug 2024
SEAL: Systematic Error Analysis for Value ALignment Manon Revel Matteo Cargnelutti Tyna Eloundou Greg Leppert 40 3 0 16 Aug 2024
LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning Jiajie Li Garrett C Skinner Gene Yang Brian R Quaranto Steven D. Schwaitzberg Peter C W Kim Jinjun Xiong 17 9 0 15 Aug 2024
What Color Scheme is More Effective in Assisting Readers to Locate Information in a Color-Coded Article? Ho Yin Ng Zeyu He Ting-Hao 'Kenneth' Huang 18 0 0 12 Aug 2024
Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Models Shachi H. Kumar Saurav Sahay Sahisnu Mazumder Eda Okur R. Manuvinakurike Nicole Beckage Hsuan Su Hung-yi Lee L. Nachman ELM 24 15 0 07 Aug 2024
Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction Albert Sawczyn Katsiaryna Viarenich Konrad Wojtasik Aleksandra Domogała Marcin Oleksy Maciej Piasecki Tomasz Kajdanowicz 18 0 0 05 Aug 2024
The Implications of Open Generative Models in Human-Centered Data Science Work: A Case Study with Fact-Checking Organizations Robert Wolfe Tanushree Mitra 33 1 0 04 Aug 2024
Automated Review Generation Method Based on Large Language Models Shican Wu Xiao Ma Dehui Luo Lulu Li Xiangcheng Shi ... Ran Luo Chunlei Pei Zhijian Zhao Zhi-Jian Zhao Jinlong Gong 69 0 0 30 Jul 2024
VolDoGer: LLM-assisted Datasets for Domain Generalization in Vision-Language Tasks Juhwan Choi Junehyoung Kwon Jungmin Yun Seunguk Yu Youngbin Kim 38 1 0 29 Jul 2024
Logic Distillation: Learning from Code Function by Function for Planning and Decision-making Dong Chen Shilin Zhang Fei Gao Yueting Zhuang Siliang Tang Qidong Liu Mingliang Xu LRM 20 0 0 28 Jul 2024
Towards a Multidimensional Evaluation Framework for Empathetic Conversational Systems Aravind Sesagiri Raamkumar Siyuan Brandon Loh 29 0 0 26 Jul 2024
LLMs left, right, and center: Assessing GPT's capabilities to label political bias from web domains Raphael Hernandes 16 4 0 19 Jul 2024
Dynamic Sentiment Analysis with Local Large Language Models using Majority Voting: A Study on Factors Affecting Restaurant Evaluation Junichiro Niimi 33 3 0 18 Jul 2024
ExploreGen: Large Language Models for Envisioning the Uses and Risks of AI Technologies Viviane Herdel S. Šćepanović Edyta Bogucka Daniele Quercia 18 7 0 17 Jul 2024
GPT Assisted Annotation of Rhetorical and Linguistic Features for Interpretable Propaganda Technique Detection in News Text Kyle Hamilton Luca Longo Bojan Bozic 26 1 0 16 Jul 2024
CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses Jing Yao Xiaoyuan Yi Xing Xie ELM ALM 31 7 0 15 Jul 2024
Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models Louis Abraham Charles Arnal Antoine Marie 50 1 0 15 Jul 2024