ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

27 March 2023

Papers citing "ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks"

50 / 489 papers shown

Title
Large Language Models for Integrating Social Determinant of Health Data: A Case Study on Heart Failure 30-Day Readmission Prediction Chase M. Fensore Rodrigo Carrillo-Larco Shivani Patel Alanna A. Morris Joyce C. Ho 27 0 0 12 Jul 2024
Good Intentions, Risky Inventions: A Method for Assessing the Risks and Benefits of AI in Mobile and Wearable Uses Marios Constantinides Edyta Bogucka S. Šćepanović Daniele Quercia 21 6 0 12 Jul 2024
Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation Biqing Qi Kaiyan Zhang Kai Tian Haoxiang Li Zhang-Ren Chen Sihang Zeng Ermo Hua Hu Jinfang Bowen Zhou LM&MA 38 7 0 12 Jul 2024
ChatGPT Doesn't Trust Chargers Fans: Guardrail Sensitivity in Context Victoria R. Li Yida Chen Naomi Saphra 35 3 0 09 Jul 2024
Exploring the Capability of ChatGPT to Reproduce Human Labels for Social Computing Tasks (Extended Version) Yiming Zhu Peixian Zhang Ehsan-ul Haq Pan Hui Gareth Tyson ALM AI4MH 26 0 0 08 Jul 2024
ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models Yuzhe Gu Ziwei Ji Wenwei Zhang Chengqi Lyu Dahua Lin Kai Chen HILM 34 5 0 05 Jul 2024
Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms Joshua Ashkinaze Ruijia Guan Laura Kurek Eytan Adar Ceren Budak Eric Gilbert KELM 16 5 0 04 Jul 2024
Towards Automating Text Annotation: A Case Study on Semantic Proximity Annotation using GPT-4 Sachin Yadav Tejaswi Choppa Dominik Schlechtweg VLM 29 3 0 04 Jul 2024
Entity-Level Sentiment: More than the Sum of Its Parts Egil Rønningstad Roman Klinger Erik Velldal Lilja Øvrelid 16 0 0 04 Jul 2024
Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I Harrie Oosterhuis R. Jagerman Zhen Qin Xuanhui Wang Michael Bendersky 46 4 0 02 Jul 2024
NLPGuard: A Framework for Mitigating the Use of Protected Attributes by NLP Classifiers Salvatore Greco Ke Zhou L. Capra Tania Cerquitelli Daniele Quercia 29 2 0 01 Jul 2024
LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives Luísa Shimabucoro Sebastian Ruder Julia Kreutzer Marzieh Fadaee Sara Hooker SyDa 21 4 0 01 Jul 2024
IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script Generation Senyu Han Lu Chen Li-Min Lin Zhengshan Xu Kai Yu LM&Ro 40 13 0 01 Jul 2024
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks A. Bavaresco Raffaella Bernardi Leonardo Bertolazzi Desmond Elliott Raquel Fernández ... David Schlangen Alessandro Suglia Aditya K Surikuchi Ece Takmaz A. Testoni ALM ELM 36 62 0 26 Jun 2024
Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labels Nicholas Pangakis Samuel Wolken 27 15 0 25 Jun 2024
The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators Tzu-Heng Huang Catherine Cao Vaishnavi Bhargava Frederic Sala 31 3 0 25 Jun 2024
$USDC: A Dataset of $\underline{U}$ser $\underline{S}$tance and $\underline{D}$ogmatism in Long $\underline{C}$onversations$ USDC: A Dataset of $\underline{U}$ ser $\underline{S}$ tance and $\underline{D}$ ogmatism in Long $\underline{C}$ onversations Mounika Marreddy S. Oota Venkata Charan Chinni Manish Gupta Lucie Flek 46 0 0 24 Jun 2024
Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation Katherine M. Collins Najoung Kim Yonatan Bitton Verena Rieser Shayegan Omidshafiei ... Gang Li Adrian Weller Junfeng He Deepak Ramachandran Krishnamurthy Dvijotham EGVM 41 3 0 24 Jun 2024
AnnotatedTables: A Large Tabular Dataset with Language Model Annotations Yaojie Hu Ilias Fountalis Jin Tian N. Vasiloglou LMTD 27 3 0 24 Jun 2024
M2Lingual: Enhancing Multilingual, Multi-Turn Instruction Alignment in Large Language Models Rishabh Maheshwary Vikas Yadav Hoang Nguyen Khyati Mahajan Sathwik Tejaswi Madhusudhan 40 3 0 24 Jun 2024
Assessing Good, Bad and Ugly Arguments Generated by ChatGPT: a New Dataset, its Methodology and Associated Tasks Victor Hugo Nascimento Rocha I. Silveira Paulo Pirozelli Denis Deratani Mauá Fabio Gagliardi Cozman 18 0 0 21 Jun 2024
VGA: Vision GUI Assistant -- Minimizing Hallucinations through Image-Centric Fine-Tuning Ziyang Meng Yu Dai Zezheng Gong Shaoxiong Guo Minglong Tang Tongquan Wei VLM 22 3 0 20 Jun 2024
SPL: A Socratic Playground for Learning Powered by Large Language Model Liang Zhang Jionghao Lin Ziyi Kuang Sheng Xu Mohammed Yeasin Xiangen Hu LLMAG 53 5 0 20 Jun 2024
Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation Hamidreza Rouzegar Masoud Makrehchi 28 9 0 17 Jun 2024
Prompt Design Matters for Computational Social Science Tasks but in Unpredictable Ways Shubham Atreja Joshua Ashkinaze Lingyao Li Julia Mendelsohn Libby Hemphill 39 12 0 17 Jun 2024
Investigating Annotator Bias in Large Language Models for Hate Speech Detection Amit Das Zheng Zhang Fatemeh Jamshidi Vinija Jain Aman Chadha Nilanjana Raychawdhary Mary J. Sandage Lauramarie Pope Gerry V. Dozier Cheryl Seals 14 2 0 17 Jun 2024
Distilling Opinions at Scale: Incremental Opinion Summarization using XL-OPSUMM Sri Raghava Muddu Rupasai Rangaraju Tejpalsingh Siledar Swaroop Nath Pushpak Bhattacharyya ... Suman Banerjee Amey Patil M. Chelliah Sudhanshu Singh Nikesh Garera ELM LRM 42 1 0 16 Jun 2024
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey Lin Long Rui Wang Ruixuan Xiao Junbo Zhao Xiao Ding Gang Chen Haobo Wang SyDa 51 90 0 14 Jun 2024
Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification Martin Juan José Bucher Marco Martini ALM AI4MH 29 23 0 12 Jun 2024
Designing a Dashboard for Transparency and Control of Conversational AI Yida Chen Aoyu Wu Trevor DePodesta Catherine Yeh Kenneth Li ... Jan Riecke Shivam Raval Olivia Seow Martin Wattenberg Fernanda Viégas 22 16 0 12 Jun 2024
Advancing Annotation of Stance in Social Media Posts: A Comparative Analysis of Large Language Models and Crowd Sourcing Mao Li Frederick Conrad 29 1 0 11 Jun 2024
DARA: Decomposition-Alignment-Reasoning Autonomous Language Agent for Question Answering over Knowledge Graphs Haishuo Fang Xiaodan Zhu Iryna Gurevych AI4CE 25 1 0 11 Jun 2024
Planning Like Human: A Dual-process Framework for Dialogue Planning Tao He Lizi Liao Yixin Cao Yuanxing Liu Ming Liu Zerui Chen Bing Qin 49 15 0 08 Jun 2024
Large Language Model-guided Document Selection Xiang Kong Tom Gunter Ruoming Pang 33 4 0 07 Jun 2024
Prototypical Reward Network for Data-Efficient RLHF Jinghan Zhang Xiting Wang Yiqiao Jin Changyu Chen Xinhao Zhang Kunpeng Liu ALM 26 18 0 06 Jun 2024
Large Language Models as Evaluators for Recommendation Explanations Xiaoyu Zhang Yishan Li Jiayin Wang Bowen Sun Weizhi Ma Peijie Sun Min Zhang LRM ELM 35 12 0 05 Jun 2024
Which Side Are You On? A Multi-task Dataset for End-to-End Argument Summarisation and Evaluation Hao Li Yuping Wu Viktor Schlegel R. Batista-Navarro Tharindu Madusanka ... Jiayan Zeng Xiaochi Wang Xinran He Yizhi Li Goran Nenadic 26 6 0 05 Jun 2024
Aligning Large Language Models via Fine-grained Supervision Dehong Xu Liang Qiu Minseok Kim Faisal Ladhak Jaeyoung Do 30 2 0 04 Jun 2024
Story Generation from Visual Inputs: Techniques, Related Tasks, and Challenges Daniel A. P. Oliveira Eugénio Ribeiro David Martins de Matos VGen 20 2 0 04 Jun 2024
Unsupervised Distractor Generation via Large Language Model Distilling and Counterfactual Contrastive Decoding Fanyi Qu Hao Sun Yunfang Wu 34 8 0 03 Jun 2024
Annotation Guidelines-Based Knowledge Augmentation: Towards Enhancing Large Language Models for Educational Text Classification Shiqi Liu Sannyuya Liu Lele Sha Zijie Zeng D. Gašević Zhi Liu 44 0 0 03 Jun 2024
Improving Text Generation on Images with Synthetic Captions Jun Young Koh Sang Hyun Park Joy Song DiffM 41 2 0 01 Jun 2024
Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings Robert Wolfe Isaac Slaughter Bin Han Bingbing Wen Yiwei Yang ... Bernease Herman E. Brown Zening Qu Nicholas Weber Bill Howe 18 4 0 27 May 2024
Automatically Generating Numerous Context-Driven SFT Data for LLMs across Diverse Granularity Shanghaoran Quan 27 3 0 26 May 2024
Picturing Ambiguity: A Visual Twist on the Winograd Schema Challenge Brendan Park Madeline Janecek Naser Ezzati-Jivan Yifeng Li Ali Emami 24 0 0 25 May 2024
Organic Data-Driven Approach for Turkish Grammatical Error Correction and LLMs Asim Ersoy O. T. Yildiz 28 0 0 24 May 2024
Unveiling the Achilles' Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models Yiming Chen Chen Zhang Danqing Luo L. F. D’Haro R. Tan Haizhou Li AAML ELM 26 2 0 23 May 2024
On the Brittle Foundations of ReAct Prompting for Agentic Large Language Models Mudit Verma Siddhant Bhambri Subbarao Kambhampati ReLM LRM 43 7 0 22 May 2024
Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planning Yuanhao Yue Chengyu Wang Jun Huang Peng Wang ALM 17 4 0 22 May 2024
Selective Annotation via Data Allocation: These Data Should Be Triaged to Experts for Annotation Rather Than the Model Chen Huang Yang Deng Wenqiang Lei Jiancheng Lv Ido Dagan 36 4 0 20 May 2024