ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

27 March 2023

Papers citing "ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks"

50 / 489 papers shown

Title
The Political Preferences of LLMs David Rozado 30 35 0 02 Feb 2024
LLM-based NLG Evaluation: Current Status and Challenges Mingqi Gao Xinyu Hu Jie Ruan Xiao Pu Xiaojun Wan ELM LM&MA 53 29 0 02 Feb 2024
Hierarchical Multi-Label Classification of Online Vaccine Concerns Chloe Qinyu Zhu Rickard Stureborg Bhuwan Dhingra 13 3 0 01 Feb 2024
LOCOST: State-Space Models for Long Document Abstractive Summarization Florian Le Bronnec Song Duong Mathieu Ravaut Alexandre Allauzen Nancy F. Chen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari 40 7 0 31 Jan 2024
Global-Liar: Factuality of LLMs over Time and Geographic Regions Shujaat Mirza Bruno Coelho Yuyuan Cui Christina Pöpper Damon McCoy HILM 11 4 0 31 Jan 2024
Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness Samaneh Shafee A. Bessani Pedro M. Ferreira 17 17 0 26 Jan 2024
APT-Pipe: A Prompt-Tuning Tool for Social Data Annotation using ChatGPT Yiming Zhu Zhizhuo Yin Gareth Tyson Ehsan-ul Haq Lik-Hang Lee Pan Hui ALM 33 6 0 24 Jan 2024
Red Teaming Visual Language Models Mukai Li Lei Li Yuwei Yin Masood Ahmed Zhenguang Liu Qi Liu VLM 25 30 0 23 Jan 2024
Cheap Learning: Maximising Performance of Language Models for Social Data Science Using Minimal Data Leonardo Castro-Gonzalez Yi-Ling Chung Hannak Rose Kirk John Francis Angus R. Williams Pica Johansson Jonathan Bright 37 1 0 22 Jan 2024
AI for social science and social science of AI: A Survey Ruoxi Xu Yingfei Sun Mengjie Ren Shiguang Guo Ruotong Pan Hongyu Lin Le Sun Xianpei Han 53 44 0 22 Jan 2024
CheX-GPT: Harnessing Large Language Models for Enhanced Chest X-ray Report Labeling Jawook Gu Hankyu Cho Jiho Kim Kihyun You Eun K. Hong Byungseok Roh 22 6 0 21 Jan 2024
Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortion Aly M. Kassem Sherif Saad AAML 23 1 0 21 Jan 2024
FAIR Enough: How Can We Develop and Assess a FAIR-Compliant Dataset for Large Language Models' Training? Shaina Raza Shardul Ghuge Chen Ding Elham Dolatabadi D. Pandya SyDa 15 9 0 19 Jan 2024
Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap Xingyu Wu Sheng-hao Wu Jibin Wu Liang Feng Kay Chen Tan ELM 34 57 0 18 Jan 2024
A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregation Jiyi Li 20 16 0 18 Jan 2024
Leveraging Large Language Models for NLG Evaluation: Advances and Challenges Zhen Li Xiaohan Xu Tao Shen Can Xu Jia-Chen Gu Yuxuan Lai Chongyang Tao Shuai Ma LM&MA ELM 26 9 0 13 Jan 2024
Visual Enumeration is Challenging for Large-scale Generative AI Alberto Testolin Kuinan Hou Marco Zorzi 20 2 0 09 Jan 2024
DepressionEmo: A novel dataset for multilabel classification of depression emotions Abu Bakar Siddiqur Rahman Hoang-Thang Ta Lotfollah Najjar A. Azadmanesh A. Gonul AI4MH 23 10 0 09 Jan 2024
MULTI-CASE: A Transformer-based Ethics-aware Multimodal Investigative Intelligence Framework M. T. Fischer Yannick Metz Lucas Joos Matthias Miller Daniel A. Keim 27 2 0 03 Jan 2024
Evaluation is all you need. Prompting Generative Large Language Models for Annotation Tasks in the Social Sciences. A Primer using Open Models Maximilian Weber Merle Reichardt 71 9 0 30 Dec 2023
EHR Interaction Between Patients and AI: NoteAid EHR Interaction Xiaocheng Zhang Zonghai Yao Hong-ye Yu LM&MA 26 2 0 29 Dec 2023
Understanding News Creation Intents: Frame, Dataset, and Method Zhengjia Wang Danding Wang Qiang Sheng Juan Cao Silong Su Yifan Sun Beizhe Hu Siyuan Ma 10 4 0 27 Dec 2023
A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators Chen Zhang L. F. D’Haro Yiming Chen Malu Zhang Haizhou Li ELM 11 28 0 24 Dec 2023
Exploring the Capabilities of ChatGPT in Ancient Chinese Translation and Person Name Recognition Shijing Si Siqing Zhou Le Tang Xiaoqing Cheng Yugui Zhang 18 0 0 23 Dec 2023
In Generative AI we Trust: Can Chatbots Effectively Verify Political Information? Elizaveta Kuznetsova M. Makhortykh Victoria Vziatysheva Martha Stolze Ani Baghumyan Aleksandra Urman 17 2 0 20 Dec 2023
ALMANACS: A Simulatability Benchmark for Language Model Explainability Edmund Mills Shiye Su Stuart J. Russell Scott Emmons 43 7 0 20 Dec 2023
VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering Chun-Mei Feng Yang Bai Tao Luo Zhen Li Salman Khan Wangmeng Zuo Xinxing Xu Rick Siow Mong Goh Yong-Jin Liu 27 5 0 19 Dec 2023
Student as an Inherent Denoiser of Noisy Teacher Jiachen Zhao 8 0 0 15 Dec 2023
Prompting Datasets: Data Discovery with Conversational Agents Johanna Walker Elisavet Koutsiana Joe Massey Gefion Thuermer Elena Simperl 18 2 0 15 Dec 2023
FG-MDM: Towards Zero-Shot Human Motion Generation via Fine-Grained Descriptions Xu Shi Wei Yao Chuanchen Luo Junran Peng Hongwen Zhang Yunlian Sun VGen 24 4 0 05 Dec 2023
How should the advent of large language models affect the practice of science? Marcel Binz Stephan Alaniz Adina Roskies B. Aczel Carl T. Bergstrom ... Emily M. Bender M. Marelli Matthew M. Botvinick Zeynep Akata Eric Schulz 18 8 0 05 Dec 2023
Analyzing the Impact of Fake News on the Anticipated Outcome of the 2024 Election Ahead of Time Shaina Raza Mizanur Rahman Shardul Ghuge 8 1 0 01 Dec 2023
Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding Sicong Leng Hang Zhang Guanzheng Chen Xin Li Shijian Lu Chunyan Miao Li Bing VLM MLLM 85 197 0 28 Nov 2023
SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification Difan Jiao Yilun Liu Zhenwei Tang Daniel Matter Jürgen Pfeffer Ashton Anderson 17 1 0 27 Nov 2023
Fully Authentic Visual Question Answering Dataset from Online Communities Chongyan Chen Mengchen Liu Noel Codella Yunsheng Li Lu Yuan Danna Gurari 27 5 0 27 Nov 2023
Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching Meng Chu Zhedong Zheng Wei Ji Tingyu Wang Tat-Seng Chua 18 9 0 21 Nov 2023
Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Tasks Yuxuan Lu Bingsheng Yao Shao Zhang Yun Wang Peng Zhang T. Lu Toby Jia-Jun Li Dakuo Wang ALM 34 18 0 16 Nov 2023
Large Language Models for Propaganda Span Annotation Maram Hasanain Fatema Ahmed Firoj Alam 19 19 0 16 Nov 2023
MAFALDA: A Benchmark and Comprehensive Study of Fallacy Detection and Classification Chadi Helwe Tom Calamai Pierre-Henri Paris Chloé Clavel Fabian M. Suchanek 10 1 0 16 Nov 2023
Large Language Models are Few-Shot Training Example Generators: A Case Study in Fallacy Recognition Tariq Alhindi Smaranda Muresan Preslav Nakov HILM LRM 38 4 0 16 Nov 2023
Reducing Privacy Risks in Online Self-Disclosures with Language Models Yao Dou Isadora Krsek Tarek Naous Anubha Kabra Sauvik Das Alan Ritter Wei-ping Xu 28 21 0 16 Nov 2023
Value FULCRA: Mapping Large Language Models to the Multidimensional Spectrum of Basic Human Values Jing Yao Xiaoyuan Yi Xiting Wang Yifan Gong Xing Xie 20 22 0 15 Nov 2023
Explore Spurious Correlations at the Concept Level in Language Models for Text Classification Yuhang Zhou Paiheng Xu Xiaoyu Liu Bang An Wei Ai Furong Huang LRM 71 20 0 15 Nov 2023
CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generation Weixiang Yan Haitian Liu Yunkun Wang Yunzhe Li Qian Chen ... Tingyu Lin Weishan Zhao Li Zhu Hari Sundaram Shuiguang Deng ELM LRM 21 34 0 14 Nov 2023
Semi-automatic Data Enhancement for Document-Level Relation Extraction with Distant Supervision from Large Language Models Junpeng Li Zixia Jia Zilong Zheng 22 11 0 13 Nov 2023
Large Language Models are Zero Shot Hypothesis Proposers Biqing Qi Kaiyan Zhang Haoxiang Li Kai Tian Sihang Zeng Zhang-Ren Chen Bowen Zhou 24 27 0 10 Nov 2023
Interpreting Pretrained Language Models via Concept Bottlenecks Zhen Tan Lu Cheng Song Wang Yuan Bo Jundong Li Huan Liu LRM 22 20 0 08 Nov 2023
Do LLMs exhibit human-like response biases? A case study in survey design Lindia Tjuatja Valerie Chen Sherry Tongshuang Wu Ameet Talwalkar Graham Neubig 19 77 0 07 Nov 2023
Context Unlocks Emotions: Text-based Emotion Classification Dataset Auditing with Large Language Models Daniel Yang Aditya Kommineni Mohammad Alshehri Nilamadhab Mohanty Vedant Modi Jonathan Gratch Shrikanth Narayanan 13 3 0 06 Nov 2023
Incorporating Worker Perspectives into MTurk Annotation Practices for NLP Olivia Huang Eve Fleisig Dan Klein 38 15 0 06 Nov 2023