ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks

27 March 2023

Papers citing "ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks"

50 / 489 papers shown

Title
FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models Yuxin Jiang Yufei Wang Xingshan Zeng Wanjun Zhong Liangyou Li Fei Mi Lifeng Shang Xin Jiang Qun Liu Wei Wang ALM 13 25 0 31 Oct 2023
Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization Prakamya Mishra Zonghai Yao Shuwei Chen Beining Wang Rohan Mittal Hong-ye Yu KELM ALM HILM 20 7 0 30 Oct 2023
MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks Allen Nie Yuhui Zhang Atharva Amdekar Chris Piech Tatsunori Hashimoto Tobias Gerstenberg 16 33 0 30 Oct 2023
EHRTutor: Enhancing Patient Understanding of Discharge Instructions Zihao Zhang Zonghai Yao Huixue Zhou Feiyun Ouyang Hong-ye Yu LM&MA AI4Ed 10 4 0 30 Oct 2023
Modeling Legal Reasoning: LM Annotation at the Edge of Human Agreement R. Thalken Edward H. Stiglitz David M. Mimno Matthew Wilkens AILaw ELM 21 16 0 27 Oct 2023
Can LLMs Grade Short-Answer Reading Comprehension Questions : An Empirical Study with a Novel Dataset Owen Henkel Libby Hills Bill Roberts Joshua A. McGrane AI4Ed 13 1 0 26 Oct 2023
Prevalence and prevention of large language model use in crowd work V. Veselovsky Manoel Horta Ribeiro Philip Cozzolino Andrew Gordon David Rothschild Robert West 11 0 0 24 Oct 2023
CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation Minzhi Li Taiwei Shi Caleb Ziems Min-Yen Kan Nancy F. Chen Zhengyuan Liu Diyi Yang 16 66 0 24 Oct 2023
LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysis Shih-Chieh Dai Aiping Xiong Lun-Wei Ku 13 61 0 23 Oct 2023
Learning to Correct Noisy Labels for Fine-Grained Entity Typing via Co-Prediction Prompt Tuning Minghao Tang Yongquan He Yongxiu Xu Hongbo Xu Wenyuan Zhang Yang Lin NoLa 23 0 0 23 Oct 2023
Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks Andrea Sottana Bin Liang Kai Zou Zheng Yuan ALM ELM LM&MA 25 53 0 20 Oct 2023
Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations Jihyoung Jang Minseong Boo Hyounghun Kim 11 28 0 20 Oct 2023
Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting Chenkai Sun Jinning Li Yi Ren Fung Hou Pong Chan Tarek F. Abdelzaher Chengxiang Zhai Heng Ji 18 15 0 20 Oct 2023
An Emulator for Fine-Tuning Large Language Models using Small Language Models Eric Mitchell Rafael Rafailov Archit Sharma Chelsea Finn Christopher D. Manning ALM 27 51 0 19 Oct 2023
Attack Prompt Generation for Red Teaming and Defending Large Language Models Boyi Deng Wenjie Wang Fuli Feng Yang Deng Qifan Wang Xiangnan He AAML 12 48 0 19 Oct 2023
Automated Evaluation of Personalized Text Generation using Large Language Models Yaqing Wang Jiepu Jiang Mingyang Zhang Cheng-rong Li Yi Liang Qiaozhu Mei Michael Bendersky 11 6 0 17 Oct 2023
Utilising a Large Language Model to Annotate Subject Metadata: A Case Study in an Australian National Research Data Catalogue Shiwei Zhang Ming-Lun Wu Xiuzhen Zhang 30 3 0 17 Oct 2023
Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformation Eun Cheol Choi Emilio Ferrara 21 17 0 13 Oct 2023
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models Seungone Kim Jamin Shin Yejin Cho Joel Jang Shayne Longpre ... Sangdoo Yun Seongjin Shin Sungdong Kim James Thorne Minjoon Seo ALM LM&MA ELM 11 202 0 12 Oct 2023
Multiclass Classification of Policy Documents with Large Language Models Erkan Gunes Christoffer Florczak 13 1 0 12 Oct 2023
Training Generative Question-Answering on Synthetic Data Obtained from an Instruct-tuned Model Kosuke Takahashi Takahiro Omi Kosuke Arima Tatsuya Ishigaki 23 0 0 12 Oct 2023
LLM4Vis: Explainable Visualization Recommendation using ChatGPT Lei Wang Songheng Zhang Yun Wang Ee-Peng Lim Yong Wang LRM 28 35 0 11 Oct 2023
SALMON: Self-Alignment with Instructable Reward Models Zhiqing Sun Yikang Shen Hongxin Zhang Qinhong Zhou Zhenfang Chen David D. Cox Yiming Yang Chuang Gan ALM SyDa 16 35 0 09 Oct 2023
XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners Yun Luo Zhen Yang Fandong Meng Yingjie Li Fang Guo Qinglin Qi Jie Zhou Yue Zhang 16 1 0 09 Oct 2023
DORIS-MAE: Scientific Document Retrieval using Multi-level Aspect-based Queries Jianyou Wang Kaicheng Wang Xiaoyue Wang Prudhviraj Naidu Leon Bergen R. Paturi 37 10 0 07 Oct 2023
Label-free Node Classification on Graphs with Large Language Models (LLMS) Zhikai Chen Haitao Mao Hongzhi Wen Haoyu Han Wei-dong Jin Haiyang Zhang Hui Liu Jiliang Tang 23 70 0 07 Oct 2023
Ada-Instruct: Adapting Instruction Generators for Complex Reasoning Wanyun Cui Qianle Wang LRM 34 7 0 06 Oct 2023
The Anatomy of Deception: Technical and Human Perspectives on a Large-scale Phishing Campaign Anargyros Chrysanthou Yorgos Pantis Constantinos Patsakis 11 1 0 05 Oct 2023
Assessing Large Language Models on Climate Information Jannis Bulian Mike S. Schäfer Afra Amini Heidi Lam Massimiliano Ciaramita ... Michelle Chen Huebscher Christian Buck Niels G. Mede Markus Leippold Nadine Strauss ELM 12 18 0 04 Oct 2023
CITING: Large Language Models Create Curriculum for Instruction Tuning Tao Feng Zifeng Wang Jimeng Sun ALM 17 14 0 04 Oct 2023
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving Long Chen Oleg Sinavski Jan Hünermann Alice Karnsund Andrew James Willmott Danny Birch Daniel Maund Jamie Shotton MLLM 6 178 0 03 Oct 2023
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models Zekun Wang Zhongyuan Peng Haoran Que Jiaheng Liu Wangchunshu Zhou ... Wanli Ouyang Ke Xu Wenhu Chen Jie Fu Junran Peng LLMAG 28 80 0 01 Oct 2023
A Large Language Model Approach to Educational Survey Feedback Analysis Michael J. Parker Caitlin Anderson Claire Stone YeaRim Oh ELM LM&MA AI4Ed 14 10 0 29 Sep 2023
Using Large Language Models for Qualitative Analysis can Introduce Serious Bias Julian Ashwin Aditya Chhabra Vijayendra Rao 18 20 0 29 Sep 2023
Watch Your Language: Investigating Content Moderation with Large Language Models Deepak Kumar Y. AbuHashem Zakir Durumeric AI4MH 21 15 0 25 Sep 2023
Large language models can accurately predict searcher preferences Paul Thomas S. Spielman Nick Craswell Bhaskar Mitra ALM LRM 17 131 0 19 Sep 2023
Fabricator: An Open Source Toolkit for Generating Labeled Training Data with Teacher LLMs Jonas Golde Patrick Haller Felix Hamborg Julian Risch A. Akbik 38 8 0 18 Sep 2023
Resolving Legalese: A Multilingual Exploration of Negation Scope Resolution in Legal Documents Ramona Christen Anastassia Shaitarova Matthias Sturmer Joel Niklaus AILaw ELM 28 3 0 15 Sep 2023
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation? Rishav Hada Varun Gumma Adrian de Wynter Harshita Diddee Mohamed Ahmed Monojit Choudhury Kalika Bali Sunayana Sitaram ALM LM&MA ELM 20 61 0 14 Sep 2023
Generative AI Stefan Feuerriegel Jochen Hartmann Christian Janiesch Patrick Zschech 21 534 0 13 Sep 2023
Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweets Ramya Tekumalla Juan M. Banda 19 6 0 12 Sep 2023
Leveraging Large Language Models for Automated Dialogue Analysis Sarah E. Finch Ellie S. Paek Jinho D. Choi LLMAG 29 18 0 12 Sep 2023
From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting Griffin Adams Alexander R. Fabbri Faisal Ladhak Eric Lehman Noémie Elhadad 19 50 0 08 Sep 2023
Evolving linguistic divergence on polarizing social media Andres Karjus Christine Cuskley 8 10 0 04 Sep 2023
Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties Taylor Sorensen Liwei Jiang Jena D. Hwang Sydney Levine Valentina Pyatkin ... Kavel Rao Chandra Bhagavatula Maarten Sap J. Tasioulas Yejin Choi SLR 11 49 0 02 Sep 2023
Simple LLM Prompting is State-of-the-Art for Robust and Multilingual Dialogue Evaluation J. Mendoncca Patrícia Pereira Helena Moniz Joao Paulo Carvalho A. Lavie Isabel Trancoso 12 19 0 31 Aug 2023
Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models Hritik Bansal John Dang Aditya Grover ALM 20 20 0 30 Aug 2023
Prompting a Large Language Model to Generate Diverse Motivational Messages: A Comparison with Human-Written Messages Samuel Rhys Cox Ashraf Abdul Wei Tsang Ooi ALM 31 12 0 25 Aug 2023
Text Style Transfer Evaluation Using Large Language Models Phil Ostheimer M. Nagda Marius Kloft Sophie Fellenz 17 9 0 25 Aug 2023
GPTEval: A Survey on Assessments of ChatGPT and GPT-4 Rui Mao Guanyi Chen Xulang Zhang Frank Guerin Erik Cambria ELM LM&MA 28 91 0 24 Aug 2023