Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters

20 December 2022

Luke Zettlemoyer

Huan Sun

LRM

ReLM

ArXiv PDF HTML

Papers citing "Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters"

39 / 39 papers shown

Title
Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models Zhanke Zhou Zhaocheng Zhu Xuan Li Mikhail Galkin Xiao Feng Sanmi Koyejo Jian Tang Bo Han LRM 42 0 0 28 Mar 2025
How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach Ayeong Lee Ethan Che Tianyi Peng LRM 42 10 0 03 Mar 2025
BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning Ercong Nie Bo Shao Zifeng Ding Mingyang Wang Helmut Schmid Hinrich Schütze KELM 129 7 0 21 Feb 2025
PICLe: Pseudo-Annotations for In-Context Learning in Low-Resource Named Entity Detection Sepideh Mamooler Syrielle Montariol Alexander Mathis Antoine Bosselut 78 1 0 16 Dec 2024
PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation Qiyao Xue Xiangyu Yin Boyuan Yang Wei Gao DiffM VGen 75 9 0 30 Nov 2024
Uncovering Autoregressive LLM Knowledge of Thematic Fit in Event Representation Safeyah Khaled Alshemali Daniel Bauer Yuval Marton BDL 38 0 0 19 Oct 2024
Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad VLM 16 2 0 16 Oct 2024
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning Zayne Sprague Fangcong Yin Juan Diego Rodriguez Dongwei Jiang Manya Wadhwa Prasann Singhal Xinyu Zhao Xi Ye Kyle Mahowald Greg Durrett ReLM LRM 111 79 0 18 Sep 2024
Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Direction Melkamu Mersha Khang Lam Joseph Wood Ali AlShami Jugal Kalita XAI AI4TS 60 26 0 30 Aug 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 74 11 0 16 Aug 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 59 0 0 31 Jul 2024
DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion Huiguo He Huan Yang Zixi Tuo Yuan Zhou Qiuyue Wang Yuhang Zhang Zeyu Liu Wenhao Huang Hongyang Chao Jian Yin DiffM VGen 52 11 0 17 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 70 18 0 02 Jul 2024
Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step Zezhong Wang Xingshan Zeng Weiwen Liu Yufei Wang Liangyou Li Yasheng Wang Lifeng Shang Xin Jiang Qun Liu Kam-Fai Wong LRM 49 3 0 23 Jun 2024
People will agree what I think: Investigating LLM's False Consensus Effect Junhyuk Choi Yeseon Hong Bugeun Kim 41 0 0 16 Jun 2024
Break the Chain: Large Language Models Can be Shortcut Reasoners Mengru Ding Hanmeng Liu Zhizhang Fu Jian Song Wenbo Xie Yue Zhang KELM LRM 27 7 0 04 Jun 2024
ACCORD: Closing the Commonsense Measurability Gap François Roewer-Després Jinyue Feng Zining Zhu Frank Rudzicz LRM 32 0 0 04 Jun 2024
DaVinci at SemEval-2024 Task 9: Few-shot prompting GPT-3.5 for Unconventional Reasoning Suyash Vardhan Mathur Akshett Rai Jindal Manish Shrivastava LRM 26 1 0 19 May 2024
Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs Yu Xia Rui Wang Xu Liu Mingyan Li Tong Yu Xiang Chen Julian McAuley Shuai Li LRM 41 16 0 24 Apr 2024
Learning to Maximize Mutual Information for Chain-of-Thought Distillation Xin Chen Hanxian Huang Yanjun Gao Yi Wang Jishen Zhao Ke Ding 35 11 0 05 Mar 2024
GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languages Spencer Rarrick Ranjita Naik Sundar Poudel Vishal Chowdhary 27 1 0 22 Feb 2024
RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses Sahil Girhepuje Siva Sankar Sajeev Purvam Jain Arya Sikder Adithya Rama Varma Ryan George Akshay Govind Srinivasan Mahendra Kurup Ashmit Sinha Sudip Mondal RALM 22 0 0 28 Jan 2024
Gradable ChatGPT Translation Evaluation Hui Jiao Bei Peng Lu Zong Xiaojun Zhang Xinwei Li 28 1 0 18 Jan 2024
FinanceBench: A New Benchmark for Financial Question Answering Pranab Islam Anand Kannappan Douwe Kiela Rebecca Qian Nino Scherrer Bertie Vidgen RALM 13 71 0 20 Nov 2023
R-Tuning: Instructing Large Language Models to Say `I Don't Know' Hanning Zhang Shizhe Diao Yong Lin Yi Ren Fung Qing Lian Xingyao Wang Yangyi Chen Heng Ji Tong Zhang UQLM 27 36 0 16 Nov 2023
Self-Contradictory Reasoning Evaluation and Detection Ziyi Liu Isabelle G. Lee Yongkang Du Soumya Sanyal Jieyu Zhao LRM 28 2 0 16 Nov 2023
Towards A Unified View of Answer Calibration for Multi-Step Reasoning Shumin Deng Ningyu Zhang Nay Oo Bryan Hooi LRM 25 1 0 15 Nov 2023
Empirical Study of Zero-Shot NER with ChatGPT Tingyu Xie Qi Li Jian Zhang Yan Zhang Zuozhu Liu Hongwei Wang LRM ReLM 19 62 0 16 Oct 2023
PromptNER: Prompting For Named Entity Recognition D. Ashok Zachary Chase Lipton 18 35 0 24 May 2023
Solving NLP Problems through Human-System Collaboration: A Discussion-based Approach Masahiro Kaneko Graham Neubig Naoaki Okazaki 22 6 0 19 May 2023
How Many Demonstrations Do You Need for In-context Learning? Jiuhai Chen Lichang Chen Chen Zhu Tianyi Zhou LRM 16 39 0 14 Mar 2023
Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following Seonghyeon Ye Hyeonbin Hwang Sohee Yang Hyeongu Yun Yireun Kim Minjoon Seo LRM 19 34 0 28 Feb 2023
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought Abulhair Saparov He He ELM LRM ReLM 116 270 0 03 Oct 2022
Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts Joel Jang Seonghyeon Ye Minjoon Seo ELM LRM 87 64 0 26 Sep 2022
Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango Aman Madaan Amir Yazdanbakhsh LRM 136 115 0 16 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022