Learning to Maximize Mutual Information for Chain-of-Thought
Distillation

Learning to Maximize Mutual Information for Chain-of-Thought Distillation

5 March 2024

Papers citing "Learning to Maximize Mutual Information for Chain-of-Thought Distillation"

13 / 13 papers shown

Title
Kongzi: A Historical Large Language Model with Fact Enhancement Jiashu Yang Ningning Wang Yian Zhao Chaoran Feng Junjia Du Hao Pang Zhirui Fang Xuxin Cheng HILM ALM LRM 35 0 0 13 Apr 2025
Predicting Through Generation: Why Generation Is Better for Prediction Md. Kowsher Nusrat Jahan Prottasha Prakash Bhat Chun-Nam Yu Mojtaba Soltanalian Ivan Garibay O. Garibay Chen Chen Niloofar Yousefi AI4TS 55 0 0 25 Feb 2025
$The Two-Hop Curse: LLMs trained on A$\rightarrow$B, B$\rightarrow$C fail to learn A$\rightarrow$C$ The Two-Hop Curse: LLMs trained on A $\rightarrow$ B, B $\rightarrow$ C fail to learn A $\rightarrow$ C Mikita Balesni Tomek Korbak Owain Evans ReLM LRM 72 0 0 25 Nov 2024
Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data Anup Shirgaonkar Nikhil Pandey Nazmiye Ceren Abay Tolga Aktas Vijay Aski ALM SyDa 19 0 0 24 Oct 2024
Data-Centric Human Preference Optimization with Rationales H. Just Ming Jin Anit Kumar Sahu Huy Phan Ruoxi Jia 22 2 0 19 Jul 2024
Distilling System 2 into System 1 Ping Yu Jing Xu Jason Weston Ilia Kulikov OffRL LRM 32 55 0 08 Jul 2024
QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models Wei Wang Zhaowei Li Qi Xu Yiqing Cai Hang Song Qi Qi Ran Zhou Zhida Huang Tao Wang Li Xiao ALM 27 0 0 14 May 2024
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 198 283 0 03 May 2023
SCOTT: Self-Consistent Chain-of-Thought Distillation Jamie Yap Zhengyang Wang Zheng Li K. Lynch Bing Yin Xiang Ren LRM 57 91 0 03 May 2023
Hard Gate Knowledge Distillation -- Leverage Calibration for Robust and Reliable Language Model Dongkyu Lee Zhiliang Tian Ying Zhao Ka Chun Cheung N. Zhang 20 3 0 22 Oct 2022
Cross-Task Knowledge Distillation in Multi-Task Recommendation Chenxiao Yang Junwei Pan Xiaofeng Gao Tingyu Jiang Dapeng Liu Guihai Chen 21 44 0 20 Feb 2022
Distilling Knowledge via Knowledge Review Pengguang Chen Shu-Lin Liu Hengshuang Zhao Jiaya Jia 144 308 0 19 Apr 2021
e-SNLI: Natural Language Inference with Natural Language Explanations Oana-Maria Camburu Tim Rocktaschel Thomas Lukasiewicz Phil Blunsom LRM 249 618 0 04 Dec 2018