Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering

Recent advancements in long chain-of-thoughts(long CoTs) have significantly improved the reasoning capabilities of large language models(LLMs). Existing work finds that the capability of long CoT reasoning can be efficiently elicited by tuning on only a few examples and can easily transfer to other tasks. This motivates us to investigate whether long CoT reasoning is a general capability for LLMs. In this work, we conduct an empirical analysis for this question from the perspective of representation. We find that LLMs do encode long CoT reasoning as a general capability, with a clear distinction from vanilla CoTs. Furthermore, domain-specific representations are also required for the effective transfer of long CoT reasoning. Inspired by these findings, we propose GLoRE, a novel representation engineering method to unleash the general long CoT reasoning capabilities of LLMs. Extensive experiments demonstrate the effectiveness and efficiency of GLoRE in both in-domain and cross-domain scenarios.

@article{tang2025_2503.11314,
  title={ Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering },
  author={ Xinyu Tang and Xiaolei Wang and Zhihao Lv and Yingqian Min and Wayne Xin Zhao and Binbin Hu and Ziqi Liu and Zhiqiang Zhang },
  journal={arXiv preprint arXiv:2503.11314},
  year={ 2025 }
}