Building Vision Models upon Heat Conduction

Visual representation models leveraging attention mechanisms are challenged by significant computational overhead, particularly when pursuing large receptive fields. In this study, we aim to mitigate this challenge by introducing the Heat Conduction Operator (HCO) built upon the physical heat conduction principle. HCO conceptualizes image patches as heat sources and models their correlations through adaptive thermal energy diffusion, enabling robust visual representations. HCO enjoys a computational complexity of O(N^1.5), as it can be implemented using discrete cosine transformation (DCT) operations. HCO is plug-and-play, combining with deep learning backbones produces visual representation models (termed vHeat) with global receptive fields. Experiments across vision tasks demonstrate that, beyond the stronger performance, vHeat achieves up to a 3x throughput, 80% less GPU memory allocation, and 35% fewer computational FLOPs compared to the Swin-Transformer. Code is available atthis https URL.

@article{wang2025_2405.16555,
  title={ Building Vision Models upon Heat Conduction },
  author={ Zhaozhi Wang and Yue Liu and Yunjie Tian and Yunfan Liu and Yaowei Wang and Qixiang Ye },
  journal={arXiv preprint arXiv:2405.16555},
  year={ 2025 }
}