Perception-Oriented Latent Coding for High-Performance Compressed Domain Semantic Inference

In recent years, compressed domain semantic inference has primarily relied on learned image coding models optimized for mean squared error (MSE). However, MSE-oriented optimization tends to yield latent spaces with limited semantic richness, which hinders effective semantic inference in downstream tasks. Moreover, achieving high performance with these models often requires fine-tuning the entire vision model, which is computationally intensive, especially for large models. To address these problems, we introduce Perception-Oriented Latent Coding (POLC), an approach that enriches the semantic content of latent features for high-performance compressed domain semantic inference. With the semantically rich latent space, POLC requires only a plug-and-play adapter for fine-tuning, significantly reducing the parameter count compared to previous MSE-oriented methods. Experimental results demonstrate that POLC achieves rate-perception performance comparable to state-of-the-art generative image coding methods while markedly enhancing performance in vision tasks, with minimal fine-tuning overhead. Code is available atthis https URL.

@article{zhang2025_2507.01608,
  title={ Perception-Oriented Latent Coding for High-Performance Compressed Domain Semantic Inference },
  author={ Xu Zhang and Ming Lu and Yan Chen and Zhan Ma },
  journal={arXiv preprint arXiv:2507.01608},
  year={ 2025 }
}