Understanding Classifier-Free Guidance: High-Dimensional Theory and Non-Linear Generalizations

Recent studies have raised concerns about the effectiveness of Classifier-Free Guidance (CFG), indicating that in low-dimensional settings, it can lead to overshooting the target distribution and reducing sample diversity. In this work, we demonstrate that in infinite and sufficiently high-dimensional contexts CFG effectively reproduces the target distribution, revealing a blessing-of-dimensionality result. Additionally, we explore finite-dimensional effects, precisely characterizing overshoot and variance reduction. Based on our analysis, we introduce non-linear generalizations of CFG. Through numerical simulations on Gaussian mixtures and experiments on class-conditional and text-to-image diffusion models, we validate our analysis and show that our non-linear CFG offers improved flexibility and generation quality without additional computation cost.

@article{pavasovic2025_2502.07849,
  title={ Understanding Classifier-Free Guidance: High-Dimensional Theory and Non-Linear Generalizations },
  author={ Krunoslav Lehman Pavasovic and Jakob Verbeek and Giulio Biroli and Marc Mezard },
  journal={arXiv preprint arXiv:2502.07849},
  year={ 2025 }
}