Similarity-Based Domain Adaptation with LLMs

Unsupervised domain adaptation leverages abundant labeled data from various source domains to generalize onto unlabeled target data. Prior research has primarily focused on learning domain-invariant features across the source and target domains. However, these methods often require training a model using source domain data, which is time-consuming and can limit model usage for applications with different source data. This paper introduces a simple framework that utilizes the impressive generalization capabilities of Large Language Models (LLMs) for target data annotation without the need of source model training, followed by a novel similarity-based knowledge distillation loss. Our extensive experiments on cross-domain text classification reveal that our framework achieves impressive performance, specifically, 2.44\% accuracy improvement when compared to the SOTA method.

@article{he2025_2503.05281,
  title={ Similarity-Based Domain Adaptation with LLMs },
  author={ Jie He and Wendi Zhou and Xiang Lorraine Li and Jeff Z. Pan },
  journal={arXiv preprint arXiv:2503.05281},
  year={ 2025 }
}