References Indeed Matter? Reference-Free Preference Optimization for Conversational Query Reformulation

Conversational query reformulation (CQR) has become indispensable for improving retrieval in dialogue-based applications. However, existing approaches typically rely on reference passages for optimization, which are impractical to acquire in real-world scenarios. To address this limitation, we introduce a novel reference-free preference optimization framework DualReform that generates pseudo reference passages from commonly-encountered conversational datasets containing only queries and responses. DualReform attains this goal through two key innovations: (1) response-based inference, where responses serve as proxies to infer pseudo reference passages, and (2) response refinement via the dual-role of CQR, where a CQR model refines responses based on the shared objectives between response refinement and CQR. Despite not relying on reference passages, DualReform achieves 96.9--99.1% of the retrieval accuracy attainable only with reference passages and surpasses the state-of-the-art method by up to 31.6%.

@article{kim2025_2505.06552,
  title={ References Indeed Matter? Reference-Free Preference Optimization for Conversational Query Reformulation },
  author={ Doyoung Kim and Youngjun Lee and Joeun Kim and Jihwan Bang and Hwanjun Song and Susik Yoon and Jae-Gil Lee },
  journal={arXiv preprint arXiv:2505.06552},
  year={ 2025 }
}