Effective information retrieval (IR) in settings with limited training data, particularly for complex queries, remains a challenging task. This paper introduces IR2, Information Regularization for Information Retrieval, a technique for reducing overfitting during synthetic data generation. This approach, representing a novel application of regularization techniques in synthetic data creation for IR, is tested on three recent IR tasks characterized by complex queries: DORIS-MAE, ArguAna, and WhatsThatBook. Experimental results indicate that our regularization techniques not only outperform previous synthetic query generation methods on the tasks considered but also reduce cost by up to 50%. Furthermore, this paper categorizes and explores three regularization methods at different stages of the query synthesis pipeline-input, prompt, and output-each offering varying degrees of performance improvement compared to models where no regularization is applied. This provides a systematic approach for optimizing synthetic data generation in data-limited, complex-query IR scenarios. All code, prompts and synthetic data are available atthis https URL.
View on arXiv@article{wang2025_2402.16200,
title={ IR2: Information Regularization for Information Retrieval },
author={ Jianyou Wang and Kaicheng Wang and Xiaoyue Wang and Weili Cao and Ramamohan Paturi and Leon Bergen },
journal={arXiv preprint arXiv:2402.16200},
year={ 2025 }
}