User Profile with Large Language Models: Construction, Updating, and Benchmarking

User profile modeling plays a key role in personalized systems, as it requires building accurate profiles and updating them with new information. In this paper, we present two high-quality open-source user profile datasets: one for profile construction and another for profile updating. These datasets offer a strong basis for evaluating user profile modeling techniques in dynamic settings. We also show a methodology that uses large language models (LLMs) to tackle both profile construction and updating. Our method uses a probabilistic framework to predict user profiles from input text, allowing for precise and context-aware profile generation. Our experiments demonstrate that models like Mistral-7b and Llama2-7b perform strongly in both tasks. LLMs improve the precision and recall of the generated profiles, and high evaluation scores confirm the effectiveness of our approach.

@article{prottasha2025_2502.10660,
  title={ User Profile with Large Language Models: Construction, Updating, and Benchmarking },
  author={ Nusrat Jahan Prottasha and Md Kowsher and Hafijur Raman and Israt Jahan Anny and Prakash Bhat and Ivan Garibay and Ozlem Garibay },
  journal={arXiv preprint arXiv:2502.10660},
  year={ 2025 }
}