In this paper, we introduce Rule-Guided Feedback (RGF), a framework designed to enhance Large Language Model (LLM) performance through structured rule adherence and strategic information seeking. RGF implements a teacher-student paradigm where rule-following is forced through established guidelines. Our framework employs a Teacher model that rigorously evaluates each student output against task-specific rules, providing constructive guidance rather than direct answers when detecting deviations. This iterative feedback loop serves two crucial purposes: maintaining solutions within defined constraints and encouraging proactive information seeking to resolve uncertainties. We evaluate RGF on diverse tasks including Checkmate-in-One puzzles, Sonnet Writing, Penguins-In-a-Table classification, GSM8k, and StrategyQA. Our findings suggest that structured feedback mechanisms can significantly enhance LLMs' performance across various domains.
View on arXiv@article{diallo2025_2503.11336,
title={ Rule-Guided Feedback: Enhancing Reasoning by Enforcing Rule Adherence in Large Language Models },
author={ Aissatou Diallo and Antonis Bikakis and Luke Dickens and Anthony Hunter and Rob Miller },
journal={arXiv preprint arXiv:2503.11336},
year={ 2025 }
}