Interpretable Reinforcement Learning via Differentiable Decision Trees

Decision trees are ubiquitous in machine learning for their ease of use and interpretability; however, they are not typically implemented in reinforcement learning, because they cannot be updated via stochastic gradient descent. Traditional applications of decision trees for reinforcement learning have focused instead on making commitments to decision boundaries as the tree is grown one layer at a time. We overcome this critical limitation by allowing for a gradient update over the entire tree structure that improves sample complexity when a tree is differentiable and interpretability when a tree is discrete. We offer three key contributions towards this goal. First, we motivate the need for policy-gradient-based learning by examining the theoretical properties of gradient descent over differentiable decision trees. Second, we introduce a regularization framework that yields interpretability. Third, we demonstrate the ability to construct an interpretable tree via policy gradient.