Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach

Tree ensembles such as random forests and boosted trees are renowned for their high prediction performance; however, their interpretability is critically limited. One way of interpreting a complex tree ensemble is to obtain its simplified representation, which is formalized as a model selection problem: Given a complex tree ensemble, we want to obtain the simplest representation that is essentially equivalent to the original one. To this end, we derive a Bayesian model selection algorithm. Our approach has three appealing features: the prediction performance is maintained, the coverage is sufficiently large, and the computation is reasonably feasible. Our synthetic data experiment and real world data applications show that complicated tree ensembles are approximated reasonably as interpretable.