Methods for retrofitting representations learned from distributional data to the structure of a knowledge graph typically assume that entities are embedded in a connected space and that relations imply similarity. However, knowledge graphs containing diverse entities and relations often do not accord with these assumptions. To overcome these limitations, we present a framework that generalizes existing retrofitting methods by explicitly modeling pairwise relations. We show that a simple instantiation of this framework with linear relational functions significantly outperforms existing retrofitting methods on complex knowledge graphs and loses no accuracy on graphs where relations do imply similarity. Finally, we demonstrate the utility of this method by predicting new drug--disease treatment pairs in a large, complex health knowledge graph.