Cost-Efficient Hierarchical Knowledge Extraction with Deep Reinforcement Learning

We extend the framework of Classification with Costly Features to work with structured samples, that can no longer be represented as fixed-length vectors. Instead, the samples can only be represented as trees of features, with a variable and possibly unlimited depth and breadth, similar to a JSON file. We provide a method that, independently for each sample, sequentially selects features from the tree. The newly acquired features can be further expanded, until the algorithm terminates with a classification decision. Each piece of information has a real-valued cost, and the objective is to maximize the classification accuracy while minimizing the total cost of the selected features. The method targets data naturally occurring in many domains, e.g., targeted advertising, medical diagnosis, or malware detection. We demonstrate our deep reinforcement learning based algorithm in seven relational classification datasets.