Hierarchical Multiple-Instance Data Classification with Costly Features

We motivate our research with a real-world problem of classifying malicious web domains using a remote service that provides various information. Crucially, some of the information can be further analyzed into a certain depth and this process sequentially creates a tree of hierarchically structured multiple-instance data. Each request sent to the remote service is associated with a cost (e.g., time or another cost per request) and the objective is to maximize the accuracy, constrained with a budget. We present a generic framework able to work with a class of similar problems. Our method is based on Classification with Costly Features (CwCF), Hierarchical Multiple-Instance Learning (HMIL) and hierarchical decomposition of the action space. It works with samples described as partially-observed trees of features of various types (similar to a JSON/XML file), which allows to model data with complex structure. The process is modeled as a Markov Decision Process (MDP), where a state represents acquired features, and actions select yet unknown ones. The policy is trained with deep reinforcement learning and we demonstrate our method with both real-world and synthetic data.