LSTMs can capture information beyond order

LSTMs have a proven track record in analyzing sequential data. But are they also useful for processing unordered instance sets? Here, we investigate the potential of LSTMs at capturing information be-yond order. We formulate the learning of the underlying structure within a set of instances using LSTM as a Multiple Instance Learning (MIL)problem. In addition, we show that LSTMs are capable of indirectly capturing instance-level information using only set-level annotations. Thus, they can be used to learn instance-level models in a weakly supervised manner. Our empirical evaluation on both simplified (MNIST) and realistic (Lookbook and Histopathology) datasets shows that the proposed method is competitive with or even surpasses state-of-the-art methods specially designed for handling MIL problems. Moreover, we show that its performance on instance-level prediction is close to that of fully-supervised method.