Long Short-Term Memory-Networks for Machine Reading

Teaching machines to process text with psycholinguistic insights is a challenging task. We propose an attentive machine reader that reads text from left to right, whilst linking the word at the current fixation point to previous words stored in the memory as a kind of implicit information parsing to facilitate understanding. The reader is equipped with a Long Short-Term Memory architecture, which different from previous work, has a memory tape (instead of a memory cell) to store the past information and adaptively use them without severe information compression. In addition, we propose a new attentional encoder-decoder that explicitly stores inter alignment information in the memory of the decoder, from which we reap significant gains. We demonstrate the excellent performance of the machine reader in language modeling as well as the downstream sentiment analysis and natural language inference.