Detecting Local Insights from Global Labels: Supervised & Zero-Shot Sequence Labeling via a Convolutional Decomposition

Zero-shot grammatical error detection is the task of tagging token-level errors in a sentence when only given access to labels at the sentence-level for training. We present and analyze an efficient, parsimonious sequence labeling approach, based on a decomposition of the filter-ngram interactions of a single-layer one-dimensional convolutional neural network as the final layer of a network, that has the characteristic of being effective in both the fully-supervised and zero-shot settings. In the zero-shot setting, with pre-trained contextualized embeddings, the approach is competitive with baseline fully supervised bi-LSTM models (using standard pre-trained word embeddings), despite only having access to sentence-level labels for training. In the fully supervised setting, the approach yields an error detection model as strong as the current state-of-the-art fully supervised approach with feature-based contextualized embeddings. Additionally, we extend these insights from natural language to machine generated language, demonstrating that the strong sequence model can be used to guide synthetic text generation, and that it is concomitantly unsuitable as a reliable detector of synthetic data when the detection model and a sufficiently strong generation model are both accessible.