  While sentence anomalies have been applied periodically for testing in NLP,
we have yet to establish a picture of the precise status of anomaly information
in representations from NLP models. In this paper we aim to fill two primary
gaps, focusing on the domain of syntactic anomalies. First, we explore
fine-grained differences in anomaly encoding by designing probing tasks that
vary the hierarchical level at which anomalies occur in a sentence. Second, we
test not only models' ability to detect a given anomaly, but also the
generality of the detected anomaly signal, by examining transfer between
distinct anomaly types. Results suggest that all models encode some information
supporting anomaly detection, but detection performance varies between
anomalies, and only representations from more recent transformer models show
signs of generalized knowledge of anomalies. Follow-up analyses support the
notion that these models pick up on a legitimate, general notion of sentence
oddity, while coarser-grained word position information is likely also a
contributor to the observed anomaly detection.

Computation and Language