  We present an introductory investigation into continuous-space vector
representations of sentences. We acquire pairs of very similar sentences
differing only by a small alterations (such as change of a noun, adding an
adjective, noun or punctuation) from datasets for natural language inference
using a simple pattern method. We look into how such a small change within the
sentence text affects its representation in the continuous space and how such
alterations are reflected by some of the popular sentence embedding models. We
found that vector differences of some embeddings actually reflect small changes
within a sentence.

Computation and Language