Skip Prop: Representing Sentences with One Vector Per Proposition
We introduce the notion of a multi vector sentence representation based on a "one vector per proposition" philosophy, which we term skip prop vectors. By representing each predicate argument structure in a complex sentence as an individual vector, skip prop is (1) a response to empirical evidence that single vector sentence representations degrade with sentence length, and (2) a representation that maintains a semantically useful level of granularity. We demonstrate the feasibility of training skip prop vectors, introducing a method adapted from skip thought vectors, and compare skip prop with "one vector per sentence" and "one vector per token" approaches.
IWCS