&gt; You can&#x27;t determine that by semantics<p>Actually, &quot;animacy&quot; is a fundamental feature in semantics. It&#x27;s part of your mental lexicon that a cat is animate and a rug isn&#x27;t, and you would simply infer from that which is the referent for &quot;it&quot;. As semantic challenges go, this is a very trivial one. In general the border between linguistic and world knowledge can become blurred. There may be limits to what can be learned purely from text, but seeing that this model achieved 94%, a lot <i>can</i> be learned purely from (annotated) text.