I suspect another way of looking at it is that these models actually learn about the real world by reading about it in the WSJ -- of course their knowledge of it is not as deep as our own, but good enough for what they do.<p>That is, if you took the well NLP trained model, then you could in principle extract out of it facts like &quot;streets are not found inside cars&quot;.