Furthermore, another insight be that, one word on its own does not have a meaning, in order for a word to have a meaning, there must be some more words (or image).
But in the current state, the embedding based approach in text experiments, keep updating the so called representation of each word, as if it has nothing to do with any other word, and as if each word be isolated on its own, I think this be a big problem, which do not consider that words be connected to each other.