If I understand you have a pre-trained word embedding model. And you have your own model that you want to train so that it produces representations very close to the one produced by the first one.
Is that it?