Have you read this article? There is a small portion that discusses why models that produce sentence embeddings are bad at embedding one or two words into a relevant latent space.

  
      
      Joe Davison Blog – 29 May 20
  
  
    

Zero-Shot Learning in Modern NLP 25

State-of-the-art NLP models for text classification without annotated data