Answers.Body: <p>The language model is not <em>only</em> about the appearance or not of the delivery fee. Therefore the sentences should not be expected to be polar opposites. Neither are they orthogonal concepts (with nothing in common) in the space of all possible sentences. Instead, it is quite reasonable that a significant part of the embeddings capture the shared concept of &quot;factual statement about visibility of a delivery fee&quot;, which does not vary between the two sentences and thus you should expect a certain amount of similarity from that.</p>
  <p>There should be a vector <em>linking</em> the two embeddings that is <em>approximately</em> about an observed number of entities. You can extract it by taking one embedding away from the other (always subtracting in the same sense). This vector may only be stable when comparing very similar pairs of sentences, it may not be as global as simpler, stronger concepts, e.g. gender of a single word subject (when using a gendered word like &quot;king&quot; or &quot;queen&quot;), because there are many ways to express the same idea, but the details of what specific words are used will be important.</p>
  <p>You can check for this, and see if the embeddings are capturing useful information for your project by comparing this difference vector with difference vectors between visibility or not of any other entities. For example, you could switch &quot;delivery fee&quot; with &quot;postcode&quot; or any other component on your form, and you should get other pairs of vectors, and those pairs should have a similar difference between them.</p>
  <p>Even if this is not the case, you may not need to find some other embedding system with different properties (and you most likely won't find any generic embedding system which has zero similarity between the two sentences). Instead you should test the utility of the embeddings in other ways. If you want to distinguish between the two sentences presented, you may need to create a model <em>other</em> than measuring vector similarity between embeddings.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-12-08T20:15:27.973Z
Answers.LastActivityDate: 2023-12-09T09:50:11.717Z
Answers.LastEditDate: 2023-12-09T09:50:11.717Z
Answers.UserDisplayName: Neil Slater
Body: <p>I am playing around with the sentence transformers pretrained models to compute embeddings for a sentence and then compare it using the cosine similarity on the two vectors. I was using the <code>sentence-transformers/all-MiniLM-L6-v2</code> model and used the <a href="https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2" rel="nofollow noreferrer">inference API</a> on the model page.</p>
  <p>For the two sentences:</p>
  <ol>
  <li>delivery fee not showing</li>
  <li>delivery fee showing twice</li>
  </ol>
  <p>the similarity score was nearly 0.72 which seems very high for the two sentences that are in fact kind of opposite of each other. I would rather believe that these are orthogonal and would like them to have a score close to 0.</p>
  <p>Can you guys help me with any other model that might give better results? Also, read about Universal Sentence Encoder by Google. Any reviews about that?</p>
  
Tags: <natural-language-processing>
Title: Are sentence-transformers the best available pretrained models for computing embeddings for a sentence?
UserDisplayName: bigbang
ThreadText: # Are sentence-transformers the best available pretrained models for computing embeddings for a sentence?
  
  (Asked by: username_0 on 2023-12-08)
  
  I am playing around with the sentence transformers pretrained models to compute embeddings for a sentence and then compare it using the cosine similarity on the two vectors. I was using the `sentence-transformers/all-MiniLM-L6-v2` model and used the [inference API](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) on the model page.
  
  For the two sentences:
  
  1. delivery fee not showing
  2. delivery fee showing twice
  
  the similarity score was nearly 0.72 which seems very high for the two sentences that are in fact kind of opposite of each other. I would rather believe that these are orthogonal and would like them to have a score close to 0.
  
  Can you guys help me with any other model that might give better results? Also, read about Universal Sentence Encoder by Google. Any reviews about that?
  
  **Tags:** natural-language-processing
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-12-08. Score: 2 | Accepted answer
  
  The language model is not *only* about the appearance or not of the delivery fee. Therefore the sentences should not be expected to be polar opposites. Neither are they orthogonal concepts (with nothing in common) in the space of all possible sentences. Instead, it is quite reasonable that a significant part of the embeddings capture the shared concept of "factual statement about visibility of a delivery fee", which does not vary between the two sentences and thus you should expect a certain amount of similarity from that.
  
  There should be a vector *linking* the two embeddings that is *approximately* about an observed number of entities. You can extract it by taking one embedding away from the other (always subtracting in the same sense). This vector may only be stable when comparing very similar pairs of sentences, it may not be as global as simpler, stronger concepts, e.g. gender of a single word subject (when using a gendered word like "king" or "queen"), because there are many ways to express the same idea, but the details of what specific words are used will be important.
  
  You can check for this, and see if the embeddings are capturing useful information for your project by comparing this difference vector with difference vectors between visibility or not of any other entities. For example, you could switch "delivery fee" with "postcode" or any other component on your form, and you should get other pairs of vectors, and those pairs should have a similar difference between them.
  
  Even if this is not the case, you may not need to find some other embedding system with different properties (and you most likely won't find any generic embedding system which has zero similarity between the two sentences). Instead you should test the utility of the embeddings in other ways. If you want to distinguish between the two sentences presented, you may need to create a model *other* than measuring vector similarity between embeddings.
  
  
  ---
  
  