Answers.Body: <p>I think you are confused about why the margin exists.</p>
  <p>The margin exists in contrastive learning because we only want the model to output embeddings where negative samples are far from each other to a <strong>certain extent</strong>.</p>
  <p>That is, we don't want the model to train itself to output embeddings where the distance between negtaive samples diverge to infinity. That would be an overkill and most likely the model will not be able to learn a meaningful representation.</p>
  <p>Like you said here,</p>
  <blockquote>
  <p>Let say, if Y is 1 and distance value is larger, the first part become
  zero (1-Y), and second also become zero, because it should choose
  whether m-d or 0 is bigger. So the loss is zero which does not make
  sense. Can you please help me to understand this</p>
  </blockquote>
  <p>The loss is zero when the distance is larger than m for negative samples. Thus, the model will not be further updated for negative pairs that it is already good at distinguishing.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-04-06T06:06:02.357Z
Answers.Text: thanks, this explanation make my confusion clear.
Answers.UserDisplayName: Talha Anwar
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-04-06T02:40:42.090Z
Answers.LastActivityDate: 2022-04-06T02:40:42.090Z
Answers.UserDisplayName: DKDK
Body: <p>I am having issue in getting clear concept of contrastive loss used in siamese network.</p>
  <p>Here is pytorch formula</p>
  <pre><code>torch.mean((1-label) * torch.pow(euclidean_distance, 2) +
                                        (label) * torch.pow(torch.clamp(margin - euclidean_distance, min=0.0), 2))
  </code></pre>
  <p>where margin=2.</p>
  <p>If we convert this to equation format, it can be written as</p>
  <pre><code>(1-Y)*D^2 + Y* max(m-d,0)^2
  </code></pre>
  <p>Y=0, if both images are from same class
  Y=1, if both images are from different class</p>
  <p>What i think, if images are from same class the distance between embedding should decrease.   and if images are from different class, the distance should increase.</p>
  <p>I am unable to map this concept to contrastive loss.</p>
  <p>Let say, if Y is 1 and distance value is larger, the first part become zero <code>(1-Y)</code>, and second also become zero, because it should choose whether <code>m-d</code> or 0 is bigger.
  So the loss is zero which does not make sense.
  Can you please help me to understand this</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-06T13:11:21.793Z
Comments.Text: You can use mathjax to format the mathematical formulas here. So, maybe you can format the loss function.
Comments.UserDisplayName: nbro
Tags: <loss><siamese-neural-network>
Title: How contrastive loss work intuitively in siamese network
UserDisplayName: Talha Anwar
ThreadText: # How contrastive loss work intuitively in siamese network
  
  (Asked by: username_0 on 2022-04-05)
  
  I am having issue in getting clear concept of contrastive loss used in siamese network.
  
  Here is pytorch formula
  
  ```
  torch.mean((1-label) * torch.pow(euclidean_distance, 2) +
                                        (label) * torch.pow(torch.clamp(margin - euclidean_distance, min=0.0), 2))
  
  ```
  
  where margin=2.
  
  If we convert this to equation format, it can be written as
  
  ```
  (1-Y)*D^2 + Y* max(m-d,0)^2
  
  ```
  
  Y=0, if both images are from same class
  Y=1, if both images are from different class
  
  What i think, if images are from same class the distance between embedding should decrease. and if images are from different class, the distance should increase.
  
  I am unable to map this concept to contrastive loss.
  
  Let say, if Y is 1 and distance value is larger, the first part become zero `(1-Y)`, and second also become zero, because it should choose whether `m-d` or 0 is bigger.
  So the loss is zero which does not make sense.
  Can you please help me to understand this
  
  **Tags:** loss, siamese-neural-network
  
  > **Comments:**
  > **username_1**: You can use mathjax to format the mathematical formulas here. So, maybe you can format the loss function.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-04-06. Score: 4 | Accepted answer
  
  I think you are confused about why the margin exists.
  
  The margin exists in contrastive learning because we only want the model to output embeddings where negative samples are far from each other to a **certain extent**.
  
  That is, we don't want the model to train itself to output embeddings where the distance between negtaive samples diverge to infinity. That would be an overkill and most likely the model will not be able to learn a meaningful representation.
  
  Like you said here,
  
  > Let say, if Y is 1 and distance value is larger, the first part become
  > zero (1-Y), and second also become zero, because it should choose
  > whether m-d or 0 is bigger. So the loss is zero which does not make
  > sense. Can you please help me to understand this
  
  The loss is zero when the distance is larger than m for negative samples. Thus, the model will not be further updated for negative pairs that it is already good at distinguishing.
  
  > **Comments:**
  > **username_0**: thanks, this explanation make my confusion clear.  
  
  ---
  
  